Secure Email Gateway for Google Workspace

Gmail’s built-in filtering is good, and for many teams it’s enough. But Google’s advanced email security, the stronger phishing, malware, and sandboxing features, lives in the higher Workspace tiers, so getting it means upgrading every user. Putting a secure email gateway in front of Google Workspace gives you that extra protection as an independent, flat per-server layer instead. This guide covers how that works with Hermes Secure Email Gateway, a self-hosted, open-source gateway.

Why put a gateway in front of Google Workspace?

As with Microsoft 365, the honest headline reason is cost, then independence and control.

  • Avoid upgrading every seat for security. Google’s strongest protections. Security Sandbox (attachment detonation), advanced phishing and malware controls, are concentrated in the Enterprise tiers. Fronting Workspace with a gateway lets you keep your users on a cheaper plan and add protection at a flat per-server cost, with Hermes Pro‘s Link Guard providing time-of-click link protection. (Straight talk: Hermes does anti-malware with ClamAV and managed feeds, robust, but not a 1:1 replacement for Google’s cloud detonation Sandbox.)
  • Defense-in-depth. A separate filtering engine catches what Gmail’s baseline misses. Two different engines beat one.
  • Vendor independence. Your security policies, logs, and quarantine aren’t tied to your mailbox provider. They survive a move off Workspace.
  • Data ownership & control. Filtering, logs, and quarantine stay on hardware you control, with full DKIM/DMARC/ARC tooling built in.
  • Encryption that stays in the mailbox. Hermes can automatically encrypt outbound and decrypt inbound mail at the gateway (S/MIME, PGP, TLS, encrypted-PDF), so messages arrive as normal, searchable mail in the user’s mailbox on your infrastructure. Google’s hosted S/MIME is limited to higher editions (Enterprise Plus, Education, Frontline Plus, not Business or Enterprise Standard), and its Confidential mode delivers external recipients a link to a hosted viewer rather than normal mail. See the S/MIME & PGP encryption guide.
The economics. Advanced Workspace security scales per user, per month; a self-hosted gateway is a flat per-server cost that protects thousands of mailboxes without changing price as you hire. See Hermes pricing · how it compares to Proofpoint & Mimecast.

How the architecture works

Inbound: Internet → MX → Hermes (spam, malware, policy, link rewriting) → relay to Google Workspace → mailbox.
Outbound: Google Workspace → outbound gateway → Hermes (DKIM signing, disclaimers, encryption) → Internet.

Inbound: filter before Gmail

Point your domain’s MX records at Hermes, add the domain as a relay domain in Hermes, and set delivery to Google’s mail servers (aspmx.l.google.com and its backups). Then, in the Google Admin console under Apps → Google Workspace → Gmail → Spam, phishing, and malware, configure the Inbound gateway with your Hermes IP address. This is the Google analog of M365’s connector-and-enhanced-filtering: it tells Gmail that legitimate mail arrives via your gateway, so Gmail evaluates SPF and spam signals against the original sender rather than treating your gateway as the source. Use the gateway settings to require TLS and to reject mail that doesn’t arrive through it, so attackers can’t deliver straight to Google and bypass filtering.

Outbound: route Google Workspace through Hermes

In the Admin console under Apps → Gmail → Routing, set an outbound gateway (or routing rule) that sends outgoing mail to Hermes as a smart host. Hermes then applies DKIM signing, disclaimers or signatures (Pro), and any encryption policy before delivering to the internet.

Getting email authentication right

This is where front-of-Workspace deployments most often break, so it’s worth being deliberate.

  • SPF. Since outbound now leaves from Hermes, authorize the gateway’s IP in your SPF record (and you can drop include:_spf.google.com if mail no longer egresses Google directly). Stay within the 10-lookup limit.
  • DKIM. Decide where signing happens. You can keep Google’s DKIM or sign at Hermes; either way publish the matching selector._domainkey public key.
  • DMARC. With SPF and DKIM aligned, publish a DMARC policy, start at p=none to monitor, then tighten to quarantine or reject.
The ARC gotcha, don’t skip this. When Hermes filters inbound mail it legitimately modifies messages (adds headers, and with Link Guard, rewrites links), which can break the original sender’s SPF/DKIM by the time Gmail evaluates it, so legitimate mail can land in spam. ARC (Authenticated Received Chain) fixes it: Hermes seals the original authentication results so the receiver can trust them despite the gateway hop. One key difference from Microsoft 365: Google Workspace has no “trusted ARC sealer” admin setting. Gmail evaluates ARC automatically. What you configure instead is the Inbound gateway (Admin console → Apps → Google Workspace → Gmail → Spam, Phishing and Malware → Inbound gateway): list your Hermes IP(s), optionally reject mail not from the gateway, and require TLS. That tells Gmail to trust your gateway and evaluate authentication against the original sender. So the whole fix is: enable ARC sealing on Hermes, and register it as your inbound gateway, there’s no sealer-trust list to manage.

What you’ll need

  • A server or VPS to run Hermes (one Docker Compose stack, see installation).
  • Control of your domain’s DNS (MX, SPF, DKIM, DMARC records).
  • Google Workspace super-admin access (to configure the inbound gateway and outbound routing).

Step by step

  1. Stand up Hermes and complete first-run setup. → Installation
  2. Add your domain as a relay domain and set delivery to Google’s mail servers. → Email Relay
  3. Configure Google’s Inbound gateway with your Hermes IP and require mail to arrive through it.
  4. Set an outbound gateway / routing rule in Workspace to smart-host outgoing mail through Hermes.
  5. Set up authentication: SPF for the gateway, DKIM signing, a DMARC record, and ARC sealing. → SPF Settings
  6. Repoint MX to Hermes once you’ve tested the flow (see cutover note).

The exact, screen-by-screen Hermes configuration lives in the Administrator Guide. This page is the architecture and the decisions; the docs are the field-by-field reference.

Common pitfalls

  • Not setting the inbound gateway. Skip it and Gmail evaluates spam against your gateway’s IP, not the real sender, wrecking accuracy.
  • Leaving Workspace open. If mail can still arrive directly at Google, attackers bypass your filtering.
  • Stale SPF / skipped ARC. The two most common causes of your own legitimate mail failing authentication after the change.
  • Hard cutover. Flipping MX before testing, with a high DNS TTL, leaves no fast rollback.

A note on cutover

Email is production-critical. Stand Hermes up in parallel, test inbound and outbound flow with a pilot domain or a few mailboxes, confirm spam/malware scoring and SPF/DKIM/DMARC/ARC alignment, then cut over MX. Lower your DNS TTL ahead of time so you can roll back quickly.

Try it

The Hermes Community Edition is free and open source. Pro adds time-of-click Link Guard and more, with a free trial, no credit card required.

Start a free trial Microsoft 365 guide