emailrules.today
Stop 8 of 8DeepThe next morning

Holdout group

People intentionally excluded from email so you can measure whether email caused the revenue.

Say it out loud

“A group we deliberately do not email, so we can prove the revenue came from the email and not from the customer.”

A holdout is a control group excluded from a campaign or a channel so their behaviour can be compared to those who received it. Without one, “email revenue” is last-click storytelling: it counts people who would have bought anyway and attributes them to whichever message they touched last.

The arithmeticThe number that survives a finance reviewExample numbers
SIZE REVENUE PER HEAD
Received email 90,000 £412,000 £4.58
Holdout 10,000 £38,400 £3.84

They bought anyway. This is the part last-click attribution silently claims as email's.

Attributed by platform £412,000
Actually caused by email £66,600

(4.58 − 3.84) × 90,000. Six times smaller, and the only figure that is true.

Yours

Nobody does this for you.

What goes wrong

Running the holdout for one campaign and declaring a result. Incremental effects are small relative to weekly noise, so a single send is usually measuring the weather. Hold the group out for a period, not a message.

The dated rule behind this

A definition is not a citation. These are the pages with the primary source, the date it was published, and what to do about it.

See also

Where this sits

Stop 8, it comes back as numbers. The reactions become a dashboard, and the dashboard is lying to you a little.

← BeforeSunset / sunsettingNext →rua (DMARC reports)

FAQ

Awkward questions

Who is this actually for?

People who ship email and are too busy to re-read every PDF: week-one marketers, lifecycle/CRM, deliverability, multi-country ops, DTC brands, agencies — on Klaviyo, Mailchimp, Braze, HubSpot, SFMC, or something else. Pick your tool and geos so product-specific pages only appear when they match. EU and UK are first-class filters. Not for people who sell tools about email and need a fake score to demo.

Why only ~40 rules? Isn’t that… thin?

On purpose. A wall of 400 undated “best practices” is how you look busy and still get burned. Every page needs a primary source we actually read. Thin and checkable beats fat and folklore. Europe, bulk inbox rules, measurement honesty, and protocol topics like BIMI/DMARC policy are on the shelf when we can cite them — not every Member State and not every vendor myth. Coverage map lists what we still refuse to invent.

Is this legal advice?

No. If it were, you’d have a billable hour and a longer PDF. This is a dated operator reference written by an email geek. Confirm anything high-stakes with counsel who knows your facts. Same for “will this make me compliant?” — no tool does that. Lawyers and judges do. We say what the sources say and what to do first on Monday.

Why no trust score out of 100?

Because we refuse to invent a number you can’t audit. Fake scores sell seed tests and panic. You get findings, dates, and links. If that feels less exciting than a red dial, good — you’re not the red-dial customer.

Why should I trust you more than my ESP’s blog?

ESPs sell seats. Seed-score vendors sell fear. AI invents citations when nobody watches. We sell nothing today — no pixels, no placement scores, no affiliate — so we can say when a tool is the problem. A human verifies, dates, and corrects in public. Tools may help draft; they do not ship unsourced claims. Check the primary links. Fail that test and leave.

Do I need an account?

No. Filters live in this browser and the URL. Share the link. That’s it. Accounts come later only if they earn it — not so we can nurture you about email.

I’m an agency. Where’s multi-client mode?

We tried a client-name CRM on the setup card. It made the product feel like work before it felt useful. Role filters + copy link + one-page brief (optional PDF title) is enough for now. Complexity comes back when the free shelf is habit, not before.

Is the quiet changelog a bug?

No. Quiet means nothing material moved. We don’t invent urgency so the homepage looks “alive.” Sticky risks still show what usually needs a person when the market is still.