PennyPost An email that goes to everyone who stalled will always look like it works: some of those people were coming back anyway. So PennyPost holds back about 1 in 10 people who stall, sends them nothing, and compares.
Who is held back
When a person is about to get a moment email, PennyPost puts them in one of two groups: emailed (about 9 in 10) or held back (about 1 in 10). The choice is random but sticks to the person, so the same person is always in the same group for that email. Held-back people show on their timeline as "Held back: no email".
What "came back" means
A person came back if they opened your results page within 3 days of the moment they would have been emailed. It's measured the same way for both groups.
Reading Results
The Results page asks "Did the emails bring people back?" and shows, for each live email:
- how many emailed people came back, and how many held-back people came back
- how many came back before the email (they got there on their own and were skipped)
- once there's enough data, how many more people per 100 the email brought back
It takes time. A comparison needs at least 2 weeks and at least 10 people who came back, and enough held-back people to tell the groups apart. Until then it says it's too early. Percentages show from 30 people.
When a comparison is done, PennyPost saves the result (the email helped, hurt, or made no clear difference) and starts the next one.
Sharing
Share on Results makes a public link at usepennypost.com/r/... that shows the counts and nothing about any person. Search engines don't index it. Stop sharing turns the link off.