Sanity-Check a Statistical Result Without a PhD
Get a plain-English read on whether a stat — a p-value, a correlation, an A/B test result — actually means what someone is claiming.
When to use this
When someone shows you a chart and a claim and you want to know if you should believe it.
The prompt
You are a statistician with a plain-English habit.
- **The result I'm looking at**: [the number, the chart, the claim]
- **Where it comes from**: [an A/B test / observational data / a paper / a vendor report]
- **What's being concluded**: [the claim someone is making from this number]
- **Sample size and structure**: [if known — n, time period, who was included]
Do this:
1. **Translate the stat** — what does this number actually MEAN in plain English? Specifically: what would have to be true for this number to be a fluke?
2. **Effect size vs. statistical significance** — what's the practical size of the effect, separate from whether it's "significant"? Many significant effects are tiny.
3. **Confounds and alternative explanations** — what else could produce this pattern? Selection bias, time effects, multiple comparisons, instrumentation changes?
4. **The claim, re-stated honestly** — rewrite the claim so it's defensible from this data.
5. **What would change my mind** — what additional evidence would make this conclusion stronger or weaker?
Don't accept the claim at face value. Don't dismiss it cynically either.
What you'll get back
A plain-English translation of the stat, an effect-size vs. significance reading, alternative explanations, an honest rewrite of the claim, and what evidence would tighten it.
How this is structured in English
Notice the English patterns this prompt uses — they're worth borrowing for your own requests.
- Plain-English habit Compresses a writing discipline into a noun. A 'plain-English habit' is the consistent practice of choosing clear words — useful framing.