ResearchSetup
Hypothesis is written after seeing results
Typical symptoms
- The research note reads like the result was known in advance
- Rejection criteria are missing
- Several metrics are cited but no primary metric is named
Likely causes
- Exploration and confirmation were mixed
- No pre-analysis note existed
- The team optimized the story after seeing outcomes
Checks
- Find the timestamped hypothesis note
- Identify the primary metric
- Ask what result would have rejected the idea
Resolution path
- Create a method brief before the next test
- Separate exploratory observations from confirmatory tests
- Label retrofitted conclusions as exploratory only
Related terms: hypothesis, p-hacking, pre-analysis, confirmation bias