Research standard
Make every pass harder to fool.
This is the publication loop behind the research notes on this site. A polished argument can still be wrong; presentation quality does not get to launder weak evidence.
Five gates
1. Provenance
Every consequential factual claim should trace to a primary artifact where practical: source code, raw measurements, specifications, filings, papers, benchmark prompts, traces, or experiment outputs. Secondary sources provide context, not the foundation.
2. Quantitative validity
A number should expose its denominator, units, comparison set, measurement window, assumptions, and uncertainty or sensitivity when material. A single seed is not a variance estimate. A target is not a measurement.
3. Adversarial thesis
Write the strongest rival explanation as if its author will review the piece. If it explains the evidence equally well, narrow the conclusion instead of writing around the ambiguity.
4. Citation integrity
A source must support the sentence adjacent to it. Facts, derivations, estimates, and engineering targets are different evidence classes and should remain visibly different.
5. Reproducibility
Link code, data, queries, configs, benchmark prompts, calculations, hardware logs, or exact commands where practical. If reproduction is impossible, state the missing dependency rather than implying it exists.
Evidence classes
| Class | Meaning |
|---|---|
| Observed | Measured directly in an experiment, system, trace, filing, source artifact, or dataset. |
| Derived | Calculated from observed inputs with an explicit method. |
| Estimated | Model-dependent or inferred; assumptions must be exposed. |
| Target | A design requirement, future gate, or intended performance level. |
The loop
- Choose a decision question, not a topic.
- Harvest primary artifacts before drafting prose.
- Lock one falsifiable claim.
- Build an evidence ledger.
- Run quantitative/sensitivity checks the claim depends on.
- Draft results before rhetoric.
- Red-team the thesis with the strongest rival explanation.
- Add visuals only when they encode evidence or mechanism.
- Audit every citation against the exact adjacent claim.
- Compress the prose until the headline is no stronger than the body.
- Publish with limitations and an update condition.
- Reopen the piece when new evidence arrives.
Failure routes backward
If provenance is weak, return to sourcing. If a sensitivity check changes the conclusion, return to the claim. If the rival explanation survives, narrow the thesis. If the required experiment is paid or unavailable, either find a zero-cost substitute, publish a bounded note, or park the stronger claim.
The most important rule is the last one:
The conclusion is allowed to get weaker.