An experiment that ran correctly can still support the wrong conclusion.
LabGuard takes a research claim, breaks it into things that can be measured, looks for the loopholes that would explain the result away, and runs the smallest experiments that could settle them. While those run, it watches every one for failures and repairs what it safely can.
Run the bundled scenario
A synthetic violence-detection benchmark with real, reproducible weaknesses built into it.
What is deliberately wrong with it
Each of these is a genuine property of the data and configuration, not a scripted output.
- The benchmark is 8% positive, so accuracy is almost uninformative.
- Model B trained for 90 epochs against Model A's 25.
- Both checkpoints were chosen on the test split.
- The result comes from one favourable seed.
- Model B's reported run overfits well before epoch 90.
- A submitted variant diverges to NaN at its learning rate.
- The reported checkpoint fails its integrity check every time.