Skip to content
Back to Projects
Active
Data Science
Tools
Open Source
Aug 2026

backtest-audit

Audits a trading backtest from its trade log and returns a verdict: TRUSTWORTHY or NOT TRUSTWORTHY. Built from the measurement bugs that made my own trading bot look profitable.

View Source
8
Checks
43
Tests
Zero
Dependencies
v0.1.0
Release
backtest-audit project visual

The constraint

My trading bot's backtests said +646%, then +53%, then +42.9%. Each was a measurement bug, and each looked like a result. A backtest that flatters you is the one you end up trusting.

The approach

Turn each bug into a mechanical check over the trade log: exits that never fire, positions the strategy cannot see, shorts with no liability booked, sizing off idle cash. The audit returns a verdict, and an unjudged check is never a pass.

Process
  1. 1
    Start from real failures

    Every check began as a bug that had already produced a convincing number, not as a hypothetical.

  2. 2
    Need only the trade log

    The audit reads the log a backtest already writes, so it works on results from code it has never seen.

  3. 3
    Fail closed

    If a check cannot decide, the verdict is NOT TRUSTWORTHY. Silence is not evidence.

Outcomes

  • Reconstructs all three of trading-bot's inflated results from their trade logs alone
  • Runs as a GitHub Action, so a committed result is audited before it is quoted
  • Zero runtime dependencies
  • An unjudged check counts against the verdict, never for it
Details

Extracted four backtest measurement bugs into a zero-dependency audit tool that returns a verdict from a trade log and runs as a CI action.

Related projects