Gauntlet: the GO / NO-GO / CONDITIONAL gatekeeper of a backtest
The Gauntlet block appears under every automatic backtest result. It reruns the whole parameter grid on the full period, computes PSR, DSR, a permutation p-value, PBO and a drawdown distribution, then applies eight criteria to answer "is this edge real?" with GO, NO-GO or CONDITIONAL.
Senzoukria · Documentation · Updated September 2026
Where to find it
- Where
- Replay → Automatic backtest → result → "Gauntlet — is this edge real?"
- Verdicts
- GO (all criteria pass), NO-GO (any eliminatory criterion fails), CONDITIONAL (only advisory criteria fail)
- Extra cost
- One full-period backtest per grid configuration, announced as "gauntlet · full-period grid", then "gauntlet · permutations"
- Session day
- Futures day rolling at 17:00 America/Chicago, the same as the prop firm engine
What it does
After a run or a walk-forward, runGauntlet takes the chosen trade path (the concatenated out-of-sample trades of the walk-forward, or the trades of a simple run) and the grid the sweep tried. It reruns every configuration on all bars to build a session-day × configuration P&L matrix, then judges three series: daily P&L of the chosen path for PSR and DSR, per-trade P&L for the permutation test and drawdown distribution, and the matrix for PBO.
The verdict itself is a pure function (buildVerdict). Each criterion records its value, its threshold, whether it is eliminatory, and a readable detail, so the block can say why, criterion by criterion, and not only "NO-GO". Criteria that need an input the software does not have are marked "not evaluated" and count neither for nor against.
The block shows PSR, DSR (green above 0.95), p-value (green below 0.05), PBO (green below 0.5, "—" with a single configuration), configurations tried, and "Drawdown, 95 % of orderings do better". Below the criteria, the Gauntlet charts add R-multiples, in-sample vs out-of-sample curves, a rolling Sharpe over 20 sessions, adverse and favourable excursion scatter plots, underwater stretches and parameter stability.
Criteria
| Criterion | Threshold | Weight |
|---|---|---|
| Account survives the whole run | true — never touches the drawdown | Eliminatory; not evaluated without a saved rule set |
| Walk-forward: enough windows, most profitable out of sample | ≥ 3 windows and ≥ 70 % of OOS windows profitable | Advisory; not evaluated on a simple run |
| Permutation test: edge distinguishable from chance | p-value < 0.05 | Eliminatory |
| Deflated Sharpe: survives the number of trials | DSR > 0.95 | Eliminatory |
| Probability of backtest overfitting | PBO < 0.5 | Advisory; not evaluated with one configuration |
| Max drawdown within the account's limit | |max DD| < evaluation drawdown of the chosen rule set | Advisory; not evaluated without a limit |
| Survives the red periods | true | Eliminatory when evaluated; currently not evaluated |
| Prop-firm cycle: lock reached, inactivity respected | survives, lock reached, no time-limit failure | Advisory; evaluated once a rule set is chosen |
How to use it
Run a walk-forward with a sweep rather than a single backtest: without a grid, N = 1, the DSR collapses to a PSR against zero, PBO cannot be computed and parameter stability has nothing to compare. The verdict is then honest but thin.
Pick a saved prop firm rule set in the "Should I buy this evaluation?" card that sits next to the Gauntlet. The verdict is replayed immediately with that rule set's drawdown limit, account survival and prop cycle, without launching another backtest. The next steps under a GO are fixed: compare the P&L with a second engine on the same data, two weeks of simulation before any funded account, and a written cut-off plan before arming the autopilot.
Limits or pitfalls
- The permutation and drawdown estimates use 10,000 sign-flip or order shuffles with a fixed seed; p-values are estimates with a standard error near 0.005.
- PBO needs at least 8 session days in the matrix and at least two configurations; the split count defaults to 16 and is reduced to the number of days when shorter.
- A GO is a statement about the sample and the grid you gave it. It says nothing about live execution, data costs or a broker's fills, and it is not a promise of results.
Related pages
- Overfitting tests
- Parameter sweep
- Should I buy this evaluation?
- Futures backtesting guide
- Quant trading for beginners
This page in other languages
Frequently asked questions
- Why does my result say CONDITIONAL rather than GO?
- No eliminatory criterion failed, but at least one advisory criterion did: fewer than three walk-forward windows, under 70 % profitable windows, PBO at or above 0.5, or a drawdown beyond the chosen account's limit. Each failed advisory criterion is listed as a caveat with its measured value.
- Which configuration does the Gauntlet judge for stability?
- The one the walk-forward chose most often across folds, with the last fold breaking ties; without a walk-forward it is the base configuration. The stability chart compares it with neighbours one step away on one axis and labels the surface plateau, slope or peak.
- Does the Gauntlet need a prop firm rule set?
- No. Without one, account survival, max drawdown and prop cycle are shown as "not evaluated" and do not influence the verdict. Adding a rule set makes them count.