Sample size (too few trades)

Sample size, in strategy evaluation, is the number of independent trades a result rests on; with too few trades, averages such as expectancy or win rate are exact descriptions of what happened but say almost nothing about what the rule tends to do, because a few outcomes dominate the total.

Senzoukria · Glossary · Updated September 2026

Why a small sample misleads

Trade outcomes vary a lot from one trade to the next. The average of ten trades can move by a large fraction if one trade is removed, so the average is describing those ten trades rather than the procedure that produced them. As the number of trades grows, the average settles toward the procedure's expectancy, at a rate that depends on how dispersed the outcomes are. High dispersion needs more trades to reach the same confidence.

The same logic applies to win rate, average win and drawdown. A drawdown figure from a short run is the worst stretch seen so far, not the worst the rule can produce.

Measures that account for the count

  • SQN, Van Tharp's system quality number: the square root of the number of trades times the mean outcome divided by its standard deviation. It rises with the sample size for the same mean and dispersion, which is the point.
  • Deflated Sharpe ratio: penalises a Sharpe figure for the number of configurations tried, which is a related problem, selection rather than sampling.
  • Rolling expectancy over the last n trades: shows where a whole-run average was earned and where it turned, so that a positive total is not read as uniform.
  • Permutation and reordering tests: use the existing trades to see how different a curve could look with the same outcomes in another order. They are sensitivity checks, not forecasts or calibrated probabilities.

Independence matters as much as the count

Fifty trades taken in one afternoon under one regime are not fifty independent observations of the rule. Trades that share a session, a news event or an overlapping holding period are correlated, and the effective sample is smaller than the count. Spreading the evaluation across sessions and periods, as a chronological walk-forward does, gives the count more meaning.

In Senzoukria

The previous-sessions view of the backtest panel, headed "Does this repeat?", reports the number of trades with every verdict and refuses to over-read small ones: with too few trades it states that no ratio below means anything and that the numbers should be read as exact, not as significant; when the result is too good for its sample size it warns that a handful of trades is carrying the result and points to the distribution. The SQN line notes that it accounts for the number of trades, and the Auto backtest objectives include SQN for that reason. The journal page of the site shows the same count for recorded trades; its demo dataset, 20 trades over three weeks, is itself an example of a small sample.

Common mistakes

  • Comparing two strategies on their expectancy without comparing their trade counts.
  • Shortening the test period to the part that worked and reporting the ratios from it.
  • Counting scaled entries or partial exits as separate independent trades.
  • Increasing size after a short profitable run, which multiplies exposure to a result that has not been established.

This page in other languages

Frequently asked questions

Is there a minimum number of trades for a valid backtest?
No single threshold applies to every rule, because the number needed depends on how variable the outcomes are and how large the expectancy is relative to that variability. A rule with small, consistent outcomes needs fewer trades than one whose result depends on rare large wins. Report the count alongside every ratio and let the reader judge.
Why does the desktop show a warning when results look too good?
Because on a small sample a very high quality figure usually means a few trades dominate the total. The verdict flags that combination of few trades and a high SQN and directs attention to the distribution of outcomes, where the dependence on one or two trades becomes visible.

Keep reading

Strategy researchTest a trading strategy, from observation to challenge rulesDefine an observation, question it with the copilot, write a rule and explore costs, drawdown and fictional challenge limits in an interactive example.Quant · Start hereQuant trading for beginners: from idea to backtestDefine a testable rule, choose suitable data and evaluate later periods after costs.Order flow · HubOrder flow trading: the complete guideThe hub. What order flow is, what it shows — aggression, delta, imbalance, absorption, liquidity — and the tools that read it, each linked to a full guide.