The easiest rules to change in a backtest are the ones you never wrote down. A promising curve arrives, a weak point becomes “context”, and the test quietly stops answering the question that started it.
Backtest acceptance criteria are the conditions a strategy must meet before its results are inspected. They turn “does this idea have enough evidence to keep studying?” into a question with a fixed answer, rather than a negotiation with an attractive chart.
Define the claim before the curve
The first job is not to choose the prettiest metric. It is to write down what the strategy claims to do, where it is allowed to do it, and what would count as evidence against it.
For a prop-style test, the claim usually has two parts. One is economic: after the stated execution assumptions, does the historical sample show a positive and coherent result? The other is operational: does the path remain intelligible when losses cluster, positions overlap, or the account sits through a quiet patch?
Neither part is a forecast. Acceptance criteria do not turn a backtest into a live record. They merely make it harder to mistake a fitted historical story for evidence.
| Before running the test, record | Why it belongs in the record |
|---|---|
| The trading hypothesis | Prevents an indicator combination from becoming the whole explanation |
| Instruments, session, and holding rules | Makes the intended opportunity set visible |
| Data window and untouched validation segment | Separates research from the evidence used to check it |
| Costs and fill assumptions | Stops a frictionless result from becoming the baseline |
| Failure conditions | Gives a poor result permission to be poor without being redesigned |
A pass, a fail, and an investigation are different outcomes
An acceptance sheet works best when it names more than one outcome. “Pass” can mean the result earned further testing. “Fail” can mean the central premise did not survive the stated assumptions. “Investigate” can mean the result is internally inconsistent and the data, code, or mechanism needs checking before any conclusion.
That third category matters. A suspiciously smooth curve, a sharp improvement after a small code change, or a result dominated by one period is not proof of an edge or proof of a bug. It is a reason to inspect the path.
This keeps the researcher from making opposite mistakes with the same evidence: throwing away an idea because one result is awkward, or rescuing it by changing every rule that made it awkward. The report becomes more useful when it preserves the original test and records the next question separately.
Write rejection rules in ordinary language
A criterion does not need false precision to be useful. It can be stated as a condition that a skeptical reader can recognise.
| Area | A usable acceptance question |
|---|---|
| Mechanism | Can the result still be explained by the stated market behaviour? |
| Costs | Does the claim remain coherent after the same spread, commission, swap, and slippage assumptions? |
| Validation | Does untouched data tell a compatible, if less flattering, story? |
| Path | Are drawdowns, trade overlap, and losing runs compatible with the account context? |
| Robustness | Does a small, justified change break the premise completely? |
The point is not to manufacture a universal scorecard. Different approaches earn differently. A trend approach can depend on extended moves; a short-horizon approach can be unusually sensitive to execution. The honest standard is that the criterion follows the mechanism and exists before the result asks to be excused.
Why this matters more under prop constraints
A prop account does not experience a final return in one piece. It experiences the sequence: flat periods, overlapping exposure, losses, and any recovery that follows. A backtest can look acceptable at the finish while its route carries risks that its own account rules make difficult to tolerate.
That is why acceptance criteria belong beside portfolio heat and profit concentration. One examines loss already committed across open positions; the other asks whether a small part of history supplied most of the gain. Neither tells anyone what to trade. Both make the historical path harder to romanticise.
The exact conditions of a prop firm belong to that firm and can change. The durable research question does not: did the test examine the constraints that can end the account before the profitable part of history arrives?
Keep the record when the idea changes
The most damaging revision is invisible. A parameter, a session filter, or a cost assumption changes; the old equity curve disappears; the new version inherits its confidence without inheriting its failed history.
Keep a dated record of the original hypothesis, assumptions, output, and decision. If the idea is modified, call it a new test. That is not bureaucracy. It is the only way to see whether research is becoming more disciplined or merely more accommodating.
realbacktesting is a trading-software studio for cTrader that publishes cBot backtests readers can reproduce, rather than asking them to accept a chart on trust. Its methodology describes the execution and validation approach, while its funding page explains the prop-account context. Those pages describe a backtest, not a performance promise.
Frequently asked
Are acceptance criteria the same as optimisation rules?
No. Optimisation rules describe how a strategy is searched or tuned. Acceptance criteria describe what evidence must be present before the resulting test is treated as worth further work.
Can a strategy pass acceptance criteria and still fail live?
Yes. Historical testing cannot remove uncertainty about future prices, execution, or behaviour under conditions not represented in the sample. Criteria make the test more falsifiable; they do not make it a guarantee.
When should criteria change?
Criteria can change when the hypothesis changes. The clean practice is to preserve the earlier test and label the revised hypothesis as a new test, rather than rewriting history around the preferred result.
The stubborn takeaway: if a backtest is allowed to change the rules after it wins, the rules were never testing it.