QUANTUM AI • GUIDES & RESEARCH

Testing and evidence

Questions to ask before treating a performance claim as meaningful evidence.

Define what was measured

Ask whether results come from live transactions, a demonstration account, a backtest or a hypothetical illustration. Record the dates, instruments, costs and starting assumptions. A screenshot without this context is not a reproducible test.

Look for selection effects

A strategy selected after many trials may look unusually strong on the data used to choose it. Ask how many alternatives were tested and whether a separate period was reserved for evaluation. A single favourable interval does not establish how a method behaves in different conditions.

Include practical constraints

Examine whether the analysis includes transaction costs, financing, delays and unavailable prices. Ask how incomplete data and unsuccessful runs were treated. Compare a strategy with a clearly defined baseline rather than a vague claim of beating the market.

Read the losses as carefully as the gains

Look for drawdowns, concentration, time spent exposed and sensitivity to changing assumptions. Understand the difference between a statistical result and a promise about your own account. Historical results cannot establish guaranteed future profits.

Use the risk checklist alongside the evidence review.