Trading System #1–Brief Notes on Statistical Interpretation

In addition to generating profit and loss numbers, sophisticated backtesting software will provide a whole slew of statistics each designed to address a different aspect of the performance results.

Without question, one of the most complicated matters in developing a system is how best to evaluate it.  Different aspects will be more or less important to different people.  One person may not care about the average win vs. average loss as long as most trades win.  Another person may not care about how few trades win and how large the drawdowns get as long as net return is maximized over time.  Younger people can afford to lose more because time is on their side.  Retirees may be risk-averse because what they lose they don’t have so much opportunity to regain.

From a graphical point of view, many backtesting statistics describe the equity curve itself.  Supposing a given initial value ($1M in our case), the equity curve shows the value of the account over time.  A curve sloping up (down) is making (losing) money.  Drawdowns are the difference (in dollars or percent) from an equity value peak to valley.  Flat times describe time intervals between new equity highs.  Most people prefer shallow drawdowns and shorter flat times to sleep easier at night.  In theory, this may be achieved at the expense of total profit and exponential (sloping up more and more over time) equity curves.  A lower total profit often means a safer system, however, that has a lower Risk of Ruin:  probability of the account going to zero (“bust”) from a string of losses or just a couple large losses.

Like trading itself, much of system development is about tradeoffs.