General Theories on System Development (Part 2)
Posted by Mark on January 30, 2020 at 10:01 | Last modified: May 1, 2020 13:34Today I present the conclusion to a two-part series written on December 5, 2012, where I discuss another issue for debate regarding the philosophy of trading system development.
In my last post, I discussed one of two general approaches to system development where I test multiple trading rules on just one ticker. The second approach flips the first on its head: backtest one trading rule on multiple tickers in search of the ticker(s) that generates widespread and consistent profit.
The statistical caveat I had regarding the first approach also applies here. If I test enough tickers on any given trading rule, then some tickers will show significant profits just by chance alone (e.g. one in 100 at the 0.01 level of significance). In case of the latter, profitable backtesting results are unlikely to be realized in live trading.
Caveat aside, I find this second approach persuasive because of this:
> Why should long-only trades outperform for S&P 500 and Nasdaq stocks but not
> small caps? I’m sure imaginative types could come up with potential explanations
> but it makes me skeptical about the pattern since they’re all broad-based indices.
This implies a common human psychology underlying all trading behavior. If this is true, then consistency across broad-based stock indices should follow. At best, this consequence seems less likely than to say different stocks have their own personalities for finite periods of time (see fourth paragraph). At worst, the consequence seems downright preposterous.
Today in 2020, I still see logical reason to support both approaches.
For the sake of trading system development, the second approach is a higher hurdle to clear because it requires a strategy to perform well on multiple markets. I think the second approach also begs the question how often and for how long do viable strategies work well for multiple markets and then stop working for some? This seems to be getting meta-meta-complicated compared to “for how long do viable strategies proceed to work?”
The gestalt of everything I have seen, read, and traded over the last 12 years leads me to favor the first approach. I would feel very comfortable with a strategy that works on one ticker but not others inside or outside the same asset class were it able to pass either the walk-forward (Part 1 through Part 4) or data-mining approach to system development.
If I had to grab for some supporting evidence in a pinch, then it would probably be correlation. Commodity trading advisors commonly seek to trade a diversified basket of futures markets to compile a low-to-slightly-negative overall correlation. To think a single strategy should work on these relatively uncorrelated components seems almost like a contradiction in terms.
These are two interesting approaches/theories, tough to sort through, and very much subject to personal preference.