The Subjective Function (Part 3)

I keep coming up with quirky analogies for different aspects of the system development process.  In http://www.optionfanatic.com/2012/09/19/trading-system-1-initial-assessment-part-1/ on 9/19/12, I talked about a Motion to Dismiss.  On 10/8/12 (http://www.optionfanatic.com/2012/10/08/the-subjective-function-part-2), I talked about visiting the eye doctor.  As promised, today I will present some graphs for study to help determine the subjective function.

The following inserts show two graphs each.  On the top is the equity curve itself:  a plot of account value over time beginning with $1M and traded with a $100K position size.  On the bottom is the drawdown (DD) curve.  DD is zero whenever the account is at a new high.  The red line on this graph tracks the max DD ever seen over the whole time period.  You will see the red line dip to levels not seen in the blue graph but that is only because the DD was too short to be seen in the blue graph.  If you were able to zoom in and magnify then you would see the blue graph dip to those levels.

As you look the following three inserts, imagine that you are studying the performance of your investment account over time.  Which would you feel best about?  Note details such as relative net profit (final value is shown in the upper left of the top curves), relative severity of drawdowns (current drawdown at the right edge of the bottom curves is shown in the upper left of the bottom curves), relative time between new equity highs, number of horizontal periods (indicating no current trade), etc.

Now without further ado, let me present…

Insert #1:

Insert #2:

Insert #3:

I will analyze these graphs based on my preferences in the next post.

The Subjective Function (Part 2)

My last post (http://www.optionfanatic.com/2012/10/05/the-subjective-function-part-1) defined the subjective function.  Today I want to present another perspective on the subjective function by explaining what I previously missed.

The gist of my previous posts is that all traders are looking for the same general thing.  “Consistent profitability over time” is a phrase I have written many times over.  Yes, Trader A may be looking for the supercharged exponential returns that carry the risk of heavy drawdowns while Trader B is more conservative and will sacrifice returns to minimize drawdown.  Aren’t they both basically looking for consistent profitability over time, though?

No.

I now see that Trader A and Trader B are looking for decisively different things.  My “consistent profitability over time” is what Trader B seeks.  Trader A, however, is in effect buying a lottery ticket and hoping for the big payout.  If financially savvy then Trader A is playing this with a limited percentage of the total portfolio so as not to go completely bankrupt.  Trader B might be all-in with proper asset allocation.

My previous lack of understanding might reflect an inadequacy of language itself.  I suspect we just don’t have enough terms to reflect the variety of individual preference for investment returns.  Compound annual growth rate (CAGR), drawdown (DD), and flat times translate directly to features seen on equity curves but others do not (e.g. profit factor, Sharpe and Sortino ratios).  Equity curves reflecting the same CAGR or DD can look wildly dissimilar, however.  The pictures tell more than words can describe.

Moving beyond language in this manner is what Dr. Bandy recommends in Quantitative Trading Systems (2007) to determine the subjective function.  After running the trading system optimization, scroll through the different equity curves to find the ones you like best.  Note which metrics score highest on your favorite equity curves and voila!  You have the subjective function.  This is not unlike a visit to the eye doctor where you endure an exhausive series of “which lens is better:  one or two?  1 or 2?  2, or 1?  2, 1?”

In fact, let’s do that:  a visit to the optometrist’s office, and more, when we return.

The Subjective Function (Part 1)

I’ve mentioned multiple times that in trading system development we’re looking for consistent profitability over time.  To do this, identification of the “subjective function” is a prerequisite. 

Much of the direction to my current work is owed to Dr. Howard Bandy.  You can read about him here:  http://www.linkedin.com/pub/howard-bandy/b/b44/270 .  In his book Quantitative Trading Systems (2007), Bandy defines the “objective function” as:

> A single-valued measurement of the fitness of a trading system. Optimization
> techniques are guided by maximizing the value of the objective function.

He also writes:

> Only you can decide which trading systems are good and which are not.
> Selection of the objective function is a management decision and must
> be made prior to extensive model development. The objective function
> itself is not a candidate for optimization.

What does this mean?

System trading is the epitome of Yin and Yang!  Beginning on 5/7/12 (http://www.optionfanatic.com/2012/05/07/strategies-vs-systems-part-i/), I wrote a number of blog posts discussing discretionary vs. systematic trading.  The upshot was that discretionary trading is subjective and systematic trading is objective.  This may be true but at the heart of a somewhat objective system development process is the very personal selection of Bandy’s objective function.   For this reason, I have chosen to call it the subjective function.

All this subjective function talk may seem highly theoretical but it really boils down to something quite simple.  On 10/4/12 (http://www.optionfanatic.com/2012/10/04/trading-system-1-spy-vix-part-4/), study the y-axis to see my subjective function:  net profit (%).  I proceeded to discuss the best combination of parameters by performance on that measure.

The only problem is that I failed to go through the process of determining my subjective function before implementing it.

Stay tuned for further discussion and a possible detour in upcoming posts.

Trading System #1–SPY VIX (Part 2)

In http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I introduced the concept of evolutionary operation and began to talk about the different variables present in this trading system.  Today I want to continue that discussion from the perspective of optimization.

Investopedia.com defines optimization as:

“…the process of adjusting one’s trading system in an attempt to make it more effective. These adjustments include changing the number of periods used in moving averages, changing the number of indicators used, or simply taking away what doesn’t work.”

In the system development process, optimization is going to allow me to look at performance metrics for every combination of specified parameter values.  In the present context, I will allow x to vary from 3-7 in increments of one [e.g. 3 (days), 4, 5, 6, or 7].  I will allow y to vary from 5-25 in increments of five [e.g. 5 (percent), 10, 15, 20, or 25].  I will allow z to vary from 6-15 in increments of one.  The result of this process will be summary statistics for 5 * 5 * 10 = 250 trading systems.

The goal of optimization is not to find out which set of parameter values performed the best in order to trade those going forward.  This is called “curve-fitting” and the reason optimization has gotten a bad name in some circles.  The more variables you have and the greater the number of total parameter value combinations, the more unlikely you are to see similar performance in the future as seen in the most profitable backtest.

Rather, the goal of optimization should be to assess the performance landscape from one combination of parameter values to the next.  Going back again to http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I want to know if these performance graphs have spike regions or plateau regions.  If they have spikes then I will discard them, considering performance to be lucky rather than good.  If they have plateau regions then I want to choose parameter values near the middle of those plateaus (margin of safety).

In my next post I’ll start running the numbers.

Trading System #1–SPY VIX (Part 1)

With the “motion to dismiss” now denied, the next step is to lay the groundwork for developing this strategy into a trading system.

As a review, the trading strategy is based on two claims.  First, five days after the CBOE Volatility Index (VIX) closes 5% or more below its 10-day simple moving average (SMA), the S&P 500 index (SPX) has lost money on average. Second, when the VIX closes 5% or more above its 10-SMA, SPX has outperformed the average week better than 2-1 over the next five trading days.

The current system in development offers three variables to be backtested:  x-bar stop y% extended from its z-SMA.  In this case, we have used x=5 (five-day trades), y=5, and z=10 (VIX closing 5% above/below its 10-SMA).  What we want to see in backtesting is solid results with these values.  We also want to see solid results when we slightly change these values, though.

The process of “evolutionary operation” looks to keep one variable constant while changing another to see its effect on system performance.  Graphically speaking, we want to plot some measure of performance (e.g. net profit) vs. the one variable being changed:

This does not have the appearance of a robust system because slight changes in the variable drastically affect net profit.  Whether a moving average is four periods vs. five or 14 periods vs. 15 should not matter.  If one value appears magically improved in the midst of mediocre results then those backtested profits are likely a matter of luck rather than the system taking advantage of consistent opportunity (and subsequent future profits).

This is what we hope to see.  All values from 6-15 produced solid profit for this trading system.  Rather than the spikes in the previous graph we see a plateau region in this one.  This should be a necessary (but not sufficient) condition for a system we might trade in the future with real capital.

These graphs were adapted from Tomasini and Jaekle (2009).

In my next post I will continue this discussion in terms of optimization.