The Subjective Function (Part 5)

Yesterday in http://www.optionfanatic.com/2012/10/10/the-subjective-function-part-4, I began the analysis of graphs posted in http://www.optionfanatic.com/2012/10/09/the-subjective-function-part-3 to determine the subjective function.  Today I will make my remaining observations.

Equity curve #3 posts the greatest profit of all three curves.  This profit of 20.3% is roughly 283% greater than equity curve #2 and about 78% greater than equity curve #3.  This relates to my previous observation of also being in drawdown (DD) more than the other two.  If curve #3 is in DD more than the others but is drawing down from a much greater net profit then I may still be happier with this system than with the others.

With regard to maximum DD (MDD), insert #2 has the edge.  MDD here is -1.8%, which is 25% less than insert #1 (-2.4%) and 18% less than insert #3 (-2.2%).  Lower DD means less risk.  I could increase position size a bit to make for comparable MDDs and a larger profit for insert #2.  While this suggests I could increase position size slightly, though, it certainly would not make up the 283% greater profit equity curve #3 has over #2.

Do not forget that in order to secure the 283% profit, insert #3 has over 10 times as many trades as insert #2 and is therefore in the market over 10 times longer.  Theoretically, this is greater risk that could turn sour on us in the future even though it did not significantly do so in 19.5+ years of backtesting.

Time to unmask the systems:

–Insert #1 (y = 25, z = 10) generated the highest profit factor of all 50 systems.
–Insert #1 generated the highest risk adjusted return (RAR).
–Insert #1 generated the highest RAR:MDD ratio.
–Insert #2 (y = 25, z = 6) is the system that generated the lowest MDD.
–Insert #3 (y = 10, z = 6) is the system that generated the highest net profit.

So which is better?

The Subjective Function (Part 4)

I’m in the process of trying to determine the subjective function, as defined in http://www.optionfanatic.com/2012/10/05/the-subjective-function-part-1.  In http://www.optionfanatic.com/2012/10/09/the-subjective-function-part-3, I showed three inserts to help us in this determination.  I will analyze those today.

Let me start by emphasizing the need to study relative differences in net profit, drawdown, and other details between the three inserts.  If I focus on absolutes then I will be disappointed.  The one and only goal of this study is to determine which I prefer most.

In looking at the equity curves, my first observation is that they are all upward sloping to the right.  I should hope so.  Each insert corresponds to the system that performed best on a different metric.  Whichever curve I like the best corresponds to the metric I will use to evaluate systems.  This will be the subjective function.  Since each of these systems is the best in some respect, I am glad to see they are all profitable.

Equity curves #1 and #2 have many discernible plateau (horizontal) regions as opposed to #3, which seems to have more incremental changes.  This corresponds to the total number of trades.  Inserts #1, #2, and #3 have 57, 37, and 399 total trades, respectively.  On this metric alone, I prefer a system that trades more because that means more opportunity for profit.

Drawdown curves #1 and #2 have a much smaller blue area than #3.  This means #3 spends more time in drawdown and less time at new equity highs.  I really like new equity highs because they make me feel like a successful trader.  The broad market does not make all-time highs very often.

Equity curve #1 is lower now than it was around the beginning of 2009.  This seems like a negative.  However, each of these systems goes into drawdown periodically and curve #1 did hit a new high roughly 1+ centimeters ago (on my screen), which is only a couple fractions of a centimeter more than the other two.  Curve #1 just happens to be in greater drawdown (-0.59%) than the other two but by no means it this drawdown remarkable.  This observation may be disregarded.

I will continue this analysis in my next post.

The Subjective Function (Part 3)

I keep coming up with quirky analogies for different aspects of the system development process.  In http://www.optionfanatic.com/2012/09/19/trading-system-1-initial-assessment-part-1/ on 9/19/12, I talked about a Motion to Dismiss.  On 10/8/12 (http://www.optionfanatic.com/2012/10/08/the-subjective-function-part-2), I talked about visiting the eye doctor.  As promised, today I will present some graphs for study to help determine the subjective function.

The following inserts show two graphs each.  On the top is the equity curve itself:  a plot of account value over time beginning with $1M and traded with a $100K position size.  On the bottom is the drawdown (DD) curve.  DD is zero whenever the account is at a new high.  The red line on this graph tracks the max DD ever seen over the whole time period.  You will see the red line dip to levels not seen in the blue graph but that is only because the DD was too short to be seen in the blue graph.  If you were able to zoom in and magnify then you would see the blue graph dip to those levels.

As you look the following three inserts, imagine that you are studying the performance of your investment account over time.  Which would you feel best about?  Note details such as relative net profit (final value is shown in the upper left of the top curves), relative severity of drawdowns (current drawdown at the right edge of the bottom curves is shown in the upper left of the bottom curves), relative time between new equity highs, number of horizontal periods (indicating no current trade), etc.

Now without further ado, let me present…

Insert #1:

Insert #2:

Insert #3:

I will analyze these graphs based on my preferences in the next post.

The Subjective Function (Part 2)

My last post (http://www.optionfanatic.com/2012/10/05/the-subjective-function-part-1) defined the subjective function.  Today I want to present another perspective on the subjective function by explaining what I previously missed.

The gist of my previous posts is that all traders are looking for the same general thing.  “Consistent profitability over time” is a phrase I have written many times over.  Yes, Trader A may be looking for the supercharged exponential returns that carry the risk of heavy drawdowns while Trader B is more conservative and will sacrifice returns to minimize drawdown.  Aren’t they both basically looking for consistent profitability over time, though?

No.

I now see that Trader A and Trader B are looking for decisively different things.  My “consistent profitability over time” is what Trader B seeks.  Trader A, however, is in effect buying a lottery ticket and hoping for the big payout.  If financially savvy then Trader A is playing this with a limited percentage of the total portfolio so as not to go completely bankrupt.  Trader B might be all-in with proper asset allocation.

My previous lack of understanding might reflect an inadequacy of language itself.  I suspect we just don’t have enough terms to reflect the variety of individual preference for investment returns.  Compound annual growth rate (CAGR), drawdown (DD), and flat times translate directly to features seen on equity curves but others do not (e.g. profit factor, Sharpe and Sortino ratios).  Equity curves reflecting the same CAGR or DD can look wildly dissimilar, however.  The pictures tell more than words can describe.

Moving beyond language in this manner is what Dr. Bandy recommends in Quantitative Trading Systems (2007) to determine the subjective function.  After running the trading system optimization, scroll through the different equity curves to find the ones you like best.  Note which metrics score highest on your favorite equity curves and voila!  You have the subjective function.  This is not unlike a visit to the eye doctor where you endure an exhausive series of “which lens is better:  one or two?  1 or 2?  2, or 1?  2, 1?”

In fact, let’s do that:  a visit to the optometrist’s office, and more, when we return.

The Subjective Function (Part 1)

I’ve mentioned multiple times that in trading system development we’re looking for consistent profitability over time.  To do this, identification of the “subjective function” is a prerequisite. 

Much of the direction to my current work is owed to Dr. Howard Bandy.  You can read about him here:  http://www.linkedin.com/pub/howard-bandy/b/b44/270 .  In his book Quantitative Trading Systems (2007), Bandy defines the “objective function” as:

> A single-valued measurement of the fitness of a trading system. Optimization
> techniques are guided by maximizing the value of the objective function.

He also writes:

> Only you can decide which trading systems are good and which are not.
> Selection of the objective function is a management decision and must
> be made prior to extensive model development. The objective function
> itself is not a candidate for optimization.

What does this mean?

System trading is the epitome of Yin and Yang!  Beginning on 5/7/12 (http://www.optionfanatic.com/2012/05/07/strategies-vs-systems-part-i/), I wrote a number of blog posts discussing discretionary vs. systematic trading.  The upshot was that discretionary trading is subjective and systematic trading is objective.  This may be true but at the heart of a somewhat objective system development process is the very personal selection of Bandy’s objective function.   For this reason, I have chosen to call it the subjective function.

All this subjective function talk may seem highly theoretical but it really boils down to something quite simple.  On 10/4/12 (http://www.optionfanatic.com/2012/10/04/trading-system-1-spy-vix-part-4/), study the y-axis to see my subjective function:  net profit (%).  I proceeded to discuss the best combination of parameters by performance on that measure.

The only problem is that I failed to go through the process of determining my subjective function before implementing it.

Stay tuned for further discussion and a possible detour in upcoming posts.

Trading System #1–SPY VIX (Part 2)

In http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I introduced the concept of evolutionary operation and began to talk about the different variables present in this trading system.  Today I want to continue that discussion from the perspective of optimization.

Investopedia.com defines optimization as:

“…the process of adjusting one’s trading system in an attempt to make it more effective. These adjustments include changing the number of periods used in moving averages, changing the number of indicators used, or simply taking away what doesn’t work.”

In the system development process, optimization is going to allow me to look at performance metrics for every combination of specified parameter values.  In the present context, I will allow x to vary from 3-7 in increments of one [e.g. 3 (days), 4, 5, 6, or 7].  I will allow y to vary from 5-25 in increments of five [e.g. 5 (percent), 10, 15, 20, or 25].  I will allow z to vary from 6-15 in increments of one.  The result of this process will be summary statistics for 5 * 5 * 10 = 250 trading systems.

The goal of optimization is not to find out which set of parameter values performed the best in order to trade those going forward.  This is called “curve-fitting” and the reason optimization has gotten a bad name in some circles.  The more variables you have and the greater the number of total parameter value combinations, the more unlikely you are to see similar performance in the future as seen in the most profitable backtest.

Rather, the goal of optimization should be to assess the performance landscape from one combination of parameter values to the next.  Going back again to http://www.optionfanatic.com/2012/09/28/trading-system-1-spy-vix-part-1/ (9/28/12), I want to know if these performance graphs have spike regions or plateau regions.  If they have spikes then I will discard them, considering performance to be lucky rather than good.  If they have plateau regions then I want to choose parameter values near the middle of those plateaus (margin of safety).

In my next post I’ll start running the numbers.

Trading System #1–SPY VIX (Part 1)

With the “motion to dismiss” now denied, the next step is to lay the groundwork for developing this strategy into a trading system.

As a review, the trading strategy is based on two claims.  First, five days after the CBOE Volatility Index (VIX) closes 5% or more below its 10-day simple moving average (SMA), the S&P 500 index (SPX) has lost money on average. Second, when the VIX closes 5% or more above its 10-SMA, SPX has outperformed the average week better than 2-1 over the next five trading days.

The current system in development offers three variables to be backtested:  x-bar stop y% extended from its z-SMA.  In this case, we have used x=5 (five-day trades), y=5, and z=10 (VIX closing 5% above/below its 10-SMA).  What we want to see in backtesting is solid results with these values.  We also want to see solid results when we slightly change these values, though.

The process of “evolutionary operation” looks to keep one variable constant while changing another to see its effect on system performance.  Graphically speaking, we want to plot some measure of performance (e.g. net profit) vs. the one variable being changed:

This does not have the appearance of a robust system because slight changes in the variable drastically affect net profit.  Whether a moving average is four periods vs. five or 14 periods vs. 15 should not matter.  If one value appears magically improved in the midst of mediocre results then those backtested profits are likely a matter of luck rather than the system taking advantage of consistent opportunity (and subsequent future profits).

This is what we hope to see.  All values from 6-15 produced solid profit for this trading system.  Rather than the spikes in the previous graph we see a plateau region in this one.  This should be a necessary (but not sufficient) condition for a system we might trade in the future with real capital.

These graphs were adapted from Tomasini and Jaekle (2009).

In my next post I will continue this discussion in terms of optimization.