Option FanaticOptions, stock, futures, and system trading, backtesting, money management, and much more!

2012 Performance Evaluation (Part 7)

As discussed in my last post (http://www.optionfanatic.com/2013/02/18/2012-performance-evaluation-part-6/), 2012 was calm for stocks as compared to any of the preceding four years.  Give me any tame market and my account should flourish.  Indeed, last year I returned over 53% when the best-performing index returned under 16%.  What a windfall!  Don’t you just wish I managed your money?  Do you want to be like me?

Not just yet.

As for the other two questions, no and no.

I cannot depend on quiescence in order to succeed.  I will only be a good trader when I can make money in calm markets and avoid losing big in violent ones.  With the exception of 2008, I have yet to demonstrate success with the latter.

I now want to adjust my focus from monthly to weekly performance numbers.  The basis for this discussion involves tracking account value alongside its moving average (MA).  The MA serves as a “loss filter” by signaling to stop trading real money in favor of paper trading when the equity curve crosses below the MA.  When the hypothetical account value crosses above the MA then trading may resume with real money.

While I think this concept has merit, I worry about the potential for curve fitting.  I would not want to optimize and pick the best MA period without surveying the parameter space to make sure good results are seen with neighboring values.  I also worry about sample size.  I have less than five years of weekly account values available and in that time, I may only have a handful of instances where account value has crossed its MA.  If I don’t have enough cases to form a valid sample then the door is left open to curve-fit results and decreased probability of future profit with real trading.

In the next post I will discuss a variation on this theme.

2012 Performance Evaluation (Part 6)

For the first time in this blog, I am reviewing my performance from 2001 to the present.  In http://www.optionfanatic.com/2013/02/15/2012-performance-evaluation-part-5/, I reviewed 2011 and began discussion of two changes made in my trading following the summer market crash.

Aside from not trading until after 3:30 PM, the second change I have adopted is a stricter limit on position sizing.  Although it is difficult to nail this down with certainty, the largest long position size (by number of contracts) I would carry today is about 40% of that carried in 2010 and about 70% (about 60% in dollar terms) of that carried in 2011.  I have a maximum monthly long contract limit in mind and I prefer to start out trading 33-67% of that limit.  In most months I do not even reach the limit.

Position sizing has been restricted on the short side as well.  I would estimate my acceptable short contract limit to be 60% or less than what I carried in 2010-11.

One thing that still concerns me is a tendency to see position size creep higher from one month to the next.  This reflects difficulty accepting loss.  Rather than close out positions and realize loss, I look to roll positions for equal or greater credit.  This will boost profitability in a tame market but lead to suffering when the bull or bear really gets angry.

In 2012, the Russell 2000 experienced two corrections over 10% but still moved higher by year-end.  For the first year in four, I outperformed:

With a 53% return and MDD of only 3.6%, this was a dream year for my portfolio.  This was a relatively tame year for the markets, however, as the largest MDD of any major index was only 8.2% (Nasdaq).  I give much more credit to the tame market than to my own skill for the exhilarating performance.  While I feel the updated trade guidelines have helped to stabilize my portfolio, it will take the next significant market correction to truly make an apples-to-apples comparison with what happened in 2010 and 2011.

2012 Performance Evaluation (Part 5)

As my own boss, I am conducting an annual review in this series of blog posts.  In http://www.optionfanatic.com/2013/02/14/2012-performance-evaluation-part-4/, I continued with evaluation of 2010 and 2011.  I now want to spend some additional time focusing on 2011.

As mentioned, 2011 was the fourth time I had been forced to exit positions turned against me because of excessively large position sizing.  In previous years, I escaped with relatively modest losses.  In 2011, however, my maximum drawdown (MDD) exceeded 50%.  That is entirely unacceptable when trading other people’s money, which is one reason why I don’t, but also unacceptable when trading my own.  I’m running a business here and losing that much money is the fast track to going “belly up.”

2011 was the third consecutive year of underperformance relative to the major indices.  Despite the ugly MDD, I managed a 42.5% rebound from the year’s low, which landed me down “only” 10.5% for the year.  I’m probably still standing for this reason alone.

In an attempt to turn things around, I have attempted to implement two major changes.  First, I have resolved to avoid trading until after 3:30 PM.  On too many days, I got scared by seeing strong market moves in one direction early only to see those moves reverse by day’s end.  Usually when I made a trade earlier in the day I was smacking myself at the close wondering why I wasn’t more patient.  This is overtrading and it may be labeled as such only in retrospect.

To facilitate this end-of-day approach, on many trading days since summer 2011 I have not even looked at the market until 3:30 PM.  Over the next 15 minutes I typically catch up with the market’s intraday movement and then place trades (if necessary) at 3:50 PM or later.  This has helped me be less reactive to random noise responsible for so much market activity.

I will continue this analysis in my next post.

2012 Performance Evaluation (Part 4)

In the last few posts, I have been conducting my 2012 annual review.  In the future, this will take only a day or two.  Since I have never blogged about it before, however, this time I am reviewing back to inception.  I reviewed 2008 and 2009 in http://www.optionfanatic.com/2013/02/13/2012-performance-evaluation-part-3/.  Today I will continue with 2010-11.

By the numbers, 2010 went like this:

May 6, 2010, was the infamous Flash Crash where the DJIA dropped over 1000 points in one hour.  May capped my MDD for the year of 20.5%.  In the last post, I said this would be an acceptable annual MDD if it never got any worse.  The largest MDD seen in the major indices for 2010 was 13.9% (Russell 2000), though.  That makes my MDD seem a bit less heartening.

As market volatility soared from April to May of 2010, my portfolio hemorrhaged cash.  The psychic pain became too much to bear on May 6, when I closed the last of my long positions.  This may sound familiar because it is:  a third time being caught trading too large.  As is often the case when trading too large, I was forced out at the absolute worst time (MDD).

2011 was very similar to 2010:

Again, this year brought a more severe mid-year market crash (Russell 2000 down 24.3% in 2011 vs. 17.1% in 2010 over roughly a 1-month time period), albeit without the magnitude of volatility increase seen during the Flash Crash (volatility spiked 65.2% in 2011 vs. 172% in 2010 over roughly a 1-month time period).  I was camping at the time (with my laptop), and was caught with my pants down carrying too large a position for–what time is this?  2007, 2009, 2010… oh yeah, the FOURTH time.  In prior posts, I have said that a 20% maximum drawdown (MDD) from one year to the next would be acceptable to me.  In 2011, my MDD was 50.1%.  Contrast this with the largest broad based index MDD of 25.6% and you have a recipe for disaster.

The analysis will continue with my next post.

2012 Performance Evaluation (Part 3)

This blog series represents my 2012 annual review.  In http://www.optionfanatic.com/2013/02/12/2012-performance-evaluation-part-2/, I discussed my performance through 2007.

2008 was my introduction to full-time trading.  Faced with a market in correction, I traded much of the year with reduced position size:

This was not a cakewalk as my maximum drawdown (MDD) hit 20.9% in March.  While discomforting, if this were the worst MDD ever encountered on a year-to-year basis then I would be content.

My MDD was only about half that seen in the major indices.  I capitalized on much of the October-November tumble by purchasing put spreads.  I remember spending hours in front of the screen with my mouth propped open as I watched the horror of a market in free fall.  I felt bad for those who could only watch big retirement savings fly out the window from one day to the next because “buy and hold” was the mantra espoused by the financial industry for years and years.

My good fortune reversed in 2009:

What killed me in 2009 was a 27.7% MDD, which occurred in the span of two months.  This was another case of trading too large and being caught with my pants down when the market made its third and final thrust lower.  I traded small for the remainder of the year as I reassessed my trading plan.

In retrospect, what frustrates me is the need to experience this lesson multiple times before I learn it.  My first lesson was in 2007 when a couple sharp selloffs turned my huge, long positions against me.  In 2009, I saw this happen once again.

I will continue the analysis in my next post.

2012 Performance Evaluation (Part 2)

In the absence of co-workers and a boss, the biggest reason I maintain this blog is to keep myself accountable.  To that end, I presented equity curves of my performance since inception (2001) in http://www.optionfanatic.com/2013/02/11/2012-performance-evaluation-part-1/.  The current blog series represents my annual performance review and a tool to tweak my trading plan, if necessary, to stay on track with long-term goals.

Because my time commitment and overall approach have been highly variable over the last 11+ years, I will not try to make generalizations about the whole.  This would be like comparing apples to oranges.

From June 2001 through the end of 2006, I worked full-time in pharmacy.  My investment plan included a set of stock screens that I ran on a monthly basis.  My net return during these years is shown below:

These years included sharp selloffs during 9/11 (2001) and 2002 along with bull market conditions in 2003 and 2004-2006.  The maximum drawdown, faced only five months into my investing career, was 32%:  larger than that seen by the Russell 2000 and DJIA indices.  In the end, I did manage to outperform the Russell 2000 by 6.2% per year and the other three indices by about 12% annually.

In 2007, I started trading options in earnest.  I was up 36% for the first five months of the year vs. the leading index (Russell 2000), which was up only 6%.  By year’s end, I gave back all these profits and more.  In the process, I learned an important lesson about luck.  Because good luck may sour, being in a position to limit losses when the market turns against me is more important than being able to make money when the market goes my way.  This is good risk management that is essential for success because the degree of loss will usually outpace the degree of gain when all other factors remain equal.

I will continue this analysis in my next post.

2012 Performance Evaluation (Part 1)

The main reason I maintain this web site is to keep myself on task with what I want to do.  Starting from the “ground up,” system development has been very difficult for me.  Without co-workers, it would be easy to give it up and do something else.  The web site keeps me accountable.

Along these lines, I want to spend some time focusing on performance to make sure I remain on course in pursuit of personal goals.  Getting focused on small details and losing sight of the big picture is very easy to do when you don’t have to check in with bosses and supervisors.  I will report here.

As a brief review, I took over the investing for my personal account in mid-2001.  In 2008 I resigned from Pharmacy and began life as a full-time trader.  My investing/trading approach has changed much over the years and that is important for the here-and-now.  With the performance statistics now updated through January 2013, though, I wish to first spend some time looking at the entire history.

Below is a graph of my total performance to date.  I have set the starting account value to be $100.  The first graph is plotted with linear scaling:

Since linear scaling can sometimes be misleading, below I show the same data with logarithmic scaling.  Note here that identical percent changes traverse the same distance along the y-axis:

In my next post, I will begin to analyze and to discuss these data.

Walking it Forward with System Validation (Part 6)

My blog series “Lingering Quandaries about System Development” concluded by discussing a paradox with regard to Howard Bandy’s WFA discussion.  In http://www.optionfanatic.com/2013/02/07/lingering-quandaries-about-system-development-part-9/, I concluded by suggesting a new and improved WFA that can achieve system development goals.

I recommend not following Bandy’s advice to select periods for IS and OOS data early in the process and retaining them throughout development.  Besides effectively burying your head in the sand as described by Example 1 (http://www.optionfanatic.com/2013/01/29/walking-it-forward-with-system-validation-part-1/), you really can’t know what values may or may not work until you actually perform the WFA.  Validation is the final step of system development.

Rather, look to perform WFA by optimizing the periods and studying the entire parameter space.  Perhaps I will vary IS period from one year to three years by increments of two months.  Perhaps I will vary OOS period from one month to six months by increments of one month.  I then need to plot values of the subjective function in a three-dimensional space (or in two dimensions with color coding to represent subjective function ranges) to get a feel for where the high plateaus exist.  I should then select the time ratio to coincide with the middle of a high plateau and use trading parameters coincident with that combination.

Every time I do a WF iteration, I am selecting WF parameter values and subsequently trading parameter values.  The two differ, in effect, by an order of differentiation.  That is, the subjective function for the WF optimization coincides with a value for the concatenated equity curve to date.  The subjective function for the iteration coincides with a value for the equity curve of the preceding IS period only.

WFA serves to both validate a trading system and to direct trading at the right edge of a chart.

Lingering Quandaries about System Development (Part 9)

http://www.optionfanatic.com/2013/02/06/lingering-quandaries-about-system-development-part-8/ continued discussion of a third System Development paradox that I have been trying to sort through:  Howard Bandy’s handling of WFA.  Bandy said to choose periods for IS and OOS data early in the process and then stick with them throughout development.

The more you vary these system parameters, Bandy said, the more the OOS data loses its “out-of-sampleness,” which is why they ought not to be changed.  I flat-out disagree with this statement.  Varying the time ratio does not change or determine parameter values involved with the trading rules.  Varying the time ratio is solely to verify that neighboring values also validate the system.  This will remove suspicion of system validation as a fluke occurrence.

One issue that remains unresolved for me is how to conceptualize “varying the time ratios” in a systematic manner that may be plotted.  I consider myself spatially challenged so I probably just need to cram into my brain the need to specify a minimum and maximum value for both IS and OOS periods and an increment by which to vary them.  I can then make a two-dimensional plot of the subjective function (e.g. RAR/MDD) and perhaps color code the values.  This way, I can see if an area of outperformance stands out.

One other issue I wonder about is whether OOS period might be limited by sample size considerations.  The OOS period will already be short relative to the IS period.  Might I need to be concerned about it being short enough to allow for any trades?  The shorter the period, the more total periods I will have in the WFA but if it is too small to allow for trades in any one iteration then I wonder if the entire WFA risks insufficiency.

I will only discover the answer to this question when I start attempting WFA myself.

In my next post, I will summarize the new-and-improved approach to WFA.

Lingering Quandaries about System Development (Part 8)

In http://www.optionfanatic.com/2013/02/05/lingering-quandaries-about-system-development-part-7/, I introduced the third paradox encountered thus far in my System Development studies–this one having to do with walk-forward analysis (WFA).

As I suggested, something about the validation process Howard Bandy describes seems like curve fitting.  A few months ago, a reader asked Bandy in a forum post about the proper time ratio of IS:OOS data to be used in WFA.  Bandy did affirm that some time ratios may produce acceptable OOS performance where others may not.  His solution was to use whatever works.  To me, that sounds like cherry picking the right combination, which is “curve fitting:”  the four-letter word of System Development.

Just the other day, I once again directed this question to Bandy on his blog.  His response:

> Yes, in-sample and out-of-sample time periods are parameters of the system and they do need to
> be chosen. My recommendation is to choose them (particularly the length of the in-sample period)
> early in the development process, then keep them fixed from that point on… Keep in mind that
> every decision to adjust any component of a system based on examination of out-of-sample results
> reduces the out-of-sampleness of that data and increases the degree that the system is curve-fit to
> the specific data.

As a parameter of the system itself, choosing set values for IS and OOS periods is like Example 1 from http://www.optionfanatic.com/2013/01/29/walking-it-forward-with-system-validation-part-1/.  This fails to take into account the shape of the parameter space.  I want to see high plateaus of performance rather than peaks.  In taking Bandy’s suggestion, I would never study the neighboring values.

I will conclude this discussion in the next post.