Backtesting Frustration (Part 4)

Today I continue to hack my way through this difficult exercise of putting into words exactly why I sometimes find backtesting to be paralyzingly frustrating.

And yes, that really is a word:

(sorry but some periodic comic relief along with that candy are what it’s taking to get me through a subject like this)

Flawed data decreases backtesting accuracy by an indeterminate amount. I do not realize the data is flawed so I don’t know how often errors occur, to what extent, and therefore how large the impact. The reason I don’t routinely screen for inconsistencies is because a backtest is usually thousands of trades that will already take months to complete. Scrutinizing every historical day would take much longer. Besides, I shouldn’t have to do this. I pay a lot for OptionVue (OV) because I trust they offer a valuable tool. If I cannot trust the data then what good is the software?

In addition to being occasionally flawed, the OV data is spotty at times (especially 2001 – 2003). Missing data must be filled in with theoretical prices (less accurate) but when theoreticals are not available I have to enter prices manually based on a logical vertical volatility skew. This adds 2-3 minutes per historical day.

I also have to deal with insufficient strike availability. Consider below with the underlying at $460:

My backtest will require options down to 410 and sometimes 380. When I manually add these:

OV automatically duplicates root symbols (see red numbers), which results in flawed (duplicated) option prices:

So when forced to add strikes manually, I need to fill in with root symbols that I may not know. Sometimes I jump to a portion of the database I feel confident will have options in that range to determine the correct letters. For example, if I need strikes in the 300s then I can try jumping ahead to September 2001 where the underlying fell into the 300s to find out what letters were used. Once entered, I have to turn off the “Auto Strike” feature to prevent OV from resetting the matrix (automatically done on the first trading day of every week), which would thereby delete the additional work I have done—additional work that takes minutes, by the way.

And realize this problem is all secondary to the primary issue of an incomplete database.

I mentioned above that a backtest usually includes thousands of trades. You can imagine how tremendously frustrating it is to be repeatedly slowed down like this.