Option FanaticOptions, stock, futures, and system trading, backtesting, money management, and much more!

Curve Fitting 101 (Part 4)

Before I move forward with trading system development approaches, I want to give one more example of how to curve fit: increasing strategy complexity.

Here’s a basic strategy (in EasyLanguage):

          Input: len(14), //optimize 2-20 by 3
                    offset(40), //optimize 30-45 by 5
                    pt(500); //optimize 500-2000 by 500

          setprofittarget(pt);

          if percentR(len) < (50 - offset) then sellshort next bar at market;
          if percentR(len) > (50 + offset) then buy next bar at market;

This is a trend-following strategy that will go long (short) when Williams %R is above (below) an extreme value. I am testing extreme %R values of 20 (80), 15 (85), 10 (90), and 5 (95):

Initial_WFO_equity_curve_(4-20-20)

Notice how the strategy performed exceedingly well until the arrow (~500 days in). After that, equity took a precipitous fall and lingered in the doldrums for the remainder of the backtest.

I can improve this by increasing strategy complexity. At the arrow, I will flip from trend following to mean reversion:

          if percentR(len) < (50 - offset) then buy next bar at market;
          if percentR(len) > (50 + offset) then sellshort next bar at market;

After adding a rule, I can look at the equity curve again. Where I see the beginning of another large drawdown, I will add a subsequent rule to flip back to trend following. I can then look at the equity curve again. Where I see the beginning of another large drawdown, I will add a subsequent rule to flip to mean reversion. Rinse and repeat a few times!

With first four lines of code unchanged, here’s what I ended up with:

          If date <= ELDate(11,01,2011) then begin
               if percentR(len) < (50 - offset) then sellshort next bar at market;
               if percentR(len) > (50 + offset) then buy next bar at market;
          end;

          If date > ELDate(11,01,2011) and date < ELDate(08,14,2012) then begin
               if percentR(len) < (50 - offset) then buy next bar at market;
               if percentR(len) > (50 + offset) then sellshort next bar at market;
          end;

          If date >= ELDate(08,14,2012) and date < ELDate(02,20,2013) then begin
               if percentR(len) < (50 - offset) then sellshort next bar at market;
               if percentR(len) > (50 + offset) then buy next bar at market;
          end;

          If date >= ELDate(02,20,2013) and date < ELDate(02,11,2014) then begin
               if percentR(len) < (50 - offset) then buy next bar at market;
               if percentR(len) > (50 + offset) then sellshort next bar at market;
          end;

          If date >= ELDate(02,11,2014) and date < ELDate(12,03,2015) then begin
               if percentR(len) < (50 - offset) then sellshort next bar at market;
               if percentR(len) > (50 + offset) then buy next bar at market;
          end;

          If date >= ELDate(12,03,2015) and date < ELDate(09,29,2016) then begin
               if percentR(len) < (50 - offset) then buy next bar at market;
               if percentR(len) > (50 + offset) then sellshort next bar at market;
          end;

          If date >= ELDate(09,29,2016) and date < ELDate(01,24,2018) then begin
               if percentR(len) < (50 - offset) then sellshort next bar at market;
               if percentR(len) > (50 + offset) then buy next bar at market;
          end;

          If date >= ELDate(01,24,2018) then begin
               if percentR(len) < (50 - offset) then buy next bar at market;
               if percentR(len) > (50 + offset) then sellshort next bar at market;
          end;

I increased the number of rules in this strategy from one to eight. The rules couple date ranges with a mean-reversion/trend-following switch.

This is some of the most blatant curve-fitting possible. I’ve used nothing but hindsight, which we know is invalid as a system development technique (see fourth paragraph). Also, recall in Part 2 where I chose a specific parameter value (VIX < 35)? With that approach, I still have to settle for whatever equity curve results. Here, every rule I add directly manipulates the equity curve in whatever direction I so choose.

This is also the epitome of curve fitting because the rules refer to historical date ranges that don’t even exist in the future. Doing so is criminal indeed:

Curve-fit WFO equity curve (4-20-20)

This has 78% winners on 479 trades, a profit factor of 1.71, and trades nearly even with regard to long and short positions (with short trade profit factor of 1.55). Slippage and commissions are included.

The more complicated the strategy, the more beautiful the equity curves. Just remember the more beautiful the equity curve, the less likely it is to be fit to signal rather than noise.

Today’s result is totally, utterly, and absolutely curve-fit. Can’t trade that, folks: no way, no how.

Curve Fitting 101 (Part 2)

In the last post, I gave examples of how optimization could be used to manufacturer an investment newsletter scam. Today, I want to get back to trading system development and give another example of how curve fitting can generate deceptive results.

Also known as overfitting, curve fitting is tailoring a trading strategy to better match historical noise than persistent signal. The strategy may look great in backtesting, but it is unlikely to look good when applied to future data because noise is random.

Besides optimization (see Part 1), a second way to curve fit is to use specific parameter values. Consider a simple strategy that buys pullbacks (in TradeStation’s EasyLanguage):

          If C = lowest(C,10) then buy next bar at market;
          If barssinceentry >= 3 then sell next bar at market;

This strategy buys after the lowest close in 10 days (bars) and sells after the position has been open for at least three days.

Suppose this strategy suffered its maximum drawdown on the VIX spike over 35 in the 2008-9 financial crisis. Let’s add another rule by changing the first line to:

           If C = lowest(C,10) and close of data2 < 35 then buy next bar at market;

Now, the long entry will occur when close is the lowest close in 10 days and when VIX is less than 35. This can improve the equity curve by avoiding a losing trade(s) and going flat in an area that previously experienced drawdown.

The problem here is adding a specific rule to fix a particular past occurrence. It’s like applying a single Band-Aid. A similar event capable of causing significant damage may occur in the future, but the filter may not be effective since we based it on a sample size of 1. Look what happened in February 2018:

Curve fit chart (4-18-20)

The arrows point to February 2 when ES had a big drop of 2.3% and VIX spiked to 17.31. That was a big move with VIX closing at its highest level in nearly 15 months (11/8/16), but it did not trigger our filter (17.31 < 35). The curve-fit strategy would enter the pullback just in time for February 5, 2018, when the market fell 5.4% and VIX spiked to 37.32. The VIX filter would now be active: one bar too late to serve its purpose. I analyzed a trading strategy that applied this rule here. My curve-fitting comments are in the seventh paragraph.

I will continue the discussion next time.

Investment Newsletter Scam

For some reason, this curve-fitting exercise reminds me of the investment newsletter scam.

This scam may be executed in two different ways, and either is good reason to avoid investment newsletters. Technically speaking, I don’t believe they are illegal. “Scam” is an informal term whereas “fraud” is a crime. Both are deceptive, however, and would be categorized as optionScam.com.

Imagine the following hypothetical scenario.

I can probably sell lots of subscriptions to my investment newsletter by heavily promoting the winners and making the losers disappear. What if I were actually to publish many investment newsletters over the next few years? Suppose I publish under different pseudonyms and offer them online for free. I don’t market or advertise, but anyone can access them. They’re located by internet search engines with verifiable time/date stamps. This is a legitimate service.

Out of 10 newsletters, nine underperform and one significantly beats the overall market. I go all-in and market the one:

      > OPTIONFANATIC NEWSLETTER BEAT THE MARKET BY 10% OVER THE
      > LAST THREE YEARS!!! ONLY 100 SUBSCRIPTIONS WILL BE OFFERED
      > FOR $59/MONTH OR $649 FOR THE FULL YEAR! ACT NOW!!!

This was Just. Random. Luck. One out of 10 did well and the others sucked! Although future subscribers are unlikely to profit based on these odds, I have the documented performance to support my prior success (I simply neglected to mention the losers). What an ingenious way to eliminate performance stress and accountability. It’s a lucrative and innovative proposition.

For all we know, this may be a popular tactic. Often when I see an enticing sales pitch, I conduct an internet search for the name + “scam.” As an example, for the hypothetical “RocketOne Trading,” I would search “RocketOne Trading scam.” I have seen lots of reviews on forums or websites from people who have been outsmarted by nefarious entities. I’ve also seen many shill [fictional] recommendations. Check out the second paragraph here. It’s not infrequent to see [sketchy?] companies go out of business and newsletters fold, but new ones are always popping up. Some of these get called out by online mentions (e.g. “X newsletter is actually being Y person who was slapped with a prior SEC complaint”). Indeed, many of these may be orchestrated by a few key players.

This investment newsletter scam could be perpetrated another way. Imagine I send out stock recommendations to 2,000 random people: 1,000 each receive buy or sell recommendations. A few months later, I follow-up with a second stock recommendation to the happy half: 500 each receive buy or sell recommendations. A few months later, I follow-up with a third stock recommendation to the very happy half: 250 each receive buy or sell recommendations. After 21 months and roughly 4,000 mailings, 15 ecstatic people will have received seven consecutive profitable stock recommendations:

      > EXCLUSIVE OFFER FOR OPTIONFANATIC NEWSLETTER SUBSCRIBERS:
      > ONE NEWSLETTER PER MONTH FOR ONE YEAR: ONLY $4,999! GET
      > IN BEFORE THE DOORS SLAM SHUT!!

How likely are the 15 to believe I am the stock whisperer and to subscribe at a moment’s notice?!

Including the curve-fitting exercise from my last post, these are all forms of optimization. I have probed everything, selected, and marketed the best [without breaking the law]. Admittedly, I haven’t told the whole truth, and my actual probability of being right next time is only 50/50 (at best). Nonetheless, from the other side the prospects sure must look amazing!

Curve Fitting 101 (Part 1)

In an effort to explain what not to do, today I go to the dark side with a tutorial on how to curve fit (overfit).

The easiest way to curve fit is through optimization. Wikipedia tells us:

     > In mathematics, computer science and economics, an optimization problem
     > is the problem of finding the best solution from all feasible solutions.

A standard optimization tests all possible variable combinations. Each one is called an iteration.

Consider a simple trading system:

          Buy when price crosses above the x-day moving average;
          Sell when price crosses below the x-day moving average.

I can code this into any number of software platforms. This is what it might look like in TradeStation, for example:

          Input: MALen(10);

          If C > average(C,MALen) then buy next bar at market;
          If C < average(C,MALen) then sell next bar at market;

I can then use a tool called the optimizer. The optimizer will backtest a strategy between specified dates. I can input values of MALen for it to test and then I look to see which values performed best.

As an example, I can test between Mar 5, 2009, and Jul 5, 2014. I can optimize moving average length (MALen) from 20 to 100 by 10. This will test nine different iterations: 20, 30, 40, 50, 60, 70, 80, 90, and 100.

The output will be a Strategy Optimization Report. For each iteration, it will give me trade statistics like net profit, gross profit, total trades, % trades profitable—a list of statistics similar (but not identical) to the “standard battery” (second paragraph).

Next, I pick something to be the subjective function (I should have done this first to avoid bias, but it’s all cheating anyway so it really doesn’t matter!), sort that column from highest to lowest, and select the value [of MALen] connected to the best performance. If best performance is a large number then the graph should look pretty good.

Finally, present the best iteration as a strategy I developed. Rather than writing “x” or “MALen,” I just substitute the actual number that led to best performance.

Voila! I have a beautifully curve-fit strategy that I can go out and sell (and hopefully get convicted of some sort of fraud if I really do). Here is our result:

Curve fit equity curve (4-17-20)

The curve isn’t perfect, but it is pretty darn good with net profit $27,600/contract, average trade $1,105, and profit factor 3.32 on 25 trades over 5 years 4 months.

Oh by the way, I did a couple other things to make this look good:

      1. I found a market that performed well (CL, GC not so much).
      2. I did not include transaction fees (something I would absolutely, never do).
      3. I adjusted the chart to end on an equity high.

The chances of this strategy performing this well over the next few years (or even months) is tiny. In fact, because of the third bullet point, you don’t even see the large drawdown that immediately afterward.

I will continue next time with more curve-fitting examples.

Inherent Underlying Motives in Finance

Let’s spend some time discussing inherent underlying motives in the financial industry. As I implied in the last post, this cannot be stressed enough.

I have addressed the topic many times throughout this blog. One that comes to mind is here. I discussed “Karen the Supertrader” here and here. I’ve touched on related subjects here, here, and here.

Let’s go back to the fifth paragraph of my last post:

      > We can read articles, blog posts, watch CNBC interviews by “industry insiders,”
      > and potentially talk to industry representatives ourselves [e.g. IARs, quants
      > we meet at dinner parties, or principals of financial firms that we casually
      > encounter (how?)], but most of these people cannot escape inherent conflicts
      > of interest. We are left to choose whether or not to believe

An inherent conflict of interest exists when someone has something to sell. IARs may sell us on their firms. Quants may solicit investment in their funds, subscription to their services, or purchase of black box systems. Principals may become our paid money managers. The more practiced, knowledgeable, experienced, and successful they seem, the more likely they are to profit from us. Even if we are not the target clients, they may profit if we refer them to someone who is (i.e. networking).

It’s always possible that a conflict of interest renders someone unbelievable because it provides motive to deceive for [direct or indirect] personal gain.

As a deep refresher, recall this post. Everything financial that we read, we see, and we hear must be critically evaluated. If we don’t work hard to avoid chicanery, then hard-earned capital has a good chance of landing in someone else’s pocket to our own detriment. This is not a certainty, but it happens far too often to demand anything less than our closest attention. It takes just one lapse of judgment to suffer losses and permanent impairment to our ability to ever trust a financial professional again. If we have the drive and time to learn for ourselves then problem solved! Otherwise, we may be in a bind if deep-seated distrust prevents us from ever again going forth to access valuable services provided by ethical financial professionals.

      > (and to what extent). While I do believe select individuals have a bead on
      > the truth, that’s not me and I’m not afraid to admit that to myself or to you.

If I worked for a financial firm then I may have direct knowledge of how firm assets were managed. That may be typical of other firms. If typical, then I could truly be said to know these things (and if not then I may be unknowingly misinformed). I have only traded personal capital to date. With no bona fide industry experience, I won’t pretend to know smart money financial truths about how (or if) the industry manages assets successfully. Most people, like me, have no experience working in the financial industry. Lots of people have spoken to people with conflicts of interest who are disqualified as objective sources of information.

If you are one with industry experience, then you can believe yourself. Because we may not be able to take credibility much farther than that, it is essential that we conduct due diligence with the awareness of what we cannot possibly know (regardless of how much it may seem to the contrary).

My Perspective on Algorithmic Trading (Part 1)

Where oh where should I possibly begin when it comes to algorithmic trading?

Let’s start with a definition. Wikipedia says algorithmic trading is:

      > A method of executing orders using automated pre-programmed trading
      > instructions accounting for variables such as time, price, and volume.

High-Frequency Trading (HFT) is a subset of algorithmic trading. More specifically, HFT is characterized by high speed, co-location, and frequent trading (high turnover). Co-location means trading computers are housed on the same premises as exchange servers to minimize delays and get the best execution (prices). HFT [presumably] uses complex algorithms and [presumably] sophisticated tools to trade.

I say “presumably” because I am unable to directly verify this. You probably can’t, either. We can read articles, blog posts, watch CNBC interviews by “industry insiders,” and potentially talk to industry representatives ourselves [e.g. IARs, quants we meet at dinner parties, or principals of financial firms that we casually encounter (how?)], but most of these people cannot escape inherent conflicts of interest. We are left to choose whether or not to believe (and to what extent). While I do believe select individuals have a bead on the truth, that’s not me and I’m not afraid to admit that to myself or to you. I’ll get back to this in another post because it really never gets old and is always applicable in the world of Finance (lest you doubt, watch more American Greed).

I don’t want to approach anything when it comes to investing or trading without having my critical analysis faculties (i.e. bullshit detector) at the ready. Remember the original post on optionScam.com?

For me, algorithmic trading carries two main implications. First, trading strategies can be programmed and executed by computer. This can free me up to do a multitude of other activities. Second, algorithmic trading takes the emotional component out of the equation. Fear and greed can lead to overtrading or failing to take trades. Algorithmic trading takes trades every single time they’re supposed to be taken.

These implications are not without some very important caveats, and I will begin discussion of these in the next post.

James Cordier: Tragedy or Laughingstock? (Part 3)

Speaking of mass deception, as I did regarding James Cordier and Optionsellers.com, I found a December 2018 press release from a legal firm that states:

     > Cordier marketed himself as a leading market expert. He co-
     > authored a book titled “The Complete Guide to Option Selling.”
     > In May, he penned a bylined article in Futures magazine about
     > the perils of trading in the gas market.

He cautioned about the perils of trading natural gas before getting blown up by it himself?! My eyebrows are raised…

     > In a 2013 lawsuit filed by the U.S. Commodity Futures Trading
     > Commission (CFTC), James Cordier, president of OptionSellers.com,
     > his partner Michael Gross, and former firm Liberty Trading Group
     > were charged nearly $50,000 for improper trading.

Any fines or citation from the CFTC are never good to see and certainly do not engender trust.

     > Separately, FCStone was involved in its own natural-gas options
     > controversy in 2013. As reported in a CFTC notice in May [2013],
     > the CFTC fined the company $1.5 million for a failure “to prevent
     > an unchecked customer from taking grossly excessive risks” and
     > the brokerage ended up with losses of $127 million.*

FCStone was the brokerage used for Cordier’s trades. Again, this is not good to see.

As these details incrementally cast doubt on Cordier’s enterprise, we must not ignore the fact that the legal firm itself has underlying motives: new business through representation of damaged investors.

The press release continues:

     > Investors in this situation should seek legal counsel now to
     > protect their rights and avoid being wiped out twice. Ironically,
     > you have a hedge fund here that didn’t hedge. Worse yet, the

As mentioned in Part 2, this was not a hedge fund but rather separately managed accounts (SMA). I believe hedge fund risk is limited to no more than the total investment. If that is true, then I am surprised to see an attorney make this obvious mistake.

     > brokerage firm that cleared and executed these trades is now
     > seeking to collect an additional $35 million in margin calls
     > from the same investors that just got wiped out … and also
     > putting the squeeze on them to relinquish their legal rights.
     > This has turned into a real double-whammy nightmare for
     > investors in OptionSellers.com natural gas scheme.

Yes on the double whammy, but if all this was agreed and consented to beforehand as SMA (see fourth paragraph of Part 2), then I really don’t see what liability James Cordier has.

I do see some exaggeration on the part of the legal firm here. The waters are muddy, and as with so many stories and lawsuits, it’s hard to know which side (if) is to blame.

* I wonder how much FCStone will be fined this time since Cordier was clearly taking “grossly excessive risks”
   in losing $150M for his clients.

Planning My Next Meetup [hopefully not MIS] Adventure (Part 9)

Surely you didn’t think I was done with potential ways to pitch my option trading group idea, right? Here is one other approach that is altogether different.

I spent some time on Hubstaff Talent the other day looking for freelancers to build my automated backtester. I entered keywords “statistics,” “programming,” and “finance.” I found many candidates had experience pertaining to bookkeeping, rectifying financial transactions, etc., which is nothing that I need. I therefore removed “finance,” thinking that I’d be happy to poach professionals from other fields and teach them what little they may need to know about trading to build the app.

This got me thinking, once again, about starting a group focused on research and backtester building to have this done for free. The results could be used for the trading benefit of all involved. And if I need to teach people how to trade in exchange for a tool that I could ultimately implement for a broad spectrum of use, then I’d certainly be willing to do so. This means putting the focus primarily on tech savvy/programming and secondarily on practicing the trading strategies themselves.

One thing I find sorely missing from the financial industry is hardcore research and testing of trading/investment strategies. From my pharmacy background, this is akin to “evidence-based medicine.” Whether this is needed in finance depends on who you ask. The absence of statistical application [testing] results in demand for good-sounding strategies. One could argue an entire industry has been built upon this with regard to investment advisors, trading educators/mentors, and sale of black box systems. Unfortunately, this absence may also create fertile ground for Ponzi schemes and fraud.

Alas, I digress. Here is the description:

     As a full-time, independent option trader for the last 11 years, I want to pursue
     real-time strategies that have worked in the past when I also think they have
     reason to work in the future (such is never a guarantee). I have spent thousands
     of manual hours backtesting trading strategies. With much more left to study, I
     seek a quicker solution: an automated backtesting app.

     I am looking to assemble of group of people with expertise in statistics, data
     science, and/or programming. Together, we can build an institutional-quality
     tool that can significantly help tailor our individual trading. If you don’t trade
     and would like to learn how, then I’m happy to share everything I have learned.
     I present an alternative view and one that may raise some eyebrows if your
     market education is based on guru talk, financial media, or classic books. This is
     exactly what you may need, though, if you also believe the oft-quoted statistics
     about how 90% of all traders fail within their first couple of years.

     If learning to trade is on your bucket list, then this is your chance to get a fresh
     take from an industry outsider who ironically has more trading experience than
     the vast majority of financial professionals. Let’s do some work, make some
     money, and have a good time establishing Meetup community along the way!

Butterfly Skepticism (Part 2)

The presentations I see on butterfly strategies often get my skeptical juices flowing.

I will not accept edge that occurs at one particular DTE and not another because I think this is one of the great fallacies in all of option trading (see here and here). I find many butterfly strategies to be guilty of this.

From across the mountaintop, the plethora of butterfly trading plans look like an attempt to place Band Aids in all combinations over any potential risk including high/low underlying price, volatility, or days to expiration. I think any advanced trader would recognize this as the Holy Grail or a flat position. The former is a mirage that does not exist. The latter poses no risk with no opportunity for reward. Any statistically-minded trader might recognize this as curve-fitting: trying different things until you find one that works perfectly. Despite that perfect match to past data, curve-fit systems are unlikely to work in the future. Also by [statistical] chance alone, every 20 attempts is likely to produce one success [alpha = 0.05].

Studying butterflies was one of my early reasons for wanting an automated backtester. Of all the butterfly trading plans I have seen pitched, many were accompanied with no backtesting at all (optionScam.com). Some of them have limited backtesting, which I have often found to be vulnerable to criticism (e.g. transaction fees not included, insufficient sample sizes, limited subset of market environments studied, survivorship bias, no OOS validation). I want to be able to take a defined trading plan and backtest it comprehensively.

For different reasons, many butterfly trading plans cannot be backtested. Some are complex enough to fall prey to the curse of dimensionality (discussed and here and here). Some trading plans emphasize multiple positions in the whole portfolio placed over long periods of time to properly hedge each other. Chewing up such a large time interval for a single instance will only allow for a small number of total instances that cannot be divided up in sufficient sample size for the different categories of market environments. Some trading plans are specific enough to be curve-fit (worthless). Other trading plans incorporate discretion. For all practical purposes, discretion can never be backtested. Any discretionary plan thereby becomes a story, which falls into the domain of sales, rapport building, advertising, and marketing (optionScam.com?). I alluded to this in the second-to-last paragraph here.

I have yet to trade butterflies with any consistency because I do not even know if they are better than naked puts, which I consider the most basic of option trades. At the very least, the automated backtester research plan should be able to address this.

Butterfly Skepticism (Part 1)

Before I continue the research plan, I want to express some skepticism I have toward butterfly trades based on what I have seen in recent years.

Butterflies have been a “secret sauce” of the options trading community for some time. They come in all different shapes and sizes with diverse trading plans incorporating varying degrees of discretion. I have seen many traders and vendors describe more/less elaborate adjustment plans that look, at cursory glance, very impressive in their presentation.

Most of these adjustment plans are simply position overlays added later in the trade. Adding [a] management criteria [criterion], giving the trade a fancy name, and selling it as a new idea for $$$ seems alarmingly deceptive to me especially in the absence of valid backtesting to support it.

Most of the complexity amounts to adding subsequent positions within specified time constraints. For example, adding a put credit spread (PCS) later in the trade to raise the upper expiration line may seem appealing. Consider these:


If they happen at all then either is going to be triggered much later than butterfly initiation since decay occurs over time. I strongly believe nothing to be magical about any particular DTE for trade inception or adjustment. Failure to explore any other time a trade could be placed or adjustment made is short-sighted as I have written about here, here, and here.

If I would consider doing the adjustment later, then I should backtest the adjustment earlier as well. With regard to the above example, I already plan to study shorter- and longer-dated PCS as part of the automated backtester research plan. Whether the trading plan works as a whole is simply a question of whether the sum of its parts (e.g. symmetrical butterfly plus shorter-dated PCS) is profitable.

I will continue next time.