Short Premium Research Dissection (Part 40)

By way of recap, let’s begin by doing an updated parameter check (see second paragraph here). Get your reading glasses:

The second column shows in what part of this blog mini-series each study is discussed.

This is too complex to be a definitive exercise in trading system development. Numerous independent variables, dependent variables, and criteria of all sorts have been covered. With all the degrees of freedom represented, only a fraction of the total number of permutations are actually explored.

I discussed my concept of trading system development in the paragraphs #2 and #3 below the excerpt here. As mentioned in the second and third paragraphs of Part 11, thousands of different permutations is too complex. The scary thing is that in some instances (e.g. paragraph after graph here and paragraph #2 here where I discussed failure to explore range extremes), I have actually been clamoring for more parameter values to be studied.

I believe each independent variable should be studied separately to understand its impact on performance.* “Independent variable” is easily confused with words like entry/exit criteria, profit target, filters, stops—when in fact they all probably represent degrees of freedom. The total number of permutations is multiplicative across the number of potential values for each independent variable. I tracked this earlier (final sentence of Part 11) before the research went in too many directions with too many inconsistencies and too much sloppiness.

Some of the tested variables are more about position sizing than trade setup. I’m talking about delta selection for long options. Narrower spreads carry lower margin requirements and allow for greater leverage (see discussion beginning here). This makes delta a factor in position sizing, which goes hand-in-hand with allocation: something she does study.

An entire knowledge domain exists to solve the problem of optimization. I have yet to write about this subject.

For now, it will suffice to say that although I think our author has failed to undertake a valid system development process, I do not have the solution to right her ship. The approach seems too multi-dimensional. In my gut, I sometimes feel that not everything needs to be thoroughly explored. For example, why not just stick with 60 DTE rather than having to look at 30, 45, and 75 DTE? This is precisely the rationale for studying daily trades (second-to-last paragraph here), though, as a check to ensure results aren’t fluke. It’s really all about cutting through the laziness.

I will conclude next time.

* This is easier said than done as interaction effects (see this footnote) should also be identified.

Short Premium Research Dissection (Part 39)

Continuing on with our author’s final analysis, she presents two “hypothetical portfolio growth” graphs for each of three delta stops: one with curves for 5% and 10% allocation and one with a curve for 25% allocation. Y-axis values and derived percentages are therefore relevant (see second-to-last paragraph here). None of these final graphs has a referent for the asterisk,* which we now know corresponds to the “hypothetical computer simulated performance” disclaimer shown here.

For each delta stop, she provides the usual table that falls short of the standard battery (second paragraph):

Similar to tables included previously (e.g. Part 15 and Part 20), a couple differences are noteworthy. First, she includes only the biggest rather than top 3 drawdowns (discussed in paragraph #2 below excerpt #1 here). This is characteristic of all tables in this final section (along with the lost 2007-2009 data, which was discussed in these final three paragraphs).

The other difference is inclusion of overall return rather than CAGR. As described in the paragraph below the first table here, things are sloppy when inconsistent from one sub-section to the next. This is the first time we are seeing “overall return.” Also as discussed in that same paragraph, the critical analyst should ask “why” when something changes without explanation. It’s no big deal here either, though. While overall return impresses more (larger in magnitude), CAGR works fine.

I have been calculating CAGR/MDD throughout this mini-series (e.g. Parts 33, 25, 22, 21, 20, 15). To convert from overall return, I’d have to approximate the backtesting interval (she never gives us the exact dates). I could then calculate CAGR/MDD, although it would not be comparable to previous sections due to the unexplained lost data.

Another source of significant sloppiness is passive disappearance of the VIX filter. The VIX filter was used in generating the final graph of the previous section (shown here). Like the lost data, the VIX filter has been absent in these “most up-to-date trading rules.” If the filter only comes into play for the 2008 crash, then it may represent curve fitting. Some explanation should be given for its sudden omission to preserve our author’s credibility.

With regard to sizing this strategy per individual risk tolerance, she unfortunately does not backtest an expanded parameter range (Part 38, paragraph #2) to help us truly understand allocation limits.

In the final sub-section, she presents a recent trade that probably serves a marketing purpose more than anything else. It’s always nice to hit the profit target after only five days. Beginning November 2, 2018, it probably was not included in the historical backtesting, which is fine (less fine is the omission of backtesting dates). I think something current stokes confidence more than something stale. I often wonder how many people click to order reports, trading systems, trader education products, etc., from websites with content a few years old at best. The graphs always look good. Only when you look close and dig deeper are you well-poised to identify errors and expose the fiction.

I will begin to wrap things up next time.

* The rare +1 she scored in Part 38 paragraph #4 is effectively offset.

Short Premium Research Dissection (Part 38)

In paragraph #2 below the first table, I said I liked our author’s exploratory backtest to assess the [MFE] distribution.

What I don’t like about this approach is a failure to explore the extremes. I mentioned this in the paragraph after the graph here. Backtesting over a range where results are directly proportional is of limited utility. Backtesting over an expanded range can illustrate floor and ceiling effects, which defines the profitable range. With regard to delta stops (profit target), I would have liked to see 0.60 (2.5%) along with 0.75 – 0.80 (10% – 15%) rather than just 0.65 and 0.70 (5% and 7.5%).

Not only is study of the extremes* useful, I can argue for it to be essential. Exploring the tails can help us understand whether we have a normal, thin-, or fat-tailed distribution. I could imagine our author giving the excuse she wanted to avoid “overwhelming” us with lots of excess data (second paragraph below table here). It’s not excessive, though, and without it we have no reason to think she actually checked it herself. Not checking would be sloppy, superficial research indeed.

In the next sub-section, our author discusses trade sizing and the commission benefits of index vs. ETF trading. Once again (as discussed near the end of Part 36), she promotes a brokerage, which signifies a clear conflict of interest. Discussing commissions but not slippage is, if you think about it, very sloppy. These are the two biggest components of transaction fees, but slippage likely dwarfs commissions. The only place slippage is even mentioned is the “hypothetical computer simulated performance” disclaimer shown here and included in all such graphs this section (+1 on consistency, for once).

The next sub-section is titled “final strategy backtests… with various allocations.” Are we now going to see what trade guidelines have made the final cut?!

No.

She proceeds by showing us “hypothetical performance growth” graphs and cursory trade statistics for a 5% profit target managed at 30 DIT or delta stops of 0.60, 0.65, or 0.70 with allocations of 5%, 10%, or 25%. She then tells us to decide based on our own individual risk (drawdown) tolerance.

For each of the three delta stops, she gives the following [incomplete] methodology:

     > Expiration: standard monthly cycle closest to 60 (no less than 50) DTE
     > Entry: day after previous trade closed
     > Sizing: 5%, 10%, 20% Allocations [emphasis mine]

This is a typographical error: 20% should be 25%. As with the first paragraph below the first excerpt of Part 36, maybe the proofreader fell asleep? Sloppiness has been a recurrent theme throughout.

     > Management: exit after 30 days, 5% profit, or at ±0.60 short delta

I will continue next time.

* Incidentally, backtesting over an expanded range would preclude the need for an exploratory
   test to determine MFE distribution.

Short Premium Research Dissection (Part 37)

In this “most up-to-date” section (see end Part 31), our author has explored time stops, delta stops, and now profit targets.

As a prelude to determine what profit targets are suitable for backtesting, she gives us this table:

Recall from the end of Part 35 that when something changes without explanation, the critical analyst should ask why. She did not include the 0.65-delta stop last time: why? In this case, it’s no big deal. I like backtesting over a parameter range and having three values across the range is better than two. If only for the sake of consistency, I would have liked to see 0.65 data both times because it feels sloppy when things change from one sub-section to the next.

Adding to the sloppiness, she is once again lacking in methodology detail. I like the idea of probing the distribution to better understand where profit might land. If we can’t replicate, though, then it didn’t happen. Did she backtest:

• Daily trades?
• Non-overlapping trades?
• Winners?
• Winners and losers?
• Any [combinations] of the above?

These factors can all shape our expectations for sample size (which determines how robust the findings may be) and magnitude of averages (e.g. winners will have a higher average max profit than winners and losers. See Part 24 calculations).

She writes:

     > Based on the above table, it appears profit targets
     > between 5-10% seem reasonable to test for all of the
     > approaches except the ±0.60 delta-based trade exits.

She then proceeds to give us “hypothetical portfolio growth” graph #17 with [hypothetical computer simulated performance disclaimer and] 30 DIT, 5% profit target or 30 DIT, and 10% profit target or 30 DIT.

Next, she gives us a “hypothetical portfolio growth” graph and table each for 0.65- and 0.70-delta stops. The graphs are all similar with no allocation, no inferential statistics, and nebulous profitability differences. The tables take the following format:

This falls far short of the standard battery and also lacks a complement of daily trade backtesting (see fourth-to-last paragraph here). We still don’t know her criteria for adopting trade guidelines, either. I therefore like her [non-]conclusion:

     > After analyzing the various approaches and management
     > levels, it seems you could pick any one of the variations
     > and run with it. Consistency seems to be more important
     > than the specific numbers used to trigger your exits.

She also writes:

     > Interestingly, not using a profit target with the ±0.70
     > delta-based exit was the ‘optimal’ approach historically.

For this reason and because she did not test [all permutations of] each condition[s], I remain uncertain whether a delta stop is better than any or no time stop. I would say the same about profit target: too much sloppiness and too few methodological details [and transaction fees as discussed at the end of Part 36] to know whether it should make the final cut.

I will continue next time.

Short Premium Research Dissection (Part 36)

Picking up right where we left off, our author gives the following [partial] methodology for her next study:

     > Expiration: standard monthly cycle closest to 60 (no less than 50) DTE
     > Entry: day after previous trade closed
     > Sizing: one contract
     > Management 1: Hold to expiration
     > Management 2: Exit after 30 days in trade (10 DIT)

I think this should be “30 DIT.” The proofreader fell asleep (like third-to-last paragraph here).

Here is “hypothetical portfolio growth graph” #15:

The two curves look different; a [inferential] statistical test would be necessary to quantify this.

She concludes the time stop results in smoother performance and smaller drawdowns. Once again, I’d like this quantified (e.g. the standard battery). I’d also like a backtest of daily trades (rather than non-overlapping) for more robust statistics.

Next, she adds delta stops:

     > Management 1: Hold for 30 days (30 DIT)
     > Management 2: Exit when short call delta hits +0.60 OR the short put delta hits -0.60
     > Management 3: Exit when short call delta hits +0.70 OR the short put delta hits -0.70

I like the idea of a downside stop, but I question the upside stop. With an embedded PCS, we are already protected from heavy upside losses. She does not test downside stop only.

Here is “hypothetical portfolio growth” graph #16:

The blue curve finishes at the top and is ahead throughout. The black curve finishes on top of the green, but only leads the green for roughly half the time. Inferential statistics would help to identify real differences.

Thankfully, each of the last two graphs are presented with that “hypothetical computer simulated performance” disclaimer (see second paragraph below graph in Part 34).

Here are selected statistics for graph #16:

As discussed in the second-to-last paragraph here, percentages are not useful on a graph without allocation. I think relative percentages can be compared, however, when derived on the same allocation-less backdrop.

Unfortunately, we have no context with which to compare total return. As mentioned above, we’re lacking the standard battery (second paragraph) and complement of statistics for daily trades.

As with Part 34, other glaring table omissions include average DIT (to understand impact of delta stops), and PnL per day.

Transaction fees (TF) could adversely affect the delta-stop groups because they include more trades. Our author now mentions fees for the first time:

     > …many more trades are made… [with] delta-based exit… we need to
     > be considerate of commissions… [At] $1/contract, the ±0.60 delta exit
     > would… [generate] $2,168 commissions… if you trade with tastyworks,
     > the commission-impact of the strategy could be substantially less…

Unfortunately, she mentions fees only to give a brokerage commercial. Her affiliation with the brokerage is clear because she offers the research report free if you open an account with them. In and of itself, this conflict of interest would constitute a fatal flaw for some.

Discussing commissions but not slippage is sloppy and suspect. It is sloppy because as a percentage of total TF, slippage is much larger than commissions. It is suspect because neither is factored into the backtesting, which makes results look deceptively good.

Thus ends another sub-section with nothing definitive accomplished. She may or may not include time- or delta-based stops in the final system and if she does, then she has once again failed to provide any conclusive backing for either (also see third paragraph below first excerpt here).

Short Premium Research Dissection (Part 35)

Finishing up the sub-section from Part 34, our author writes:

     > At the very least, when incorporating short premium options
     > strategies… it may be wise to implement time-based exits
     > to avoid large/complete losses and reduce portfolio volatility.

By not stating definitive criteria, she [again] tells us nothing conclusive. As discussed in the second paragraph after the first excerpt here, we’ll have to wait to see if a time stop makes the final cut.

Unlike the 50% profit trigger for rolling puts to 16 delta, I think the time stop is beneficial. I wish she had shown us the top 3 drawdowns as done previously (e.g. Part 31 table). Seeing improvement on each of the top 3 would be stronger support for time stops than improvement on just the worst. Even better would be the entire standard battery (second paragraph).

Unfortunately, we do not know how time stops would impact the high-risk strategy, which was backtested before 2018.

I can’t help but wonder why time stops were not studied before 2018 (final excerpt Part 31). Admittedly, I would have suspected curve fitting had she written “due to the horrific drawdown suffered in Feb 2018, I tried to add some conditions to make the strategy more sustainable.” Not giving any explanation makes me suspect the same. I think the best approach to avoid being influenced by disappointing results is to plan the complete research strategy in advance (second paragraph below excerpt here). Exploring the surrounding parameter space (see second paragraph here) is critical as well.

     > The next trade management tactic we’re going to explore
     > is the idea of ‘delta-based’ exits.

Like time stops, this is another completely new idea for her. As just discussed, this ad hoc style bolsters my suspicion (see third paragraph below excerpt in Part 30) that she spontaneously tosses out ideas with the hope of finding one that works. Given enough tests, success by chance alone is inevitable. “Making it up as we go along” to cover for inadequate performance is a flawed approach to system development.

     > To verify the validity of using delta-based exits, let’s look
     > at some backtests to compare holding to expiration, adding
     > a time-based exit, and then adding a delta-based exit.

Layering conditions is confusing because order may matter* and all permutations are not explored (see second and third paragraphs here). For example, she does not study a delta-based exit without a time-based exit. I would rather see each condition tested independently with those meeting a predetermined performance requirement making the final cut.

     > First, let’s compare… holding to expiration… [with]…
     > exiting after 30 DIT.

Where did 30 come from? The exploratory studies (Part 34) looked at 10, 20, and 40 DIT.

When something changes without explanation, the critical analyst should ask why. This is becoming a recurring theme (as mentioned hear the end of Parts 32 and 33).

I will continue next time.

* In statistics, this is called interaction.

Short Premium Research Dissection (Part 34)

Continuing with exploration of time stops, our author gives us hypothetical portfolio growth graph #15:

As discussed last time, this is based on one contract traded throughout. The hypothetical portfolio is set to begin with $100,000. This is fixed-contract position sizing. One consequence of fixed-contract versus fixed-risk (fractional) is a more linear equity curve rather than exponential. As discussed [here], the latter reaches higher and looks more appealing despite having greater risk. I have traditionally been a proponent of omitting position sizing from backtesting to allow for what I thought would be apples-to-apples comparison of drawdowns throughout (see here). In these graphs without any allocation, position sizing is effectively eliminated from the equation.

Accompanying the graph is that disclaimer about hypothetical computer simulated performance. This was a big deal earlier in the mini-series when I discussed the asterisk following the y-axis title “hypothetical portfolio growth” (second paragraph here). The disclaimer appeared for the first time in Part 16 and alleviated many pages of confusion. After that, the disclaimer disappeared leaving the asterisk without a referent until the two most recent graphs.

Score a point for completeness—however transient it may be—rather than footnote false alarm and sloppiness.

I can’t discern much with regard to differences on this graph. Final equity is ~$130,000 for all. 40 DIT > 10 DTE > 20 DTE for most of the backtesting interval, which suggests these differences might be significant were inferential statistics to be run.

She gives us the following table:

This is not the first time she tells us number of trades, but she falls far short of reporting it every time. Number of trades range from 50 – 243 over roughly eight years. That is ~6 – 30 trades per year or ~0.5 – 2.5 trades per month. On their own, these aren’t tiny samples. Backtesting one starting every trading day (e.g. second paragraph below graph here), though, would give a sample size in the thousands. I think that would be a useful complement to what we have here.

Glaring omissions in this table include average DIT (for the expiration group), total return or CAGR, and PnL per day. A big reason for using a time stop is to improve profitability (either gross or per day): show us! Time stops aim to exit trades earlier: show us [how long they run otherwise]! Nothing is conclusive without these.

Another big shortfall is the exclusion of transaction fees. Number of trades varies 2-5x across groups. The fees could add up.

I would still like to see that lost data [discussed last time] back to 2007.

On the positive side, the table does a decent job of showing performance improvement with max DD and max loss if we assume equal total return as suggested in the graph.

Short Premium Research Dissection (Part 33)

I continue today with the second-to-last paragraph on allocation.

All graphs from previous sections assume allocation. Some graphs study allocation explicitly (e.g. Part 20). Others incorporate a set allocation to study different variables (e.g. 5% in Part 18). Return and drawdown (DD) percentages may be calculated from any of these allocation-based graphs.

I remain a bit uneasy about the fact that so many of the [estimated] CAGRs seen throughout this report seem mediocre (see fifth paragraph here). I am familiar with CAGR as it relates to long stock, which is why I have mentioned inclusion of a control group at times (e.g. paragraph below graph here and third paragraph following table here).

While [estimated] CAGR has me concerned, CAGR/MDD would be a more comprehensive measure (see third-to-last paragraph here). Unfortunately, I am not familiar with comparative (control) ranges for CAGR/MDD on underlying indices, stocks, or other trading systems. Unlike Sharpe ratio and profit factor—metrics with which I am familiar regardless of market or time frame—I rarely see CAGR/MDD discussed.

The larger takeaway may be as a prerequisite to do or review system development. I would be more qualified to evaluate this research report were I to have the intuitive feel for CAGR/MDD that I have for Sharpe ratio and profit factor.*

With regard to the Part 32 graph, our author writes:

     > The two management approaches were profitable… Holding trades
     > to expiration was an extremely volatile approach, while closing
     > trades after 10 days… resulted in a much smoother ride.

That [green] curve looks smoother, but volatility of returns cannot be precisely determined especially when four curves are plotted on the same graph. This is a reason I promote the standard battery (see second paragraph of Part 19): standard deviation of returns and CAGR/MDD are the numbers I seek. Inferential statistics would also be useful to determine whether what appears different in the graph is actually different [based on sample size, average, and variance].

Now back to the teaser that closed out Part 32: did you notice something different between that graph and previous ones?

For some unknown reason, we lost three years of data: 2007-2018 in previous graphs versus 2010-2018 in the last.

This “lost data” is problematic for a few different reasons. First, 2008-9 included the largest market correction in many years. Any potential strategy should be run through the period many people consider as bad as it gets especially when the data is so easily available. Second, inclusion of system guidelines thus far has been made based on largest DDs and/or highest volatility levels: both of which included 2008 [this isn’t WFA]. Finally, when something changes without explanation, the critical analyst should ask why. Omitting 2008 from the data set could be a great way to make backtested performance look better. This would be a curve-fitting no-no, which is why it raises red flags.

* This is a “self-induced shortcoming,” though. Any mention of CAGR/MDD in this mini-series comes
   from my own calculation (e.g. second paragraph below table in Part 20). Our author makes no
   mention of these while omitting many others as well.

Short Premium Research Dissection (Part 32)

Early in Section 5, our author recaps:

     > In the previous section, we… constructed with a 16-delta long put…
     > and long 25-delta call.
     >
     > The management was quite passive, as trades were only closed
     > when 75% of the maximum profit… or expiration was reached.

We now know the rolling was not adopted after all (see second paragraph below first excerpt here).

     > …achieving 75% of the maximum profit potential is unlikely… we
     > are holding many trades to expiration, or very close to expiration.
     > That’s because if the stock price is… [ATM]… most of the option
     > decay doesn’t occur until the final 1-2 weeks before expiration.

I addressed this in the last paragraph here.

     > On average, trades were held for 50-60 days, which means few
     > occurrences were traded each year and the performance of those
     > trades sometimes swung wildly near expiration.

We finally get an idea about DIT (part of my standard battery as described in this second paragraph). She also admits use of a small sample size, which backs my concern about a couple trades drastically altering results (e.g. third-to-last paragraph of Part 29). In the paragraph below the first excerpt of Part 31, I talked about that high PnL volatility of the final days.

     > To solve the issue of volatile P/L fluctuations when holding…
     > near expiration, we can… [close] trades… after a certain
     > number of calendar days in the trade. I’ll refer to this as
     > Days In Trade (DIT).

Yay time stops! I suggested this in that last paragraph of Part 25.

For the first backtest of time stops, she gives us partial methodology:

     > Expiration: standard monthly cycle closest to 60 (no less than 50) DTE

In other words, trades are taken 50-80 DTE. I would like to see a histogram distribution of DTE since these are not daily trades (see Part 19, paragraph #3).

     > Entry: day after previous trade closed
     > Sizing: one contract

Number of contracts is new. This makes me realize she changed from backtesting the ETF to the index. It shouldn’t matter one way or another, but when something changes without explanation, the critical analyst should ask why.

     > Management 1: hold to expiration
     > Management 2: exit after 10 DIT

She gives us hypothetical portfolio growth graph #14:

She writes:

     > Note: Please ignore the dollar returns in these simulations.
     > Pay attention to the general strategy performance. The last

I have wondered about how to interpret these graphs since the second one was presented at the end of Part 7.

     > piece we’ll discuss is sizing the positions and we’ll look at
     > historical strategy results with various portfolio allocations.

This suggests the graph is done without regard to allocation. With the strategy sized for one contract, I’m guessing the y-axis numbers to be arbitrary yet sufficient to fit the PnL variation. This also means I cannot get meaningful percentages from the graph. As an example, the initial value could be $100,000,000 or $100,000 and still show all relevant information. The percentages would differ 1000-fold, however.

I will now leave you with the following question: as seen above, what unexplained change was made to the graph format that frustrates me most?

Short Premium Research Dissection (Part 31)

Our author concludes Section 4 with a study of different profit targets. She briefly restates the incomplete methodology and gives us hypothetical portfolio growth graph #13:

She gives us the following table:

My critiques should be familiar:

• No sample size given
• No inferential statistics provided
• CAGR not given to reveal profitability
• Uncertain DD improvement with four of 18 comparison results unexpected (see second paragraph below table here)

She writes:

     > …the downside of waiting until 50% profits is that… long put
     > adjustment might not be made at all (which was the case 57%
     > of the time), which means the trade’s risk was not reduced. This
     > explains why the 50% profit… trigger suffered a very similar
     > drawdown compared to… no trade adjustments.

My comments near the end of Part 27 once again apply. This adjustment helps only in case of whipsaw. With the 50% profit trigger, that whipsaw would have to occur inside 4 DTE. If I’m going to bet on extreme PnL volatility, then I would bet on the final four days. However, based on my experience the higher probability choice would be to simply exit at 4 DTE.

I don’t think rolling the put to 16-delta should be part of the final trading strategy. Doing so looks to underperform as shown in the Part 27 graph. If rolling is adopted then, in looking at the above graph, is there any way a 50% profit target adjustment trigger (as opposed to 25%) should not be adopted as well? This would be something to revisit after Section 5 when the completely trading strategy is disclosed.

Criteria for acceptable trading system guidelines should be determined before the backtesting begins as discussed in the second paragraph below the excerpt here. By defining performance measures (also known as the subjective function) up-front, whether to adopt a trade guideline should be clear.

Let’s move ahead to the final section of the report. Our author writes:

     > In late 2018, I dove back into the research to analyze
     > trade management rules that would accomplish two goals:
     >
     > 1) Reduced P/L volatility (smoother portfolio growth curve)
     > 2) Limit drawdown potential

These should be goals of any trading strategy. Certainly #2 was a focus before because she has talked about top 3 drawdowns throughout the report. I have spent much time discussing it.

I have been clamoring to see standard deviation (SD) of returns as part of the standard battery (see second paragraph here) throughout this mini-series. SD would be a measure for #1.

     > In this section, you’re going to learn my most up-to-date
     > trading rules and see the exact strategy I use…

What I do not want to see are additional rules added only to make 2018 look better. That would be curve fitting.

I will go forward with renewed optimism and anticipation.