Does Technical Analysis Work? Here’s Proof! (Part 4)

Today I continue with commentary and analysis of Janny Kul’s TDS article with the same title.

I was a bit confused where we left off. Kul continues:

     > It appears as though there may be Alpha reversing filtered technical
     > indicators… We’d need to keeping [sic] rolling this forwards to
     > actually find if this relationship continually holds.

I think he’s basically suggesting we test the worst performers from the training set for outperformance. That is a very interesting idea. I would want to know if the worst training indicators do better on the test set than the best training indicators. This reminds me of the Callan Periodic Table of Investment Returns, which I mentioned in the middle of this post.

     > Obviously adding transaction costs and bid/offer would mean we can’t…
     > capture this but this does give us something to investigate further.

Does he mean we can’t realize any profits from this or just diminished profits? He could have included sample transaction fees to get more clarity on this.*

He then teleports ahead to Bitcoin. Say whaaaaaat? Speaking of transaction fees, though, exactly what vehicle is being used to trade it and what are the usual slippage and commissions to do that? I (and most veteran investors, probably) would be very interested to know since Bitcoin is relatively new.

     > So our train period has a monthly average of 20.4% and our test period
     > has annualised returns of 14.3%…it appears as though there may be
     > some Alpha on all technical indicators for Bitcoin.

That sounds encouraging…

     > Interestingly in our train period we outperform Bitcoin but in the test
     > period Bitcoin outperforms.

If buy-and-hold outperforms, then the indicators have no alpha. Why did he just say otherwise?

     > In order to say with certainty if this relationship holds we’d again
     > need to test again over a longer period of time.

Kul then repeats the backtest for all 12 months of 2018. This extends the backtest by five months since the first six months were the training set and July was the testing set.

     > I think it’s fairly safe to say that the performance of all the
     > indicators decays over time however we do actually outperform
     > buying and holding Bitcoin (although, granted, 2018 was a terrible
     > year for Bitcoin).

I think it’s fairly safe to say we really can’t make any conclusions over such a short period of time where the results are so inconsistent with what we saw before.

Kul concludes:

     > We found… reversing filtered indicators may have Alpha for non-
     > Bitcoin instruments and for Bitcoin… our regular indicators
     > may have… Alpha although it does severely decay over time.

Indicator performance declined over the course of these several months, which is still a short time interval. I wouldn’t generalize to “over time,” which sounds much more substantial.

     > We’d need to test on a much larger data set to see if these
     > relationships do actually hold.

Kul catches himself here and I totally agree. Indeed, the biggest critique I have of this article is the limited backtesting interval. Although he uses a 5-minute time frame, the total study period is one year or less. In case we are looking at a large sample size, Kul could have boosted credibility by reporting number of trades in each group, which he never mentions.

In the final analysis, I can’t help but respond to Kul’s title with “Where’s the beef?”

I will continue next time.

* — I feel strongly about including transaction fees in backtesting as discussed in paragraphs 2-3 here.