Option FanaticOptions, stock, futures, and system trading, backtesting, money management, and much more!

Can a Retail Trader Succeed at Algorithmic Trading? (Part 4)

Today I continue presentation and commentary on an internet thread (subtitled “KD vs. EC”) regarding algorithmic trading that took place about 18 months ago.

     > I think they are both “right” in their own way. If you look at what KD does,
     > it’s a little different from EC’s strategies.

KD gets his customers to work for him (see second paragraph here)! I wonder how many are aware of this fact.

     > The “trading robots” you see on sites like collective2 aren’t serious edges.
     > Only an idiot would sell a serious proven edge to random Internet anons for
     > pennies on the dollar, and idiots don’t develop profitable trading algos.
     > If you have a proven edge, you raise capital to trade it yourself or sell
     > it to a quant fund for serious lump-sum cash and a job.

As I discussed near the end of Part 3, it’s not so easy to just “raise capital to trade it yourself or sell it to a quant fund.” Proving the edge probably means creating an incubator fund (or friends and family fund, if one can be lucky enough to gather a reasonable minimum amount of capital this way) for purposes of trading and auditing. That can cost many thousands of dollars, which singularly establishes a high barrier to entry.

     > Both of them are nice guys, and I’ve complimented them for their contributions,
     > but neither one of them is really “killing it” in this game. I tend to learn a lot
     > more from those with serious (1M+) skin in the game, and I don’t mean OPM.

We don’t know whether either is “killing it,” much less how much “skin in the game” (actual capital deployed) either has (see fourth paragraph here).

     > And there lies the bait. Yes, in theory a large account can withstand large
     > drawdown if low leverage is used, but most people do not have large accounts,
     > and those that do, will pull the plug if drawdown gets to 20 percent or more.
     > I have seen clients closing accounts after 10 percent loss.

This max drawdown is consistent with the kind of performance I have come to believe is necessary to have something of institutional quality.

     > The algorithms the largest banks and funds use depend on infrastructure or size.
     > There’s almost no way an at-home type trader can participate in this. Many
     > markets have large capital requirements — 100s on millions just to enter.
     > For others you need best co-location or contacts in the industry.

I would imagine these algos are of the HFT variety. I don’t know whether this person knows from experience or is just hypothesizing, but s/he makes some really good logical points. The post continues:

     > I’ve been following people who sell signals for a long time. There are very rarely
     > profitable systems but they always go private fast because selling something that
     > can make millions for thousands is not a business. On top of that you have people

Speaking of someone who has been following signal sellers for a long time, I’d like to hear Mark Hulbert’s take on this.

I will continue next time.

Can a Retail Trader Succeed at Algorithmic Trading? (Part 3)

Today I continue presentation and commentary on an algo trading thread that took place about 18 months ago.

     > Ok, wait a minute: every time traders are talking about big players and market
     > manipulation, everyone seems to agree that algos are running the game. So,
     > assuming that this is right – someone stating that 85% of market is driven by
     > them – that means the big banks, players, etc. actually have financial
     > algorithms that work and are profitable.
     >
     > If this is true, why is there, after decades of banks using them, no market for
     > retail traders to buy them, even if it is not the best/newest version of all?!

This is a very interesting point. Question the premise, though: who is to say that many big firms don’t have profitable algos?

Aside from banks, the “etc.” is probably asset managers and hedge funds. With regard to the former, passage of the Dodd-Frank Wall Street Reform and Consumer Protection Act (July 2010) made it illegal for deposit-taking institutions to engage in prop[rietary] trading. That would include banks and many asset managers. With regard to hedge funds, I studied 2008 – 2016 and found them to underperform (see table here). This may leave some smaller firms that attempt prop trading for a period only to realize they can’t make [enough] money at it and quit. I therefore think it likely that the overall universe for prop trading, where algos are being used to generate the huge Wall Street profits of legend, is significantly diminished.

I’m not convinced that “everyone” (who “seems to agree that algos are running the game”) is all that informed—especially in the face of the kind of reasonable doubt just discussed. People repeat what the financial media writes or broadcasts. Does anyone actually check and scrutinize sources or underlying motives? Not nearly enough, I suspect.

One person responded:

     > The reason is only bank and large firms’ algos can work for them such as
     > market making and HFT. Hardware barrier of entry is in 10s of millions.

I can’t verify the claim of eight figures, but I do believe in a large and formidable barrier to entry.

     > What you are looking for is a strategy can can be deployed on retail
     > level. While those exist, no one in their right mind would disclose it.

This would imply the strategies given by KD (see second-to-last paragraph here) are bogus.

     > It is not that difficult to raise funds and trade it, as long as it
     > indeed has positive expectancy.
     >
     > You will not find it anywhere in public domain.

I don’t believe it is ever easy to raise funds for investment. This article claims the financial services industry has a 12% success rate. Another article claims 90% of financial advisors will fail within the first three years. Claims like these along with the personal roadblocks I have discovered around the wealth management industry make me believe raising money is anything but “easy”—a word that I as a marathon runner will never take for granted (injuries!).

And once again: forget the public domain. You will have to uncover it yourself.

I will continue next time.

Can a Retail Trader Succeed at Algorithmic Trading? (Part 2)

Today I continue presentation and commentary on an internet thread regarding algorithmic trading that took place about 18 months ago.

     > …financial machine learning (ML) is virtually impossible to do right because of the
     > large amount of data you need, awareness of bar types and binning, and many
     > other seemingly trivial but hugely important things. To me, it’s borderline
     > worthless to even try… Leave ML to the MMs and big desks.

I took several Datacamp courses on ML but have yet to do any work of my own in this area. That aside, this sounds like an argument with some teeth.

Someone else commented:

     > BTW, I am disappointed with KD as he doesn’t seem to contribute to the forum.

I couldn’t disagree more. One of my biggest pet peeves is a vendor who shows up online to defend themselves anytime someone writes something that even hints toward negativity (KK is the worst). Do they think they save face by trying to discredit everyone who doesn’t portray them in a brilliant light? While I am skeptical toward reviews and testimonials (see here and third paragraph here), I would sometimes rather let these speak for vendors than the vendors themselves.

This strikes me as one of the best comments of the entire thread:

      • Finding a decision rule with positive expectancy (FDRPE) is… [almost] trivial.
      • FDRPE on daily data that goes back 20+ years is (IMO) foolhardy.
      • FDRPE on daily data that may be tweaked every 6-9-12 months to continue that
         positive outcome… may benefit from [ML] pattern-recognition routines… but…
      • ML depends on data stability that is at odds with the very nature of time-series
         phenomena…

To me, this argues for just how difficult trading strategy development really is (see fourth paragraph here).

     > In my tick-scalping days, I put T/A up that looked good for 20-30 minutes; I relied
     > on it for the next minute or two. For trend-exploitation, I put up daily candles
     > that go back maybe 6-12 months, and rely upon it for the next week or so.

This is shocking, but consistent with my own studies. As discussed in the last three paragraphs here, I did not succeed in looking at four years IS followed by four years OOS and then testing on a subsequent four years. Maybe I should just test the first few trades of OOS2 to see if any edge exists.

Another post reads:

     > You are new here, so I will give you some benefit of the doubt. Get it in your head,
     > you cannot find algo with positive expectancy ANYWHERE. The only way to get it is
     > to create it yourself.

KD disagrees. In fact, if you purchase his product then he gives you several strategies that he says still work in addition to his entire strategy development methodology (don’t mistake this for an endorsement).

I will continue next time.

Can a Retail Trader Succeed at Algorithmic Trading? (Part 1)

I recently stumbled upon a thread on a prominent online forum that is roughly 18 months old. I started reading and realized I have some unique perspectives to share since I am actually ankle-deep in algo trading myself and have taken one of the courses discussed (see second-to-last paragraph here).

The reason I am responding here rather than in the forum itself is because the walls have ears. Internet forums and social media are primary vehicles for communication between retail traders, but vendors often frequent these sites as well. On multiple occasions, I have seen someone subscribed to a trading service get banned by the vendor for posting negative feedback in forums and social media. More details would probably be needed to determine whether this is legal (i.e. do they just get cut off once the subscription runs out? Are they banned immediately with pro rata reimbursement for what subscription remains?), but bottom line is that it happens.

Since I have been a customer—and a customer who does not like to burn bridges—I am keeping personal comments to my unmonetized, low-traffic blog rather than posting in the middle of Grand Central Station.

The thread begins:

     > KD, an (apparently) successful algorithmic trader/author claims simple strategy
     > works and will always work.
     >
     > EC, another guru in this field, however, says simple quant strategies don’t work
     > anymore and machine learning is a must if you want to succeed in trading. He also
     > says that it is impossible to do ML-based system trading on your own; you need a
     > team based approach, since it is so [labor] intensive…

Although I only tested ~300 simple strategies the KD way, none came close to working. Yes, he does teach the simpler the better, and the strategies he gives you as part of his course are simple themselves. Because I couldn’t find anything that met his criteria, though, I have to remain skeptical.

I like the EC comment about doing ML-based strategy development with other people. For multiple reasons, I totally believe trading strategy development should be done in groups (see last paragraph here).

The post continues:

     > I believe EC has gone on record saying the strategies in his book are old strategies
     > that don’t work for him anymore. I wouldn’t be surprised if KD did the same in his
     > book. Seems kind of silly to put strategies in your book you are still using. Both…

Right?! The post continues:

     > publications… are seminal works in retail algo trading. KD is one of the few people
     > with a verifiable record in a trading competition (though we sort of take his word
     > he did it algorithmically). I’d listen to what he has to say…

Interesting comments about KD’s credibility. Yes he was probably verifiable at one point. Is he now? We really don’t know. I’ve been writing on this topic going back many years.

I will continue next time.

Review of Python Courses (Part 33)

In Part 32, I summarized my Datacamp courses 95-97. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #98 was Practicing Statistics Interview Questions in Python. This is a very comprehensive course that covers the following topics:

My course #99 was Intermediate Spreadsheets for Google Sheets. Topics covered in this course include:

My course #100 was Practicing Coding Interview Questions in Python. This is probably the most comprehensive and dense course of all. I took a long time getting through this, but the amount of material covered is really incredible. Props to instructor Kirill Smirnov! The course covers:

I will review more courses next time.

Review of Python Courses (Part 32)

In Part 31, I summarized my Datacamp courses 92-94. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #95 was Dimensionality Reduction in Python. This course covers:

My course #96 was Writing Efficient Python Code. Topics covered in this course include:

My course #97 was Machine Learning for Finance in Python. This course covers:

I will review more courses next time.

Review of Python Courses (Part 31)

In Part 30, I summarized my Datacamp courses 89-91. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #92 was Market Basket Analysis in Python. This course covers:

My course #93 was Winning a Kaggle Competition in Python. Topics covered in this course include:

My course #94 was Machine Learning for Time Series in Python. This course covers:

I will review more courses next time.

Review of Python Courses (Part 30)

In Part 29, I summarized my Datacamp courses 86-88. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #89 was a case study in Python machine learning. This course covers:

My course #90 was Ensemble Methods in Python. Topics covered in this course include:

My course #91 was Data Analysis in Spreadsheets. This course covers:

I will review more courses next time.

Review of Python Courses (Part 29)

In Part 28, I summarized my Datacamp courses 83-85. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #86 was Hyperparameter Tuning in Python. This course covers:

My course #87 was Case Studies in Statistical Thinking. Topics covered in this course include:

My course #88 was Analyzing Police Activity with Pandas. This course covers:

I will review more courses next time.

Review of Python Courses (Part 28)

In Part 27, I summarized my Datacamp courses 80-82. Today I will continue with the next three.

As a reminder, I introduced you to my recent work learning Python here.

My course #83 was GARCH Models in Python. This course covers:

My course #84 was Cleaning Data in Python. Topics covered in this course include:

My course #85 was Quantitative Risk Management in Python. This was deep and needs more study. The course covers:

I will review more courses next time.