Get The Ticker Tape delivered right to your inbox.


Backtesting Stinks (And Why You Should Do It)

Share Print
January 1, 2011
back testing OnDemand thinkorswim
Fredrik Broden

Many years ago, I watched a late-night infomercial hosted by a “trader” with the requisite fast-talking rap and wisps of gray hair that suggested survival in one of the world's toughest businesses. His spiel boiled down to: “Over the past 1 0 years, if you had bought an XYZ contract on November 4 and sold it on November 6, you would have had a winning trade 10 times out of 10.”

Since it was late October, I figured, what the heck? I'll wait a few days, buy one of these things near the open on November 4, and sell it at some point on November 6 if it's profitable. And I'll use a stop order in case the guy is wrong.

I was a professional options trader at the time, learning how to survive by managing risk and paying bills with positive time decay (strategies involving selling premium, not buying). I wasn't much of a directional trader, and although I knew what back-testing was, I was already making money trading options without it. And back then, I didn't have the suite of free back-testing tools on the thinkorswim® from TD Ameritrade platform. If you haven't heard of it before, back-testing is the evaluation of a particular trading strategy using historical data. For example, say you have an idea for a trading strategy, such as “buy XYZ on November 4 and sell it on November 6,” or “buy XYZ every time its price drops 10 % lower than the previous month's high and sell when it rises 20 % after that.” Then you use historical price data to figure out whether, if implemented, the strategy might have made or lost money in the past. Sounds good, right?

Where Do We Go Wrong?

The holy grail for a lot of traders is to find some trading strategy that they don't have to think about. They want to roll out of bed in the morning just before the opening bell, press the “Go” button, and watch the profits flow in. And some of them think that the way to find that holy grail is to pore over historical price data, looking for some pattern that will consistently repeat itself. In other words, back-testing.

There's nothing wrong with the idea of back-testing, but traders can get sloppy and form unrealistic expectations if they're careless. The cardinal sin in this quest for the grail is known as curve fitting, and it is an unrelenting temptation for the back-testing crusader. Curve fitting happens when you start with an idea, such as the down 10 %, up 20 % scenario. But in looking over historical data, you might notice that buying XYZ after it had dropped 11 % from the previous month's high, and selling it when it rises 18% after that, made more money. So, you adjust the parameters of your strategy in an attempt to maximize its profits. You're assuming that the future will look just like that data you're testing. But a guarantee that XYZ will rise every November 4 through November 6 is exactly what the guy on TV was selling.

So, what happened to my trade? I bought XYZ on November 4, and at the open of trading on November 6, the trade was a loser. It came back somewhat during the day, and I sold it near the close for a small loss. But what happened to the 10 out of 10? Where did I go wrong? One of my mistakes was that I listened to a guy on TV. Another one was that the trade was a big departure from how I was trading at the time. I wasn't familiar with XYZ. I wasn't a pure directional trader. I didn't do my homework on the trade.

And what was infomercial man's mistake? Curve fitting. He assumed that three specific trading days in the future would act just as they had in the past 10 years. What he did was back-test a strategy and look only for the cases in the past that showed winning results. How about October 4 through October 6? Or July? Nope. Just three specific days in November.

Another problem with back-testing is that it doesn't take into account specific market conditions that are occurring at the time of the trades, such as news events, corporate events, overall market sentiment, and so on. In addition, back-testing assumes that you will take action without fail when a signal is given. But what if the market is crashing when the buy signal comes? A lot of traders would be fearful of following the rules in that scenario. Back-testing doesn't reflect such emotions.

It's not enough to create a trading strategy by back-testing some assumptions about market behavior in any old product in order to find a specific set of parameters that might have made money in the past. Even if the back-testing results were profitable, how do you know it wasn't random? Remember, a broken clock is still right twice a day.

thinkorswim ondemand


OnDemand can be set back to any date from December 07, 2009. Streaming quotes and charts move along with the market, tick by tick, as it happened that day. Source: thinkorswim from TD Ameritrade. For illustrative purposes only.

How's Your Real-Time Reflex?

Now, back-testing with end-of-day data is nice, and you can do it on the thinkorswim from TD Ameritrade platform anytime with thinkBack on the Analyze tab. But what would you do if the market tanked more than 5 % intraday, only to come roaring back by the end of the day? (Flash crash, anyone?) One emerging way of using historical data is to replay intraday data to see how your positions do—as well as how you react. Most back-testing programs that use end-of-day data assume that you'll hold the position no matter what happens intraday. But most people don't do that. They watch their positions, monitor their P/L, and sweat when things go against them—in the middle of the trading day. So, how do you test your reaction to huge intraday price volatility?

That's why thinkorswim from TD Ameritrade provides OnDemand. To get it started, click on the “OnDemand” button in the upper right-hand corner of any page of the trading platform. The OnDemand tool lets you replay all the data, tick by tick, for any day from December 7, 2009, onward. Think of it as TiVo for traders. Except, instead of watching any TV show you want, you can go back and watch any stock, index, future, or option on any trading day, tick by tick, as it really happened. And you can trade right along with it, using fake money.

OnDemand gives you a virtual account with a fake balance of $100,000, so you can enter simulated trades based on the replayed prices and see the P/L change. You can fast-forward and rewind the data if you want. Click on a bid or ask to buy and sell, just as you would trading the real deal. When you're done, click the “OnDemand” button again to go back to your real, live trading account. thinkOnDemand is available 24/7, so you can test your trading skills and strategies with replayed live data at any time.

The truth is, even though you might think you would have held on to a real position based on end-of-day statistics, in live trading you'll be faced with potentially large intraday losses and volatility. OnDemand gives you a way to see whether large intraday price moves might have scared you out of a position. Unless you plan on putting on a trade and going to sleep, testing your strategy with OnDemand can help you prepare for the ride.

Thanks to modern technology, back-testing for everyone is here to stay. But remember, use it to support your trading strategies—not to create them. And do it for free with thinkorswim from TD Ameritrade.