You are backtesting stock trading strategies the wrong way

Yes, that's right that you probably have been backtesting your stock trading strategies the wrong way. But it isn't your fault. It is the limitation that most backtesting software available in the market imposes on retail traders today.

Let's look at how typically a backtesting software works.

  1. Some software lets you define a limited pre-defined set of stock symbols to backtest.
  2. Then you define the entry condition(s) and exit condition(s)
  3. You run the backtest to get the performance metrics on a particular stock symbol or a portfolio of pre-defined symbols.

At first look, it may seem like a perfectly good way to backtest stock trading strategies. But there are fundamental issues with the above, which pretty much makes backtesting the above way misrepresent what happens in actual trading. More often, traders end up seeing very different results than the backtests.

Let's dive deep into understanding the flaws of backtesting strategies the way above that most software available in the market lets you do.

The problem when we look at performance metrics of backtests on a single stock or a small universe of pre-defined symbols tells us how the strategy behaved in those stocks in the past. But then, it doesn't give us any information on how the strategy would have behaved outside those symbols. Why is this important? Because of the fundamental adage of the market that "past performance is not indicative of future results." So the information contained in the past behavior of a single stock or a small subset of stocks is not very useful in determining how the strategy would perform on the same set of stocks in the future. If it were, then trading would be the easiest game of prediction!

A related problem to the above is that when we backtest with a single stock or a small universe of pre-defined symbols, it is very easy to overfit a strategy by adding conditions after-the-fact that fit well to the historical data to give good-looking performance results. And this is almost guaranteed not to work in actual trading unless the stocks repeat the price path precisely in the future. This is not possible and even if it happens, it is only a rare fluke.

Another problem is that the above approach distorts the objective of the backtesting. Backtesting is not to derive an absolute value of past performance. Instead, the objective is to get an idea of the distribution of performance across the entire universe of eligible stocks and different market regimes. This is important because we don't want backtesting to give us a binary outcome of whether a strategy works or doesn't work on a pre-defined set of symbols in the past - this would be a distortion. Instead, we want to understand the edge of a strategy, i.e. the probability of where and when a strategy works and where and when it doesn't, so we can apply that knowledge to pick strategies appropriately going forward.

So then, can't we just run backtesting on the entire universe of stocks historically by specifying the full list of stocks statically in the universe of existing backtesting software?

There are a few issues with that. First, that's not traders' trade in actual trading. One doesn't go into the market saying that let's trade anything and everything. Instead, the most important component of trading a strategy is identifying the preconditions that need to be met before applying the strategy. These are typically referred to as setups. Unless a symbol doesn't meet the setup criteria, one wouldn't want to take any entry positions.

A simple example is that one wouldn't want to take a long position using a trend-following strategy on a stock trending down. So the up-trend direction is a setup criterion to select stocks for which appropriate low-risk entries are applied in a trend-following strategy. So by nature, these preconditions make the universe of stocks where a strategy can be applied dynamically and cannot be provided statically in a backtest.

What is the correct way to backtest, then?

Backtesting software needs to support dynamic setups. It isn't just sufficient to provide rules only for entry and exit but also for the setups in the backtesting workflow.

Dynamic Universe Selection with Setup Rules on

Having a dynamic universe does a few things:

  1. It ensures that backtesting mimics real-world trading workflow where traders scan for trade setups within the universe of stocks.
  2. It ensures that traders don't miss out on opportunities that are setting up outside a hardcoded list of stocks. Since the backtesting environments are extended to trading systems, it imposes serious limitations if the backtesting and the live trading system support only a narrow/hardcoded list of stocks.
  3. It ensures that many scenarios are covered in the backtesting itself, as it is difficult to overfit a dynamic universe of stocks unless they are all extremely highly correlated (in their returns), in which case the trade setups rules may be too strictly defined.
  4. It provides a good idea of the boundaries of a trading system, where it works and where it doesn't.

Why most vendors don't support this?

It is computationally intensive and data-heavy to backtest workflow rules of setup, entry and exit flows across thousands of stocks across years and decades of data. This requires a cluster-computing setup and typically on an auto-scaled cloud infrastructure to support this. Most backtesting and trading software available to retail traders are downloadables designed to run on the trader's workstation. These don't scale for the real workflow. And the limited few that are cloud-based are all targeted toward programmers and require writing code. This is not useful for most retail traders who are not coders.

How helps traders? was started to solve this very problem. We create a platform designed to scale on the cloud for heavy-duty backtesting and live trading that enables traders to do things properly. Also, we create a simple rule-builder UI that enables traders to test various hypotheses without having to write any code. We even offer various starter trading system templates for new traders to learn and get started, which they can further adapt to fit their own personality and style. Traders can also analyze historical results of the backtests across different timeframes and market regimes to identify what works and what doesn't to help them form better hypothesis.

Once a trader is satisfied with their trading system they can deploy it to get alerted on real-time basis for new setups, entry, and exits. This helps forward test the system as well as use it for live trading.

We will be launching soon. If you want to learn more and be updated when we launch, please sign-up for the invite at