Walk forward testing is a powerful method that is used in financial trading to determine the best set of parameters for an optimization. This may sound complex but in reality it’s a simple and powerful concept to help you build the best trading systems around. It’s also a concept that TradersStudio implements very well.
Often when discussing trading systems, we talk about system robustness. The ability of a system to perform well under all market conditions is important. If you have a system that does very well during upward trend markets, but loses significant amounts of money during downtrend markets (like 2008), we would say that system is not robust. There are market conditions that would cause it to do really, really poorly. If we were able to filter out those bad trades – maybe shut off the system during downward trends – we would call the system “robust” because it can be traded with confidence in many market conditions.
Walk forward testing helps us determine system robustness by allowing us to divide historical data into two groups. “In-sample data” is the data that we use to optimize our system. “Out-of-sample data” is the data that we use to test our system. We can ask hypothetical questions like “if I were trading this system between 2005 and 2007, how would my system have performed in 2008?” These types of questions help to test the system’s ability to work under many different market conditions. If we see that the optimal parameter for the in-sample windows also does well in the out-of-sample windows, then we can have better confidence in the system’s future profitability. Obviously we cannot predict the future and profits are not guaranteed by any means, but we can gain confidence and trust in the system. In a couple of articles, I have written extensively about how walk forward testing is really the foundation for determining a trading system’s robustness. These articles which are published in Futures Magazine, include “Optimization: Teaching an Old Dog New Tricks” and “Can Your System Do The Walk“. I highly recommend that you take the time to read these articles if you are interested in learning more about walk forward analysis (after reading this article of course!).
For those that are interested, there is a great article on using “backtesting” and “forward testing” on Investopedia. This article highlights how the two concepts are both distinctly different but also related.
It is worth noting that modern day computers have advanced in processing power so much that we can now do this type of analysis extensively to ensure we have robust systems. I have written about this advancement of computers in an article for Futures Magazine which is entitled “Trading System Analysis: Then and Now“. All of the techniques that I discuss on this site are ones that you can run on your home computer or laptop. No fancy hardware is required!
Walk Forward Testing by Example
Many times, it is best to illustrate how a concept works by example. As with all our examples, we will use TradersStudio to highlight how this concept works. Suppose you have a super simple trading system that consists of looking at the RSI (relative strength indicator) for Microsoft. Let’s say our system is coded to buy when the RSI goes below 50 and to sell when the RSI goes above 50. The RSI is calculated over a series of days. Coding this system is trivial in TradersStudio:
'-------------------------------------- ' MSFT RSI Test System ' This is a super simple system that is ' designed to show how walk forward analysis ' works. It is not intended to be a ' tradeable system. '-------------------------------------- Sub MSFTRSITest(Days as Integer) If RSI(Close, Days, 0) > 50 Then ExitLong("Sell", "Buy", 1, 0, Low, Day) End If If RSI(Close, Days, 0) < 50 Then Buy("Buy", 1, High, Market, Day) End If End Sub
Now we want to answer a basic question: what number of days produces the most profitable results? Should we calculate the RSI over the past three days? The past five? We could run the system with trial and error, but this is a time consuming task. We can ensure robustness in the system by doing walk forward testing on this.
Let's test for an RSI length of 1 to 30, stepping 1 day at a time. We'll use 500 as the training window (about two years) and run it for 250 (about one year, given that the market is open about 252 days a year). We'll optimize for the maximum net profit to drawdown ratio. Put another way, the computer is going to calculate the RSI using days from 1 to 30 for the first two years of data. It is then going to get the best number of days for those two years of data (the in-sample data) and test how it runs over the year after (the out-of-sample data) and report those results. Then, TradersStudio will advance the marker one year. This will drop off the oldest year of data and include one year of the most recent data in the training set. We will then calculate the optimal number of days for that new data. It will then find the best number of days and test on the year after. This is how walk forward testing got its name. We are "walking forward" through the data, testing how our system performs in-sample and then out-of-sample.
We first click on the walk forward optimize button within TradersStudio.
Then we set the parameters as we described above.
We can see how this type of analysis works from the results below:
We have training start and training end periods which are approximately two years apart, then we test for about one year after to see how the system performs. We can see the optimal parameter during the first two years of testing and how that parameter performed in the year afterwards. This allows us to build more robust trading systems. Traditional optimization works by answering the question: over the past 20 years how would my system have performed under this set of parameters? The problem with this basic question is that this does not provide any insight as to how your system will work in the future or any insight as to how your system would have worked over smaller time periods in the past. All too often, it's possible to have a system that works really well over very long times (maybe it hits one really big trade in 2011) but overall is mediocre at best. A good system should pass walk forward testing - meaning that we should be able to test it over smaller chunks of historical data and see that the system does, indeed, work with the out-of-sample data.
Why Is This Important?
Confidence in a system is important. I have witnessed so many times when people have lost money in their trading accounts because they could not follow the system or did not trust it. Understanding your trading system and having faith in it, is really the difference between traders who make money and those who do not. Many systems do work, and walk forward testing helps you gain confidence in those that do and those that do not. All professional traders use walk forward testing in their work and I would strongly recommend that you also do the same!
If you are interested in learning more about this concept, I highly encourage you to read our blog entries on the subject:
- Introduction to Walk Forward Analysis
- Walk Forward Testing in TradersStudio
- Walk Forward Testing on a Portfolio
These entries go over a more complex example of using TradersStudio to do this type of analysis.
Also, since optimization is a topic that has been so critical for both myself and other traders over the years, I would also encourage you to consider taking a look at some books on the subject by well-respected authors such as Robert Pardo.