We have discussed Walk Forward analysis and its importance in the past. The first part of this tutorial series introduced the concept. The second part demonstrated how to use Walk Forward in TradersStudio. Finally, the third part described how to use Walk Forward on a portfolio. One of the biggest issues with system development is that many systems might test well using historical results, but they do not perform well in the future. There are several reasons for this. The most common one is that the system is not based on a valid premise. Other reasons include:
- Lack of robustness in the system due to improper parameters. A system is considered robust if it runs well in any market condition.
- Inconsistent rules and improper testing of the system using “out-of-sample” and “in-sample” data.
As described through our other tutorials on this site, Walk Forward Analysis does optimizations on a training set, then tests on a period after the set and keeps rolling it forward repeatedly. Thus we have multiple out-of-sample periods and we can look at these results combined. Originally discussed by Robert Pardo, this type of analysis can keep a trading system model a step ahead. The reason is that you are not over-fitting the training set but rather you are fitting a portion of it, and then testing it on the out-of-sample data to ensure that it performs well outside of the training set.