In the first part of this series on TradersStudio Walk Forward testing and analysis, we analyzed why walk forward testing is so important. In the second part, we showed how we could do walk forward testing on the Natural Gas market.
One of the awesome features of TradersStudio is Walk Forward Testing on a Portfolio of markets. Let’s create a session using a basket of markets and run the walk-forward analysis like the single market session. TradersStudio runs the walk-forward analysis across the basket of markets.
Create a session “WalkForwardSimpleBasket.” Once again, we will run the test on the Channel Breakout system.
We have also deducted $75 for slippage and commission as we did in the previous example.
For this example, select the following markets – Natural Gas (ZN), Corn (ZC), Crude Oil (ZU), Cotton (CT), the Dollar Index (DX), the Japanese Yen (JN), Coffee (KC), and the Ten-Year Note (symbol TY for day session only).
We started our run from 1991 because that is when trading in Natural Gas began. It is important to remember that walk-forward analysis only works on overlapping date ranges.
Here are the results of the Session:
The next step is to run simple optimization with the settings in the screen shot:
The results for the optimization are as follows:
Now, doing Walk-Forward Optimization, we used the same (1000, 200) windowing selection and optimized using ‘Net Profit’ as the filter.
If we check the Walk-Forward Periods Report, we will find that the ideal length of the Channel Breakout ranged from 10 to 20 periods on this basket of markets during our testing period.
We can also study the ‘Clean Trades’ and ‘Clean Trades by Period’ reports for more clues. These reports include out-of-sample trades and depending on the rules used to handle the transitional trades, the results are different:
- ‘exit all trades’ is selected; then the closed trade profits are $163,088.16 in closed trade profits and $34,153.66 in open trade profits.
- ‘exit if direction changed’ produced the highest profits at $209,665.89 and the open trade profits with the same is $34,153.66.
- ‘exit and reenter’ is selected; then the closed trade profits are $170,825.27 and the open trade profits with the same is $34,153.66.
You can see that by changing this switch, profits are substantially less. This demonstrates the point that these switches do affect system performance.
Here is a screen shot of the Active Order Report providing current live trades that are tomorrow’s trades based on walk-forward analysis.
For further analysis of a particular strategy, click the Run button on the ‘Clean Trades’ or ‘Clean Trade by Period’ reports to generate a full back test session for a given walk-forward system. The next screen shot shows the results for “exit all trades.”
The first trade occurred on March 16, 1995. In order to compare apples to apples, we re-optimized this model using standard optimization from January 3, 1995. This causes the first trade in our walk-forward analysis to match closely with the first trade in the standard optimization. You can check the results via the Optimization Main Result tab by clicking the “Activate” button next to the parameters and checking out the values of the open and closed profit figures in the summary report produced. Below are shown the results for parameter ‘5’, whose closed profit is ($277,725.29) and open profits is ($2,752.64).
On the standard re-optimization, ‘15’ was found to be the optimal parameter with this 1995 start date. This setting produced $387,785.78 in closed trade profits and open profits of $34,153.66. The walk-forward results produced $209,665.89 as closed profits while the open ones are $34,153.66, which is good considering that our walk-forward results are out-of-sample.
It is important to remember that you cannot expect walk-forward results to be as impressive as the results from in-sample optimization. If the results from walk-forward testing are between 50% and 75% of the in-sample testing profits, it shows that the system is working as expected and gives realistic numbers as to how the system will perform moving forward in real time.