Simulated Forward Test
How to test your strategy development process to see if your are really a profitable system trader?
Hereby I want to propose a validation method called ‘simulated forward test (SFT)‘. (Note: This is not WFA or WFM, WFA/WFM is a different topic, this is also not an additional OOS test as used during strategy generation since the SFT period is not used for strategy selection)
Simulated forward test is basically a ‘walk forward’-like test of your ‘total strategy design workflow/method’.
In the last few months I tried to develop a fully automated workflow using SQ-X ‘custom projects’ module and I got very interesting results (which I will post in another topic). My small breakthrough was based on an observation that majority of traders, and even experienced traders giving online courses, just assume they are right about their strategy validation method based on their experience and their past results. There is nothing wrong with that assumption, but it can be killing for new traders or people who are trying to design their own strategy design methods.
What you normally see is the following development sequence:
1. All traders have their own ideas or methods for generating strategies and they use these methods to generate their new automated trading systems.
2. In 90% of all cases, new traders generate and run their systems on demo (or small live) accounts to see if their strategies are profitable or not.
3. This results in a ‘generate & hope’ methodology.
In my opinion the proper approach should be as follows:
1. At first you need to develop or learn a new strategy design workflow (a new system design method or process including strategy generation/testing/validation and strategy selection).
2. In order to test if your method works you go back in time and pretend to be living in the past, e.g.: 2014. This is your point of reference.
3. Then you develop a sufficient (significant) amount of strategies using your workflow using data <=2014. In this case using data >2014 is forbidden to prevent any type of ‘Data Mining Bias’.
4. Then you test ALL strategies that have passed your workflow and strategy selection on a simulated forward period (your simulated future) >2014 and verify how many of selected strategies have worked for you. You do this without cheating or fooling yourself, so you use ALL strategies that have passed your workflow successfully. You draw your conclusions based several different pass/fail criteria that are important for you e.g.: Max DD, Net Profit, Red/DD, Stagnation etc….and you write down the effectiveness percentage of your workflow e.g: 65% of selected strategies were profitable (or did not exceeded Max. expected DD) in the simulated future.
5. You repeat the whole process by selecting a different point of reference in the past e.g.: 2016. And start all over again by generating, testing and selecting new strategies. Then you test again ALL of the selected strategies in a simulated period >2016.
You can repeat this process many times, by ‘walking forward’ your whole workflow in time using historical data and see if your strategy generation method really produces robust strategies in a simulated forward period.
An example of time slots as used in a typical workflow:
– Point of reference (in the past) = 2014.12.31 (you are forbidden to use data > 2015.01.01 until you have generated 30 strategies that passed your workflow and selection method for live trading, this can be including portfolio building methods)
– Your first IS/OSS for strategy generation is e.g.: IS: 2008.01.01…2011.06.01 and your first OOS is 2011.06.02…2014.12.31.
– Your second OOS2 for cure-fitting check is e.g.: 2003.01.01…2007.12.31
– All other validation tests (spread,skipping trades, MC, cross market etc..) are done using data <=2014.12.31
– You select 30 strategies and test them ALL on data >2015.01.01
After you are confident with your workflow and your results you can jump-back to present time and generate/validate and select new strategies and feel confident that selected systems have a significant chance of being profitable in the upcoming future.
Just wanted to share this because this approach has led to my first small breakthrough with my 100% automated workflow.
Conclusion: stop fooling yourself, stop generating strategies based on untested methods and stop hoping those will work out for you well in the future…
Any comments are welcome 🙂 Visit our forum HERE.