How to test your strategy development process to see if your are really a profitable system trader?
Hereby I want to propose a validation method called ‘simulated forward test (SFT)‘.
(Note: This is not WFA or WFM, WFA/WFM is a different topic, this is
also not an additional OOS test as used during strategy generation since
the SFT period is not used for strategy selection)
Simulated forward test is basically a ‘walk forward’-like test of your ‘total strategy design workflow/method’.
In the last few months I tried to develop a fully automated workflow using SQ-X
‘custom projects’ module and I got very interesting results (which I
will post in another topic). My small breakthrough was based on an
observation that majority of traders, and even experienced traders
giving online courses, just assume they are right about their strategy
validation method based on their experience and their past results.
There is nothing wrong with that assumption, but it can be killing for
new traders or people who are trying to design their own strategy design
methods.
What you normally see is the following development sequence:
1.
All traders have their own ideas or methods for generating strategies
and they use these methods to generate their new automated trading
systems.
2.
In 90% of all cases, new traders generate and run their systems on demo
(or small live) accounts to see if their strategies are profitable or
not.
3. This results in a ‘generate & hope’ methodology.
In my opinion the proper approach should be as follows:
1.
At first you need to develop or learn a new strategy design workflow (a
new system design method or process including strategy
generation/testing/validation and strategy selection).
2.
In order to test if your method works you go back in time and pretend
to be living in the past, e.g.: 2014. This is your point of reference.
3.
Then you develop a sufficient (significant) amount of strategies using
your workflow using data <=2014. In this case using data >2014 is
forbidden to prevent any type of ‘Data Mining Bias’.
4. Then you test ALL
strategies that have passed your workflow and strategy selection on a
simulated forward period (your simulated future) >2014 and verify how
many of selected strategies have worked for you. You do this without
cheating or fooling yourself, so you use ALL strategies that have passed
your workflow successfully. You draw your conclusions based several
different pass/fail criteria that are important for you e.g.: Max DD,
Net Profit, Red/DD, Stagnation etc….and you write down the effectiveness
percentage of your workflow e.g: 65% of selected strategies were
profitable (or did not exceeded Max. expected DD) in the simulated
future.
5.
You repeat the whole process by selecting a different point of
reference in the past e.g.: 2016. And start all over again by
generating, testing and selecting new strategies. Then you test again
ALL of the selected strategies in a simulated period >2016.
You
can repeat this process many times, by ‘walking forward’ your whole
workflow in time using historical data and see if your strategy
generation method really produces robust strategies in a simulated
forward period.
An example of time slots as used in a typical workflow:
–
Point of reference (in the past) = 2014.12.31 (you are forbidden to use
data > 2015.01.01 until you have generated 30 strategies that passed
your workflow and selection method for live trading, this can be
including portfolio building methods)
– Your first IS/OSS for strategy generation is e.g.: IS: 2008.01.01…2011.06.01 and your first OOS is 2011.06.02…2014.12.31.
– Your second OOS2 for cure-fitting check is e.g.: 2003.01.01…2007.12.31
– All other validation tests (spread,skipping trades, MC, cross market etc..) are done using data <=2014.12.31
– You select 30 strategies and test them ALL on data >2015.01.01
After
you are confident with your workflow and your results you can jump-back
to present time and generate/validate and select new strategies and
feel confident that selected systems have a significant chance of being
profitable in the upcoming future.
Just wanted to share this because this approach has led to my first small breakthrough with my 100% automated workflow.
Conclusion:
stop fooling yourself, stop generating strategies based on untested
methods and stop hoping those will work out for you well in the future…
Any comments are welcome 🙂 Visit our Discord HERE.
Gr
Chris