How to test your strategy development process to see if your are really a profitable system trader?

Hereby I want to propose a validation method called ‘simulated forward test (SFT)‘.
(Note: This is not WFA or WFM, WFA/WFM is a different topic, this is
also not an additional OOS test as used during strategy generation since
the SFT period is not used for strategy selection)

Simulated forward test is basically a ‘walk forward’-like test of your ‘total strategy design workflow/method’.

In the last few months I tried to develop a fully automated workflow using SQ-X
‘custom projects’ module and I got very interesting results (which I
will post in another topic). My small breakthrough was based on an
observation that majority of traders, and even experienced traders
giving online courses, just assume they are right about their strategy
validation method based on their experience and their past results.
There is nothing wrong with that assumption, but it can be killing for
new traders or people who are trying to design their own strategy design
methods.

What you normally see is the following development sequence:

1.
All traders have their own ideas or methods for generating strategies
and they use these methods to generate their new automated trading
systems.

2.
In 90% of all cases, new traders generate and run their systems on demo
(or small live) accounts to see if their strategies are profitable or
not.

3. This results in a ‘generate & hope’ methodology.

In my opinion the proper approach should be as follows:

1.
At first you need to develop or learn a new strategy design workflow (a
new system design method or process including strategy
generation/testing/validation and strategy selection).

2.
In order to test if your method works you go back in time and pretend
to be living in the past, e.g.: 2014. This is your point of reference.

3.
Then you develop a sufficient (significant) amount of strategies using
your workflow using data <=2014. In this case using data >2014 is
forbidden to prevent any type of ‘Data Mining Bias’.

4. Then you test ALL
strategies that have passed your workflow and strategy selection on a
simulated forward period (your simulated future) >2014 and verify how
many of selected strategies have worked for you. You do this without
cheating or fooling yourself, so you use ALL strategies that have passed
your workflow successfully. You draw your conclusions based several
different pass/fail criteria that are important for you e.g.: Max DD,
Net Profit, Red/DD, Stagnation etc….and you write down the effectiveness
percentage of your workflow e.g: 65% of selected strategies were
profitable (or did not exceeded Max. expected DD) in the simulated
future.

5.
You repeat the whole process by selecting a different point of
reference in the past e.g.: 2016. And start all over again by
generating, testing and selecting new strategies. Then you test again
ALL of the selected strategies in a simulated period >2016.

You
can repeat this process many times, by ‘walking forward’ your whole
workflow in time using historical data and see if your strategy
generation method really produces robust strategies in a simulated
forward period.

An example of time slots as used in a typical workflow:

Simulated Forward Test


Point of reference (in the past) = 2014.12.31 (you are forbidden to use
data > 2015.01.01 until you have generated 30 strategies that passed
your workflow and selection method for live trading, this can be
including portfolio building methods)

– Your first IS/OSS for strategy generation is e.g.: IS: 2008.01.01…2011.06.01 and your first OOS is 2011.06.02…2014.12.31. 

– Your second OOS2 for cure-fitting check is e.g.: 2003.01.01…2007.12.31

– All other validation tests (spread,skipping trades, MC, cross market etc..) are done using data <=2014.12.31

– You select 30 strategies and test them ALL on data >2015.01.01

After
you are confident with your workflow and your results you can jump-back
to present time and generate/validate and select new strategies and
feel confident that selected systems have a significant chance of being
profitable in the upcoming future.

Just wanted to share this because this approach has led to my first small breakthrough with my 100% automated workflow.

Conclusion:
stop fooling yourself, stop generating strategies based on untested
methods and stop hoping those will work out for you well in the future…

Any comments are welcome 🙂 Visit our Discord HERE.

Gr

Chris


[DISCLAIMER]: This site contains valuable and informative high quality material, that can be used for any kind of trading. The presented information is meant for non-commercial, informational and personal purposes only, and all of freely downloadable materials are meant for non-commercial home use only. To the maximum extent permitted by applicable law, in no event shall Coensio.com, or its suppliers be liable for any special, incidental, indirect, or consequential damages whatsoever (including, without limitation, damages for loss of business time, business profits, business information, or any other kind of loss) arising out of the use of, or inability to use the Software developed by coensio.com, or the failure to provide support services, even if Software, or one of its supplies has been advised of the possibility of such damages. Copyright © 2021 coensio.com. The majority of the links in coensio.com posts and sidebar are affiliate program links. This means that (most of the time) when you purchase a product linked from my site, owner(s) of this website receive a commission. CFTC RULE 4.41 – HYPOTHETICAL OR SIMULATED PERFORMANCE RESULTS HAVE CERTAIN LIMITATIONS. UNLIKE AN ACTUAL PERFORMANCE RECORD, SIMULATED RESULTS DO NOT REPRESENT ACTUAL TRADING. ALSO, SINCE THE TRADES HAVE NOT BEEN EXECUTED, THE RESULTS MAY HAVE UNDER-OR-OVER COMPENSATED FOR THE IMPACT, IF ANY, OF CERTAIN MARKET FACTORS, SUCH AS LACK OF LIQUIDITY. SIMULATED TRADING PROGRAMS IN GENERAL ARE ALSO SUBJECT TO THE FACT THAT THEY ARE DESIGNED WITH THE BENEFIT OF HINDSIGHT. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFIT OR LOSSES SIMILAR TO THOSE SHOWN. [OUR PRIVACY POLICY]: READ HERE