Advanced (micro)-futures strategy building workflow for StrategyQuant

Hi traders!

Below, you will find my proprietary ‘blueprint’ workflow for building micro-futures strategies using the StrategyQuant platform. It took me a few years to develop and fine-tune every single step of it. The main goal of this project was to see if it was possible to create a general workflow that would produce ‘tradable’ strategies on all futures markets. I would like to extend my thanks to’all of the passionate traders who have supported me (especially with their personal and computing time) in achieving this goal. The resulting workflow is as follows:

I know it’s a lot of steps, but nobody said it would be easy. Hopefully, most of the steps are fully automated, but as an algo-trader, you already know that it all takes a lot of time. On this blog you can see few examples of my strategies that are built using this method: HERE. In the next few weeks, I will try to explain it all in my new eBook;) Until then, you can ask me questions anytime:) I will answer them here:

Q1: If you have to pick only one robustness tool, which one would it be? (Anonst)

A: Strategy validation is an extremely wide topic and it will be deeply discussed in this section. So, simplified answer is that it depends on many different factors, however most of the work we did with (micro)-futures is based on WFA/WFM. These are also the tests that are widely used by professional traders. But, for example after several thousands of workflow runs and running some simple statistic, we have seen that some of other robustness tests like Monte-Carlo parameter test did not really add any additional filtering (any usable information) in the selection of the final strategies. However this is only true for this specific case (my automated workflow(s)). The main goal of validation is to find stability area for all selected setpoints. But as I already mentioned this topic is very deep and will be continued…

B.T.W I can not choose only 1 validation test, simply because it is not enough! Since each of tests, targets a different setpoint! Read more here: Meta-stability in algo-trading. Testing of your trading “robots” must be pretty harsh in order to make it work.

Q2: Another question if you allow: what are the tricks to making sure SQX and trade station backtests generate the same trades ( I see a lot of variation in performance/trades between the two). I am using trade station data in SQX but still see differences. (Anonst)

A: Yes indeed, there are many things that need to be considered in order to achieve sufficient synchronization between SQX backtest and TradeStation results. Since I plan to make a short video about it, for now I will give you only a general answer. I can imagine it will be not satisfactory, at least not for now. So things you need to consider are:

• Exporting of TradeStation historical chart data and importing to it to SQX, must be done properly
• Proper symbol settings must be calculated
• Some of the SQX building block(s) must be avoided (disabled)
• SQX trade execution needs to be synchronized with TradeStation (yes it’s different! and nobody talks about it or even knows about it)
• Cost-of-Trading needs to be properly calculated (in case these are considered during the backtest)
• And finally when working with WFA systems, proper optimization settings that corresponds to their WFA period must be added

So I know this rises even more questions, but you need to be patient, sooner or later I will provide this info to you guys…t.b.c.

Q3: When you do volume analysis and session analysis, are you basically looking to focus the search on the times of day the majority of volume and trading range historically occur? (OURocketMan)

A: I perform a very simple (excel based) volume analysis to check two things:

1. Is the volume high enough for any given chart to be considered as a tradable instrument. This is specially critical for micro-futures charts, since most of them have a very thin volume (very difficult to trade or not tradable at all).

2. Where are the most active hours = probably the best session to start with (but there are also profits to make outside the most active hours!)

A: The session analysis is a very different story, since it requires a lot of effort. What I basically do is: I brute-force it;) Meaning that I make several different version of the same builder (each targeting different session hours) and I run it and collect the data. For example, below few almost randomly selected sessions, I used on one of my instruments:

The result of such a scan is a very large table that tells me what session works the best and which hours I need to target to find my robust trading strategies. Hereby I consider some basic parameters e.g.:, #trades, Ret/DD, Avg.Trade and Worst Year DD.

Q4: For micro futures are you using Macro data or micro data to find the strategies ? Any of your strategy combine both source of data ? Or you are using only 1 source of data (Micro only) ? (HangTuah)

A: If for any given micro-contract historical data quality is sufficient and long enough (>=10years of continuous contract data), then I always prefer to use micro-data over the main contract data. However, some of the micro-contracts instruments have only few months of data, and if you want to develop on such instruments you are forced to use the main contract data. BUT, this will cause some issues like:

• The volume on the main contracts is very different than on micro-contracts, so you need to avoid using volume based indicators during the development!
• There are also some small but sometimes very significant differences in OHLC levels between micro and main contracts, so you always need to recheck/backtest your final systems on the small part of the historical data that is available for your micro contract. Just in order to see if it still aligns with what you see on the main contract.
• In the end you will end up with mixed systems (developed on main contracts and developed on micro contract data). This will give you some problems when assembling your portfolio (profits and losses on main contracts will be like 10x larger than on micros). So you need to be creative and ‘scale’ down your main contract results in order to properly assemble your portfolio in your QA.

Q5: Are you helping folks out that use the promo code in your e-book / providing them with some tools to get started? (OURocketMan)

A: I will consider it…but it is not my main objective right now…

Q6: How do you export historical chart data from TS and import to SQX? (Kaizen)

A: You can export historical chart data in TS using just few simple steps:

1. E.g.: Export @MNQ.D 1min data from chart using the following settings:

2. Add new SQX instrument according to proper TS symbol settings:

P.S.: If you will ask nicely @lmp2021 on our discord…he has made some nice Excel sheet with symbol overview and settings 😉

3. Add new SQX symbol “MNQD_M1” using “Add symbol” option:

4. Import TS data (see step 4) to SQX using the following settings:

Q7: What are the statistical analysis that you are performing? (Kaizen)

A: I collect as many data as I can e.g.:

• The most profitable/robust building blocks for any given symbol (you start with almost all blocks but after few runs you can already see what works and what doesn’t work on the selected market.)
• The most common initial strategy selection parameters in my final strategy pool, this means: I look at my final strategies and try to find which criteria/parameters have the most impact in selection of potentially good strategies (directly after the first builder step).
• Slippage data, in order to better estimate Cost-Of-Trading.

Q8: What happened in Strategy segregation step and stage2 improver? (Kaizen)

A: In strategy segregation step I segregate my strategies by their “Best WF” run (SQX databank column filter), this gives me an idea where SQX has found the most stable WF region in the WFM matrix. So my WFA test is done using this Best WF setting.

A: Stage 2 “improver” is an additional builder step that takes previously built strategies as input in its “Initial population” databank. By doing so we are “telling” the stage 2 builder, which specific building blocks to target, in order to build new and much stronger/robust strategies.

Q9: You may recall that I recently asked on Discord about annually re-optimizing the strategies in SQX.  Unfortunately, I’m still struggling a bit with this exactly.  Would you mind further explaining?  (Hemmo)

A: Periodic re-optimization can be done in several ways:

1. You just add/import new historical data into the SQX and re-run the final WFA step. It’s is worth mentioning that when using #RUNS and %OOS WFA settings in development phase you need to translate it to proper number of IS/OOS days and re-run WFA using DAYS not %OOS.
2. Keep using #RUNS and %OOS but then correct the length of available WFA data so it is still the same as it was during strategy developement (e.g. 12 years).  Hereby it is worth mentioning that, now each of previously used WFA IS periods can shift a little bit. In that case if you will see a significant decrease in your strategy performance you should ask yourself if you want to trade this strategy.
3. Configure your WF settings during development phase in such way that each OOS period will be exactly equal to 1 year and will end on 31-December. Then you will need to reoptimize at the end of each year. However this method will limit you to one specific #RUNS + %OOS setpoint in the search of WF(M) stability.

Q10: I followed your idea posted on SQX forum about simulated forward test method and found out that my Workflow only gives 55% profitable strategies in the next two years This is normal? Or acceptable? Or should I add more robustness checks? (Mangel98)

A: So you already increased your winning edge above 50% 😉 With a little more of practice it is possible to reach >80% or more. What you should consider is adding a manual strategy selection step where after your automated workflow is done, you browse through your final strategies, look carefully at them, and analyze visually all critical aspects. Then you are manually selecting only the ones that are looking like the winners to you. And then you can see what will be the winning rate of strategies that you have manually pre-selected. After few workflow cycles, while shifting workflow using SFT method you should easily be able to learn how to select the winners. But yes it is a lot of work. Algol-trading is nothing like baking buns 😉

BTW. No workflow in the world will give you 100% guarantee that selected system will make you money in next 2 years. That’s is why most professional traders diversify over the multiple markets and multiple strategies.

Q11: How do you select markets at the first step? What do you look for a set of markets to run in your portfolio? (Kaizen)

A: It depends..if you want to trade micro futures then requirements are more practical, because there are not that many tradable micro’s. So in general for micros and also for main contract I would basically check the following things:

• Will the margin-cost of this contract “fit” into my portfolio’s equity?
• Is the volume high enough? Can I find any info on typical slippage?
• What is the value of 1 point? (in $) What will be the round-turn cost for executing of 1 trade (including fees and commissions). = How high my Avg.Trade must be in order to cover to cost of trading?
• How badly is the target market correlated to markets I already trade. If you google around you can easily find portals providing correlation matrices for futures.

Q12: What is Strategy labeling? (Kaizen)

A: It’s a simple method of renaming of my initial strategy candidates so I can trace back their initial performance, builder type, workflow etc…simply by analyzing their file names.

Q13: Why do you have multiple wfa and WFM? What are the differences? (Kaizen)

A: Over the years I have learned to look for strategies in step-by-step manner. On “difficult” markets when you will try to find any usable strategies by using all of your pass criteria, you will find nothing since probably nothing will pass your robustness tests. So in first stage I look for some strategies that will pass SOME of my criteria (=stage1) and then I put them in my Initial Population database and use “Improver” builder to build more strategies with similar characteristics (like the same building blocks). Those strategies have an increased chance to pass my full robustness tests (stage2).

Q21: What books or courses would you consider interesting for anyone working with SQX? (Rogerio)

A: I do not want to promote any external courses rather than what SQX has prepared so far, however if you really want to dive deeper then go to the “source”. And there is only one source when it comes to robust validation of strategies: the legend and my hero: Robert Pardo.

Q22: Do you have any tips about what to avoid or be aware of in SQX and algo-trading? What is your preference Random strategy generation or generation by Genetic evolution ?(Rogerio)

A: Two things that come to mind:

• When you will develop a working workflow and you will find your first profitable strategies, always save/backup your strategies TOGETHER with used workflow!
• Watch out with automatic building blocks/indicator calibrations in SQX it can dramatically change the resulting strategies that are build with SQX. So always save your workflow + settings + strategies as “1 package”. These always all go together.
• If you switch to new SQX version always perform a “sanity” check by re-running your final strategies through your workflow to see it if still works as on the previous SQX version.

A: Genetic evolution, because I like to “improve” my strategies using Initial Population database.

Q23: I would like to start trading Micro futures, which broker could I start with? And with how much money can I operate? (Mangel98)

A: I do not promote any brokers, that’s my rule. However I can tell you that I trade on TradeStation. You can start with micro-futures trading with as low as $2500, but you can also trade with millions if you like 😉

Q24: In strategy builder settings SQX by default uses maximum “3” in “Conditions in entry rules”. Do you think it should be a maximum of 2 or could it even be 3, due to future robustness tests? (Rogerio)

A: 3 should be fine, the better way to limit complexity of the systems is to use ‘complexity’ filter in the builder. I use “complexity” =< 10.

Q25: What does the Reserved bars setting in Trading options within the builder actually do? SQX seems to generate strategies with lookback period that still exceed this setting when set equal to global indicators max period back setting (OUrocketman)

A: I think the initial goal of ‘Reserved bars’ was to emulate the ‘Maximum Bars Back‘ Setting in TradeStation. I always keep these settings the same in SQX and TS. In that way if you find a SQX strategy that exceeds this period then you will simply get an error in your TS. Yes, I agree it should not happen, but SQX has still some small bugs in it. The simple workaround is to set reserved bars to a high value like 50 or 100, and keep lookback periods far below this value.

Q26: You are being a true inspiration 🙂 When building Genetic strategies, some people prefer not to put any “filter generated initial population” to give more freedom to search for diversified strategies. Other people already prefer to start with only profitable strategies. What is your opinion? (Rogerio)

A: Yes indeed you could leave it almost unfiltered in the first stage, there are not many filtering parameters that can somehow guarantee you the selection on good robust final strategies. Especially when you work with WFA which changes the settings on to go…So what I do is I keep it very basic in the first stage and then as I progress (from stage to stage) I can see which parameters give me more chances of selecting good final strategies and then I use those (or increase their filtering thresholds). PF and Ret/DD are the good settings to start with. Do not forget to add “complexity” to your filter 🙂

Q27: I notice that the tradestation engine does not support using minute data. Is it possible to use the metatrader engine on tradestation data to take advantage of the more precision available in the metatester engine while still exporting strategies in tradestation and have things match for the most part? It seems Tradestation doesn’t match well when building higher time frames from 1M data, so wondering if this would be a possible workaround. (OUrocketman)

A: I think you meant tickdata, since M1 for TS should not be a big issue. However the question is if you really want to trade on M1. From my experience M15 is the minimum TF you can trade on. MT4/5 is not the best choice for futures, it has different trade execution engine and supports no sessions. TS results match very well with SQX even with very high TFs, but you need to do your homework, analyze your data and find out why you see the mismatch, “the truth is out there”, it’s all in the data 😉

Q28: Do you think that the multimarket test determines if a strategy is robust? (Mangel98)

A: Basically yes, I always perform at least one 2nd market test. The more the better? Probably yes, but I have no proof for that. The theory behind this test (at least my theory) is that when developing a strategy on 1 single market your strategy could be curve-fitted to the market “noise”. When running it on another, but correlated market (with a different “noise”), you can verify if your strategy is able to capture a global market behavior or just the “local noise”.

Q29: What hardware do you use for SQX to develop strategies? (Anonst)

A: I have a pretty old intel i7 machine with 16 cores 32GB RAM and 500GB drive. So, you do not need ultra fast PC in order to make it work, but sure faster is always better.

Q30: How many years of data do you usually test over IS and OOS at a time prior to each walk forward? (OUrocketman)

A: I do not have any rules for that, because I think that it does not matter. The only thing that matters is the ‘meta-stability‘ verification of the selected WFA setpoint. But of course for practical reasons it would be nice to see a new OOS every single year, then you would need to optimize once a year.

Q31: What criteria do you use to choose which indicators and signals will enter your workflow? Do you always use the same indicators or do you make several flows with different indicators? Do you have an ideal maximum amount of indicators to use per workflow? (Rogerio)

A: Each specific market will act differently so it will require a different set of indicators. In my standard workflow I just start building with as many indicators possible in order to see what works on specific market. Then gradually I “shrink” my workflow to best performing indicators on a given market.

Q32: What are your thoughts on adapting the workflow process in such a way that it generates prop-firm compliant EAs? i.e staying within daily and max drawdown limits, avoiding news releases etc without sacrificing profitability. Is this something you have thought about or looked into? (jojoarmand)

A: No, not really because I’m not that interested in prop trading. This is mostly for FX markets = not for beginners. For FX I mainly use self coded EAs and indicators.

Q33: Are you still using MT4 EAs? (Kaizen)

A: Over the last 10 years I have developed and helped to develop countless MT4 EA’s (still do), however no I do not use EAs for trading. The main reason behind this is a very poor correlation between historical backtest and live results (poor historical data quality), while on Futures contracts this correlation is very good and reliable for trading.

Q34: I have a question. How do you get futures tik data into MT5? I get a message in the data manager in SQ that I can’t export futures data. (Stu)

A: We don’t use MT5. Everything we do here is based on TradeStation/MultiCharts.

Q35: Coensio, SQX creates strategies with names full of numbers that don’t seem to mean much. Do you usually rename created strategies? And what kind of information do you like to put in the name of the strategies? (Rogerio)

A: Yes as you can see in my workflow, there is a step called “strategy labeling”, in this step I rename initial strategies using some basic info like RetDD, PF, stability, number of trades etc…etc…This helps a lot since, (at the end) you can see what characteristic should strategy candidates have in order to have increased chances to pass a given workflow.

Q36: Looking at your workflow diagram, I didn’t see Monte Carlo, do you not use it or is it embedded within some other task? (Rogerio)

A: In almost all strategy building workflows we have left MC test out (with few exeptions). Why? Because after millions of workflow runs we have notice that MC doesn’t really add much of filtering to the final strategies. That means strategies that have passed all other test almost always passed our MC runs, so MC became a waster of our computing time. The only point where we still do MC (trades) is in QA where strategies are already imported back from TS.

Q37: Could you share a workflow or a system made by sqx? I haven’t found anything in futures. Coensio robust forex strategies, do you know where they can be found?? (Amador)

A: Sorry, we stopped sharing our strategies and workflows long time ago. We are not considering sharing them again for many different reasons.

Q38: In WFA did u include multiple TF or u just focus on 1 TF? i.e data1 (H1), data2 (H4), data3 (Daily). (HangTuah)

A: I run WFA only the main TF, but you could run the resulting strategy on another TF just to see if it doesn’t die but it is not a hard pass/fail validation test for me only an additional confirmation.

Q39: Hi, in a workflow I’m using multiple robustness checks like multimarket, OSS, MonteCarlo and SPP. Last one is WFM (with default settings), but no strategy pass this test. Is it really a problem ? Could some strategies not be suitable for WFM ? (plawi)

A: Yes indeed some strategies will never pass WFM since these strategies are not suitable for periodic re-optimization (WFA), in that case you could look more into Monte-carlo and SPP.

Q40: Thank you for answering. From your experience, what kind of strategies would not pass WFM ? (plawi)

A: It depends on many factors for example, I find it very difficult to find good WFM strategies on currency markets or systems trading on higher TimeFrame (>1H) or mixed TF systems. Also systems with a very very large complexity >12 to not perform well in WFM.

Q41: Hi there, May I know if I were to run my default chart at H1, but in the condition it contains a DailyHigh(1) or DailyLow(1), do I need to add additional D1 chart in the set up? I am doing that in Algowizard. Tks for any help. (peter56)

A: No I do not think so, since this should be handled by internal logic of the selected block. I’m regularly using Daily/Weekly blocks like CloseD OpenD etc… without adding additional chart.

Q42: I’m quite new to algo trading and SQ. Can you recommend where to start? Does it make sense to buy SQ right now or maybe wait until get more knowledge? Can you share your path to the level you are right now? (Jangg)

A: The best way to start is reading my blog (including my free eBook), I mean it there are a lot of resources here that will help you get “there” faster. It is always a good idea to do some investigation before buying tools like SQX. You could always download a demo and play with it. The easiest path to success is by understanding that there is no easy path 😉 It’s a learning process and it always takes time…no getting rich quick offers here.

Q43: Where do I start when buying SQX and how do you develop a good workflow that is robust? I read your blog about the proprietary workflow, it gives an idea but obviously it is vague enough to keep it proprietary. (Rianco)

A: I will repeat my mantra 😉 A good start is to synchronize your SQX with TS or MC and learn the most commonly used strategy validation and robustness testing methods. This will allow you to start producing and filter your strategies. Yes there are many “proprietary workflows” since every profitable trader has his own strategy development method and there are many methods that work. For example my workflows are mainly based on WFA, but it is not a hard requirement. Some trades use MC/SPP together with other validation methods. The key is to get confidence in the method you will adapt.

Q44: Hi is possible applying this test? https://www.buildalpha.com/noise-test/ (alexrios22)

A: No, not directly, however this kind of test can be easily replaced by corss-market test using a data from an another correlated market.

Happy trading!

Q45: 1. Just to double confirm in your workflow, the Final WFA (OOS), is that simulated forward test method? 2.For session analysis, what is the meaning of green colour? Any specific criteria to arrive that? Some green colour on RetDD and #. 3.What is the session B1-B5? Any specific period? (HangTuah)

A: AD1: Yes this OOS period can be perfectly used for the SFT method, however before running it you can do two things here: a fully automatic strategy pre-selection (filtering) OR a manual strategy selection, in order to see which method has better success rate. AD2: Green color is my own personalized filter of numbers and their thresholds which I like too see, eg: I like systems with above 700 trades. AD3: Yes these are some pre-selected trading hours as in the figure posted above the table.

Q46: Chris, I followed your suggestion and did a step-by-step chekcing the success rate. I found that the results are very dependent on data. For example, for 2021-2022 data, as I do more robustness testing , the success rates continuously drop from 12% to 2 % finally, which scares me. However, for 2022-2023 date, as I do more robustness testing (the same as previous one), the success rates keep improving from 74% to 96%, which satisfy me. Did you face such situation before? How to handle this? (Kaizhi)

A: Lol;) welcome in the world of algo-trading…yes this can happen, and due to many many reasons:
– at first you should look into how you decide if your system has ‘failed’ in the SFT OOS period, e.g.: my goal is to see similar behaviour as in the IS period, however I also want to see a decent number of trades in SFT OOS.
– maybe your validation tests are not harsh enough
– maybe strategies you produce are not robust enough
– maybe strategies are based on logic that cannot adapt to changed market conditions (this is the most common reason)
– maybe the market has a completely different behaviour in your FST OOS period than in your training / IS period
– maybe your IS period is too short
– or maybe the numbers of trades is too low
– etc etc…

I know most people expect “black and white” answers but it is not how the markets work. I tried and scanned millions of maybe even trillions strategies before I saw decent results. But once you know what to look for, it suddenly becomes very easy….every market has it’s own “mood”, like different “good” trading hours, different indicators that will work, different type of entry types etc etc…what helped me the most is analyzing the statistical figures from millions of backtests in order to see what works and what doesn’t. It is an extremely important step in my workflow. Let the data be your guide….

Chris