My first 100% automated and 100% accurate workflow, StrategyQuant test case
DISCLAIMER: The presented results below are still preliminary, there is still a small chance that my positive results are influenced by an undiscovered bug in the current version of SQ-X (build 118.84) or that I’ve just made a stupid mistake somewhere in my workflow resulting in a huge ‘Data Mining Bias’. However I did my best and rechecked everything multiple times…Moreover since this all is based on a relatively new ‘custom projects’ feature of SQ-X, nothing of this has been tested yet on a real account…but I think I have built a strong case supporting I could be right on this one 😉
My claim: It looks like I’ve managed to create a 100% automated and 100% accurate workflow using StrategyQuant feature called ‘custom projects’.
100% automated means: I push 1 button before going to bed, and every morning my workflow automatically generates, validates and selects few new strategies which are ‘ready to go’
‘Ready to go’ means: I can deploy them immediately to my live account. Without a need of further processing.
100% accurate means: Every single strategy that has been selected by this automatic workflow (~50 so far), has been profitable in the 2 years period from the generation date.
To test this all I’ve adapted my SFT method as described in this topic: See HERE.
The workflow is based on standard validation test (common knowledge) as shared by SQ team in their free courses, however with a very rigorous settings. The workflow does not use any advanced validation methods like WFA,WFM,OP,SPP. Instead a customized Monte-Carlo test is used to simulate behavior of a SPR method. No portfolio analysis is performed (some systems can be correlated).
My automatic workflow test case is split into two verification moments:
1. End of year 2014.
2. End of year 2016.
At each point in time 1 and 2, I used my workflow to automatically generate and select 20 NEW strategies (out of several hundreds thousands systems) without ANY manual intervention and then ALL of selected systems where forward tested using SFT (future data). Let me be clear on one thing: I did not cherry picked any strategies.
It seams that every single selected strategy was profitable in the period following the selected generation date. See figures blow:
Test case 1: Strategy creation @ 2014.12.31, Simulated Forward test: 2015.01.01…2016.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)
Test case 2: Strategy creation @ 2016.12.31, Simulated Forward test: 2017.01.01…2018.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)
My conclusions so far:
1. If there are no mistakes, then it seems that it is totally possible to use SQ-X automated ‘custom projects’ to automatically generate and select profitable trading systems.
2. No advanced validation/filtering methods needed. Of course these tests should only improve total result and minimize DD on portfolio level.
3. The results in SFT of >2014 are slightly better than >2016. Workflow is somehow sensitive to used data during strategy generation (due to changing market condition). It seems that years 2017 and 2018 are very difficult years for trading using selected trading type.
4. It is not 100% proven yet, but it’s a pretty damn good result so far, taking into account it’s based on a simple workflow that is using basic filtering principles.
5. Some of the strategies can be correlated, but for the sake of this investigation no manual correlation filtering has been performed. This would jeopardise the objectivity of this test case.
6. The filtering settings are very rigorous, this workflow filters out only the most robust strategies. According to my statistics only 0.05% of the generated strategies are able to pass this workflow.
– Refine the workflow and implement further strategy selection, perform correlation tests, WFM analysis and additional portfolio level related tests.
Simulated Forward Test
How to test your strategy development process to see if your are really a profitable system trader?
Hereby I want to propose a validation method called ‘simulated forward test (SFT)‘. (Note: This is not WFA or WFM, WFA/WFM is a different topic, this is also not an additional OOS test as used during strategy generation since the SFT period is not used for strategy selection)
Simulated forward test is basically a ‘walk forward’-like test of your ‘total strategy design workflow/method’.
In the last few months I tried to develop a fully automated workflow using SQ-X ‘custom projects’ module and I got very interesting results (which I will post in another topic). My small breakthrough was based on an observation that majority of traders, and even experienced traders giving online courses, just assume they are right about their strategy validation method based on their experience and their past results. There is nothing wrong with that assumption, but it can be killing for new traders or people who are trying to design their own strategy design methods.
What you normally see is the following development sequence:
1. All traders have their own ideas or methods for generating strategies and they use these methods to generate their new automated trading systems.
2. In 90% of all cases, new traders generate and run their systems on demo (or small live) accounts to see if their strategies are profitable or not.
3. This results in a ‘generate & hope’ methodology.
In my opinion the proper approach should be as follows:
1. At first you need to develop or learn a new strategy design workflow (a new system design method or process including strategy generation/testing/validation and strategy selection).
2. In order to test if your method works you go back in time and pretend to be living in the past, e.g.: 2014. This is your point of reference.
3. Then you develop a sufficient (significant) amount of strategies using your workflow using data <=2014. In this case using data >2014 is forbidden to prevent any type of ‘Data Mining Bias’.
4. Then you test ALL strategies that have passed your workflow and strategy selection on a simulated forward period (your simulated future) >2014 and verify how many of selected strategies have worked for you. You do this without cheating or fooling yourself, so you use ALL strategies that have passed your workflow successfully. You draw your conclusions based several different pass/fail criteria that are important for you e.g.: Max DD, Net Profit, Red/DD, Stagnation etc….and you write down the effectiveness percentage of your workflow e.g: 65% of selected strategies were profitable (or did not exceeded Max. expected DD) in the simulated future.
5. You repeat the whole process by selecting a different point of reference in the past e.g.: 2016. And start all over again by generating, testing and selecting new strategies. Then you test again ALL of the selected strategies in a simulated period >2016.
You can repeat this process many times, by ‘walking forward’ your whole workflow in time using historical data and see if your strategy generation method really produces robust strategies in a simulated forward period.
An example of time slots as used in a typical workflow:
– Point of reference (in the past) = 2014.12.31 (you are forbidden to use data > 2015.01.01 until you have generated 30 strategies that passed your workflow and selection method for live trading, this can be including portfolio building methods)
– Your first IS/OSS for strategy generation is e.g.: IS: 2008.01.01…2011.06.01 and your first OOS is 2011.06.02…2014.12.31.
– Your second OOS2 for cure-fitting check is e.g.: 2003.01.01…2007.12.31
– All other validation tests (spread,skipping trades, MC, cross market etc..) are done using data <=2014.12.31
– You select 30 strategies and test them ALL on data >2015.01.01
After you are confident with your workflow and your results you can jump-back to present time and generate/validate and select new strategies and feel confident that selected systems have a significant chance of being profitable in the upcoming future.
Just wanted to share this because this approach has led to my first small breakthrough with my 100% automated workflow.
Conclusion: stop fooling yourself, stop generating strategies based on untested methods and stop hoping those will work out for you well in the future…
Any comments are welcome 🙂 Visit our forum HERE.
Free StrategyQuant X crash course 2019
When you join our community, you get an illustrated step-by-step guide on how to ‘jump start’ your own strategy generation process.
You will be able to download free strategies generated by other forum members! Including a start-up package with 50 example strategies and 3 strategies which are ready to go.
You get our new SQ-X workflow template that has proven to produce interesting results on EURUSD.
You also get access to additional algo-trading lessons, where you will learn the basics of systematic trading.
So why we do it for free? What’s the catch?
1. Because we are passionate about algorithmic trading.
2. Our mission is to share knowledge and learn from each other.
3. Strategy generation takes time and lots of computing power. With more active forum users and more computers working in parallel we can generate and test more strategies.
To join this free course please subscribe to our free algo-trading forum.
Note: After short registration you will recieve an email with further instructions. Make sure you have received the activation email! Otherwise mail to: info(at)coensio.com.
Will a free trading group help us?
How about we create our own!
many years of EA coding for other traders (also for many of you) I
think, I have seen it ALL. I have seen people creating EA’s and
strategies based only on one chart screenshot or just because they have
read something on a ‘forum’. Obviously it doesn’t work!
Now I’m fully convinced that the only proper way to go is automatic strategy generation and testing.
But this method is also not perfect, it is like trying to find a needle in a haystack. Learning curve takes several weeks, finding robust strategies takes several days, testing takes up to several months.
So why not to combine our efforts?
Here is my idea: I want to create our own private forum, our own trading group where we can share ideas, experiences and knowledge. A real gentleman’s (and ladies) club for serious traders.
YOU CAN JOIN HERE: Coensio’s Algo-Trading Group
I want to concentrate only on automatic strategy generation. This is how ‘they‘ do it. This is how EA vendors find their strategies, this is how professional quants and algo-traders design their strategies.
So if there is any chance you are already using StrategyQuant platform, and you want to join this club, just let me know (reply to this email). I am looking for experienced traders and also for forum moderators.
I’m still learning this platform especially the new ‘SQ X‘ version, so the only thing I can promise you, is that I will share my knowledge for free. Starting from 2019 I will also prepare some kind of free algo-trading course.
For traders which do not know what I am talking about, please read my review of this platform: HERE. I also provide 20% discount + other bonuses for people that want to purchase this platform, but first download a demo version and see if this way of working will work for you.
ATTENTION: If you will use my discount code you will get ArrowHunterEA and CoensioMt4Charts for free: COENSIO-20-OFF-ZSHXY
You can read more on my blog. or join here:
Coensio’s Algo-Trading Group
Coensio MT4 charts
Start monitoring your trading systems!
This tool allows you to generate equity graphs and to monitor all interesting
statistics of every trading system on your MT4 account.
Click on the figure below to see it in action!
This tool is an MT4 compatible indicator which can be attached to any chart to any timeframe. The generated report is a .html file, which is stored locally in MT4 folder.
Features & calculated statistics
- Generates charts of all (or selected) trading systems by Magic number.
- Total Profit, DrawDawn and WinRate.
- Wins, Draws and Losses.
- Long and short trade profits.
- Swaps and commissions.
- Daily and hourly Profit/Loss overview.
- System selection by Magic Number, LotSize or Start Date
- Possibility to add custom comments for each system!
This tool is an MT4 indicator that can be attached to any MT4 chart using any timeframe. On each new candle the indicator generates a full report which is stored as a .html file in local MT4 directory.
Yes a DEMO version of this indicator is available in the download link below. After a short registration you can test it with your favorite indicator on AUDUSD pair. DEMO version is limited to 3 systems only!
After purchasing the CoensioMt4Charts indicator, you will get your own License Key which will be linked to your personal MT4 account. This means 1 License Key is valid only on 1 MT4 account, at a time. You can always change your MT4 accounts, by sending me an email (without any additional costs).
In theory MT4 platform supports up to 100 different charts (=100 different tradings systems).
The CoensioMt4Charts indicator has been successfully tested on: Firefox, MS edge, Chrome and Safari browsers. It should work on any other moderrn browser.
Test or Purchase
Purchase NOW and get all future releases for free!
- EmailAddr: Your valid email adddress.
- LicenseKey: Your valid product LicenseKey.
- GraphFile: Name of .html file output. E.g: ‘MyCharts’
- CalcFromThisDate: Start date from which statistics are generated.
- MagicsToCheck: Magic numbers that need to be analysed. E.g: ‘2335,33344,234,…’. Empty = All systems.
- FilterByLotSize: Allows to filter all trades by specific lot size. E.g.: 0.1 or 0.01.
- TimeHourOffset: Time offset in hours of the MT4 broker, used to synchronise time visible on charts.
For Order Support, please contact ClickBank HERE
CLICKBANK® is a registered trademark of Click Sales, Inc., a Delaware corporation located at 1444 S. Entertainment Ave., Suite 410 Boise, ID 83709, USA and used by permission. ClickBank’s role as retailer does not constitute an endorsement, approval or review of these products or any claim, statement or opinion used in promotion of these products.
© 2019 www.coensio.com
FxMagnetic indicator review
+4500pips in the last 100 signals.
This time I will have a closer look at the FxMagnetic indicator. According to the makers of this indicator, the signals provided by this indicator are very accurate. The indicator has several different versions, each version is fine-tuned to specific currency pair (which makes sense from the algo-trading point of view). Of course everything is based on statistics and probabilities, but for example the current GBPJPY version of this indicator ‘predicted’ last 100 trades with the resulting profit of +4500pips and only <20% draw-down. Pros of FxMagnetic indicator:
- Well designed indicator based on statistics
- Each version is fine tuned to specific currency pair
- Non-repainting signals
- Indicator provides best possible settings for each time-frame (no configuration needed)
- Indicator provides targets for TakeProfit and StopLoss levels
- Impressive profit/loss statistics and very low draw-down
- Reasonable price
Cons of FxMagnetic indicator:
- The indicator buffers are well protected, backtesting is not possible
- Each purchased license works only on one specific currency pair
This all makes this indicator very interesting trading tool. Watch the video below to see an example of FxMagnetic on EURUSD pair.https://www.youtube.com/watch?v=RysU9UhfWy4 But is this indicator really profitable? Hard to say…the statistics look very promising and I know the coders behind this product and I know those guys know what they are doing. So let’s test it!
Automating the FxMagnetic with ArrowHunterEA.
For our forward test of FxMagnetic indicator we will use our ArrowHunterEa, which has been updated with dedicated ‘FxMag’ mode. In this special mode of operation the ArrowHunterEa will automatically recognize the trading signals of FxMagnetic and place the orders together with the corresponding TakeProfit and StopLoss levels. For this specific test we will use GBPJPY version of FxMagnetic. See it in action here: https://www.youtube.com/watch?v=rlUCbw71G7o
Trading results of the FxMagnetic with ArrowHunterEA.
Below the first 21 trades generated on GBPJPY pair using 0.01LotSize (+750pips since interception date at 06-11-2018):
Get this indicator here: FxMagnetic.
Free Video course: How to prevent curve-fitting
And how to properly optimize trading systems
is the most common mistake made by inexperienced algo-traders. This is
because the MT4 based optimization is nothing else but pure
Curve-fitting is a fancy name for cherry-picking (or ‘falsely’ connecting the dots), where you optimize your system using all possible combinations of all parameters and you select only the best results, the highest profits or the best looking equity charts.
This topic is so huge I can write a book about it. But let’s keep it very pragmatic for now. Because you are here to see some results. So, without going into details here is the approach that I use to keep cure-fitting minimized:
- Keep the number of optimized parameters as low as possible: each new parameter which is optimized adds an additional degree-of-freedom (or dimension), in which equity graph can be curve-fitted.
- Following the logic of point 1: avoid complex trading systems with many parameters and dependencies. Keep it simple!
- Split your historical data range in: ‘in-sample’ and ‘out-off-sample’ periods: the in-sample period is the longest period where you run your optimization. The out-of-sample period is the period where you will validate your settings after the optimization.
- Make sure you have sufficient amount of historical data. Note that you need at least 100 trades in your in-sample period, which is minimum required amount of trades needed for statistical significance. More is better.
- Optimize for profit-growth stability not for the maximum profit. In the end you need to see a straight equity line with stable increase over the whole testing period. (During optimization the ArrowHunterEa returns a custom ‘profit-factor’ in OnTester results column, which determines the stability of growth)
- Perform a robustness test on optimized parameter set. This can be done by manual ‘monte-carlo’ like simulation using the MT4 tester.
know it is a lot of info in one post. That is why I have prepared a
free video course for you with an real life example of strategy
Altcoin maximum price calculator
This tool calculates the maximum theoretical price estimation for “.$Ticker.” coin, with an assumption it will reach the same
marketcap as Bitcoin with a similar supply level. The equation is:
MaxPrice(“.$Ticker.”) = MarketCap(BTC) / (SupplyLevel(BTC) * MaxSupply(“.$Ticker.”))
Note1: if \”Supply\” is equal to \”MaxSupply\” like for monero coin, then SupplyLevel(BTC) is set to 1
Note2: if \”Supply\” > \”SupplyLevel(BTC)\”, then SupplyLevel(BTC)* MaxSupply(“.$Ticker.”) = \”Supply\”
$Raw = file_get_contents($Url);
‘; echo “
Usage: https://www.coensio.com/wp/altcoin-price-calculator/?ticker=TICKER, or click one of the tickers below:
“; echo “
GROWTH POTENTIAL (ROI) RANKING TABLE TOP 200:
“; echo “
Data source: coinmarketcap.com
The best crypto exchange
(Europe-based, trusted cryptocurrency market)
(Europe-based, trusted cryptocurrency market)
Step 1: Create your free account
Go to LiteBit.eu website and click “Create new account”:
Fill your personal information:
Complete your registration by clicking activation link in your email box:
Note: In some cases before purchasing your first crypto–coins you need to complete at least “Tier 1”, verification. During “Tier 1” verification the system will send you a SMS with a short code you need to enter in order to proceed with purchasing crypto’s. You can complete “Tier 1” step, under “My account > Verification & Limits” page. Click here for direct link.
Step 2: Getting a free cryptocoin wallet
A “wallet” is a tool which is connected to an online decentralized ledger (database) and it is used to store, send, or receive digital currency. The safest option is to get an offline wallet for each given currency type. However this requires installing many tools (since each cryptocurrency has its own wallet software) and storing information on external USB drive. The easy way is to use free online wallet inside your LiteBit.eu account. Go to “My Wallet” tab and click on “Deposit” button for selected cryptocoin.
After that you need to hit “generate” button in order to generate a new wallet address for selected currency. Note: in all cases you can always transfer your funds from
LiteBit.eu account to your offline wallet or a wallet software installed on your PC.
Step 3: Purchasing your cryptocoins
To purchase any desired cryptocoin just go click on currency symbol in the main window, for this example: XRP icon:
Then fill in the required information like: amount of coins to buy, your payment method, your wallet address and hit “Next”.
Note: Do not forget to check the “I agree” checkbox.
After that you can proceed with your usual payment routine:
Step 4: Checking transaction status
To check if your transaction is finished, you can go to “My account > Order History”:
Note: Depending on the crypto currency it can take few minutes to few hours to complete the transaction! Remember that each transaction need to be confirmed by crypto currency “miners” and it can take some time.
The LiteBit.eu is an easy, fast and reliable currency exchange market. It is based in europe which gives another level of confidence when compared to Asian based exchanges. I hope this guide will help you buy you first cryptocurrency without any problems.
P.S.: if you want to thank me by sending me 1 ripple ($0.20), my ripple address is:
How to backtest & optimize in MT4
Just like me, many of you are using the MetaTrader platform for daily trading using Experts Advisors. However, not many people know how to backtest in MT4 properly. In this short blog I will try to explain you the logic behind the EA optimization process on a MetaTrader 4 or 5 platform. If you do not know how to run the strategy tester on your MT4 platform then there are many places to go like YouTube to learn from. This article explains only the optimizing steps needed to get the best results during the optimization.
So here we go:
- How to approach mt4 strategy tester optimization
- Is 99% tick data accuracy one big lie?
- What level of tick data accuracy is needed for backtesting?
- Putting it all together
Now imagine today is the point where we have drawn a red vertical line. Of course we can not predict the future and the only thing that we can see are the backtested results from the past. So how can we make sure the optimized EA results are not curve-fitted and the optimized EA parameters have any chances of being profitable in the future? Let’s split our optimization process in two steps.
Optimize a given trading strategy on a small historical data fragment, from a given date in the past (Start) until today (End). It is extremely important that the number of trades in the selected period is representative for the given trading system. You need to optimize using at least 10% of the total number of trades which have occurred within history data range.
Test if the optimized parameters found in STEP 1 are not curve-fitted! Yes, this is my big secret magic trick, that I use in my all trading systems, so read and think about the following sentence for a while, let it sink in:
Basically, if I see nice results on a backtest using history data which was never used during the optimization (so called out-of-sample period), then it gives me confidence, that selected set of parameters could also be profitable in the feature.
modelling quality! I was obsessed by tick data accuracy. I spent hundreds of dollars on tools which gave me 99% accurate tick data….but then I’ve discovered the following:
The 99% accuracy visible after a backtest is just a number in the FXT file header!
And moreover what does it mean to have 99% of accuracy? The answer is: it doesn’t mean anything! It is just representation of tick modeling quality for the broker where the tick data comes from and in almost all cases it is Ducascopy tick data. Furthermore in most cases people will use different broker with different tick prices, spread, latency etc…So 99% backtest data modeling is a myth, but 99% data is not entirely useless, read the flowing point.
- You are using MT5 platform and run your optimization using: “Every tick based on real ticks” mode. (Even then, most brokers provide slightly “adjusted” tick data, so no, they will not let you win that easy, hehe)
- Your broker will provide you the real recorded tickdata from a live account. (It does not happen a lot)
- You know how to record live tick data by your self and know how to export it to strategy tester. (Too difficult for most people)
In order to test your historical data for accuracy just run the following simple test:
- Run your (or any other) EA on a live account on a selected broker. To minimize the potential losses minimize the lotsizes of open positions.
- Collect the live trading results for at least few days (or even weeks) in order to capture different market moods and events.
- After that, backtest the same EA using exactly the same EA settings and using the same time period! (In this step you can use 99% modeling or the tick data you currently have)
- See how big are the differences in trading results between live and backtested data.
- If the difference in all critical parameters such as: number of trades (very important), drawdown and profit etc..are not that big, then your history tick data is sufficient to be used in optimization and backtesting. If the differences are significant, then you need to test it on a different broker or using different source of tick data.
- Your EA settings are not curve fitted
- Trading results are also profitable outside your optimization period (in the past or the future)
- There is a good correlation between results from backtesting and optimization and the your live account
P.S.: do not forget to like or share this article!