Clients often ask for strategies that trade on very short time frames. Others have heard of High Frequency Trading: The Zorro developers had been pestered for years until they finally implemented tick histories and millisecond time frames.
Or has short term algo trading indeed some quantifiable advantages? An experiment for looking into that matter produced a surprising result. Additionally, short time frames produce more bars and trades — a great advantage for strategy development. The quality of test and training depends on the amount of data, and timely price data is always in short supply. Still, scalping — opening and closing trades in minutes or seconds — is largely considered nonsense and irrational by algo traders.
But never trust common wisdom, especially not in trading. I can confirm reasons number 3 and 4 from my own experiences: And I never managed to develop a strategy with a significantly positive walk-forward test on bar periods less than 30 minutes. But this does not mean that such a strategy does not exist.
Maybe short time frames just need special trade methods? Higher costs obviously require more profits for compensation. How many trades must you win for overcoming the trading costs at different time frames? This script calculates the minimum win rate to compensate the trade costs for different trade durations. We assumed here a spread of 0. For breaking even, the win minus the loss must cover the cost. The required win rate for this is.
The win rate is averaged over all bars and plotted in a histogram of trade durations from 1 minute up to 1 day. The duration is varied in steps of 1, 5, 30, and 60 minutes. Or alternatively, a 9: This exceeds the best performances of real trading systems by a large amount, and seems to confirm convincingly the first reason why you better take tales by scalping heroes on trader forums with a grain of salt.
Naturally, short time frames produce more high-frequency components than long time frames. They could be detected with a highpass filter, or eliminated with a lowpass filter. Price curve noise is not always related to high frequencies. Noise is just the part of the curve that does not carry information about the trading signal. For cycle trading, high frequencies are the signal and low-frequency trend is the noise.
So the jaggies and ripples of a short time frame price curve might be just the very inefficiencies that you want to exploit. Thus we need a better criteria for determining the tradeability of a price curve. You can not trade a random market, but you can potentially trade anything that deviates from randomness. Randomness can be measured through the information content of the price curve. A good measure of information content is the Shannon Entropy.
It is defined this way:. A very ordered, predictable signal has low entropy. A random, unpredictable signal has high entropy. The Shannon Entropy is measured in bit. Zorro has tons of indicators, even the Shannon Gain, but not the Shannon Entropy!
This is the source code of the Shannon Entropy of a char string:. The frequency of each char is counted and stored in the Hist array. They are multiplied with their binary logarithm and summed up; the result is H S , the Shannon Entropy. In the above code, a char is a pattern of the signal. So we need to convert our price curve into char patterns. This is done by a second ShannonEntropy function that calls the first one:. PatternSize determines the partitioning of the price curve.
A pattern can consist of up to 8 bits, equivalent to combinations of price changes. The patterns are stored in a char string. Their entropy is then determined by calling the first ShannonEntropy function with that string both functions have the same name, but the compiler can distinguish them from their different parameters. So the patterns overlap.
Now we only need to produce a histogram of the Shannon Entropy, similar to the win rate in our first script:. The entropy is calculated for all time frames at every th bar, determined with the modulo function. I cannot use here the hack with skipping the next bars as in the previous script, as skipping bars would prevent proper shifting of the price series. Two code lines should be explained because they are critical for measuring the entropy of daily candles using less-than-a-day bar periods:.
This line was missing at first and I wondered why the entropy of daily candles was higher than I expected. A day has often less than one-minute bars due to weekends and irregularities in the historical data. The Shannon Entropy is calculated with a pattern size of 3 price changes, resulting in 8 different patterns.
As price changes are not completely random, I expected an entropy value slightly smaller than 3, steadily increasing when time frames are decreasing.
This confirms that price patterns are not absolutely random. We can see that the minutes time frame has the lowest Shannon Entropy at about 2. This was expected, as the daily cycle has a strong effect on the price curve, and daily candles are thus more regular than candles of other time frames. This is an unexpected result. The lower the time frame, the less price quotes does it contain, so the impact of chance should be in fact higher. But the opposite is the case. The x axis is now in second units instead of minutes.
We see that price randomness continues to drop with the time frame. There are several possible explanations. Price granularity is higher at low time frames due to the smaller number of ticks. All this reduces the price entropy of short time frames. I have again uploaded the scripts to the scripts collection. For the seconds time frames the tick data plugin is needed.
Interesting article — congratulations! Two details, seemingly contradictory to each other, irritate me: I would expect both values to be the same. Did you change conditions between those runs? Yes, they are based on different history files, the first on a file with M1 bars and the other one on a tick based file. Ok, but this would mean that those timeframes between 2 and 60 seconds aggregated from tick data seem to have a higher randomness as compared to native M1 data.
Recently I coded several MT4 scripts to check price data for quality, and the results were horrendous. So my guess would be to look at the price data when trying to find a reason for this mismatch. I work with various entropy-related metrics in my ML tasks. Could you please inform what the number of cases you have used per timeframe? I think something could be improved in your approach. When I add a counter to the script and count the number of cases, I get cases per time frame for the minutes and cases for the seconds.
Minutes were tested from the last 5 years and seconds from January and February First of all, each time we compare any results we should have the right understanding of the diff scale between them, eg. In other words, we first have to grade the values: Secondly, lets go back and recover what actually we assessed: Is that really what we wanted to count?
Splitting the price differences into grades would most likely produce a more accurate result since we then have more data. The Shannon Entropy can be monetized in some strategies by using it as a filter, i. We had a very successful trading system consisting of few mean reversion strategies that some people would consider HF. The number of trades was in thousands per week on several currency pairs. The trade lengths varied from few minutes to many hours. Well, with very high compound risk and some margin calls but still…: The more the better of course but even less than k can be sufficient if you are lucky enough.
Something like HFT is possible even for relatively small traders if they are able to find the right tools. Thanks for sharing your fx experience.
IB claims to give its member DMA for Forex, they also claim to have separate companies for market making and brokage services. Are you targeting them in your comment or just a coicidence? I consider them for my hf-like fc strategy, thats why the question.More...