The Trading Sciences Technologies platform uses Tradescapes as the principal visualization for assessing the tradable performance of financial entities.
A tradescape is a visualization of over 700 backtests whose principal results are plotted in a single 3D response surface. The premise is that many of the key questions that typically go unanswered in the investment and trading sciences can be had in a very fast procedure that is independent of all real-world trading system signalers and algorithms. By trading order, using a generalized ideal signaler, we can learn a great deal before any step is taken in opening a position or designing a trading system signaler.
To understand why a tradescape is constructed as it is requires an understanding of trading order as well as each of the three axes used in the tradescape visualization.
Trading Order - Investing and "Trend Following"
A tradescape specifically addresses the ordered signaling aspect of any investment or trading scenario, whether such is from technical analysis or human judgement.
For something universal like a tradescape to even exist, it must necessarily deal only with the order in the price movement of a time series. That means the entry into a position does not occur until a price movement in the direction to be traded actually begins. It also means the exit or close of that position does not occur until the growth or trend generating the profit has dissipated or turned. This is what most trend traders and investment professionals using human judgment seek to do. One can safely assume that order, from the perspective of tradescapes, means that movements to a new trading range are gradual and sustained in a wide sense, not an abrupt one-day movement to an entirely new trading range. If most of a time series' significant movements occur in sharp jumps, we regard that as disorder or chaos.
We suspect it is impossible to build a generalized signal algorithm for accurately processing chaotic price movements in a time series. What we can do, however, is build a generalized signaler for this situation where the price movements are well-behaved, where the trending is assumed to have this quality of order.
In such a design, we can answer perhaps the most critical question of all:
For trading a given entity, how good do I have to be in the entries and exits to realize more reward than pain in my trading or investing?
Intuitively, we know that some entities move erratically and turn sharply and swiftly in price. Others move more gently and are more forgiving. Tradescapes help identify those entities that are well-behaved in how trends are formed and how they dissipate and turn.
When an investment is made long term, there is an implicit assumption that the price of that entity will grow across time. The same is true for any intermediate to long term trend following system. There has long been the understanding that it is perilous to invest in weak entities on the bet they will turn around. The traditional advice is to invest only in strong entities, those that have performed effectively in the past.
We not only want to find entities particularly amenable to trending in a well behaved manner, but also those that have consistently and strongly experienced growth in valuation across whatever measure of history is deemed pertinent to the present state of the entity.
The reasoning for seeking entities of strength is mathematical. If we filter out the influence of starting and ending dates, and fit a smooth trend to the prices across a historical period, we can estimate a robust CAGR (a cumulative annual growth rate) that is less sensitive the the starting and ending dates of the segment. If one entity has a 30% robust CAGR (or "trend") across the past 10 years, and another has a 0% robust trend, the mathematics is quite simple. If one builds a trading signaler for the first entity that is in the market half the time, using purely random signals, one will, on average across many Monte-Carlo trials, realize a 15% robust trend. That would be accomplished, again on average, with no skill whatsoever in how the trading signals are generated. Of course, one would realize the 30% from a simple buy and hold across time, but the point is that the random equivalence point for 50% in-market is a lovely 15% baseline. All intelligence in the signaling adds to this baseline.
Now consider the security that has shown no growth in price across ten years. If we randomly signal that security in the same manner, we can expect no return whatsoever, again on average. Any intelligence we add from the signaling must start at zero from a Monte-Carlo statistical baseline.
Of course, one invests in that security that performed poorly in its recent history because there is a belief it will turn around, and once again evidence true growth. From our perspective we say that such a scenario, if it occurs, is also representative of disorder. The future behavior is not presaged by the recent history. Something quite profound must change going forward.
If we are looking at order, we look for true growth historically, and we do that by computing a fitted trend where a regression takes every point into account, not just the two points representing the start and end points. Most of the metrics that assess the performance of trading systems do not do this. And most of us know that one can make almost any entity look quite good (or conversely quite bad) by selecting starting and ending dates that portray the picture one wants to convey.
For trend followers and long term investors, the reward-risk metrics such as the Sharpe ratio convey little of the real world pain of investing or trading. Such reward-risk ratios compare the annual return for a given period against the annualized volatility for that same period. Risk is defined as a daily volatility that is extrapolated to an annual value based on statistics that assume normality. Most traders know these assumptions are nonsense, that fat tail events occur far more than normal statistics would suggest, and also that prices combine across days in ways that are very different from the random walk such a normal assumption implies. While our tradescapes can be plotted as Sharpe ratios, such a metric is probably not what a trend trader or long term investor would want.
In long sustained holds that ideally follow the trends in growth, it is the drawdowns in equity that are so painful. The value of one's portfolio reaches a peak and then declines. The measure of that decline in both magnitude and duration is what most trend traders and investors know as pain, and this is for investments that rebound and eventually recover that peak. For those that never do, the pain is decidedly greater.
There are two very effective approaches for defining this kind of pain. One is R³, a metric that looks for some n count of the worst drawdowns over some period of time, and uses that to generate a pain value that is used to offset the gain in a ratio. While we like R³, it is somewhat ad hoc and it is not continuous in terms of its computation. The pain computation for each drawdown uses a duration that requires the full recovery to former price. The difference between an entity just barely recovering and then going nowhere for a long period of time and one that doesn't quite recover and again continues going nowhere for this same time can be dramatic, and the only difference is whether or not the high on one day comes back to this peak or fails to do so.
For the reason of consistency, and to remove all ad-hoc elements, such as the count of drawdowns to include for a given count of years, we prefer RRt, a robust reward (R, the robust CAGR or trend) ratioed against the average retracement from the peak (Rt). This average retracement accounts what could be viewed as an exact measure of the pain at any given point in time. If the retracement is near zero, one is near the all-time high, and pain is non-existent. If the retracement is near 50%, then one has watched half of the earnings on paper disappear in a large drawdown. The pain is severe. By averaging the retracement for each day or bar across the data, one comes up with a reasonable estimate for the pain. It will not be the worst-case pain. It will be rather the average pain across the historical period. Entities that quickly recover from drawdowns and spend most of their time near a succession of all-time highs will have a very attractive average retracement. Those that recover very slowly, if at all, will have very unattractive retracements.
In the sample plot above, a ten year long-only tradescape for QQQ that uses daily data, the Z-dimension and the color scale reflect the RRt value. In this QQQ tradescape, we show only RRt>1, meaning we only wish to see those zones where there is more reward than pain when trading the order in the price movements. That doesn't mean one necessarily loses money in all zones that are not visible. It is a thresholding that looks only at those zones where the historical reward exceeds the historical pain.
A Real-World Trading System for QQQ
Once we know the particular metric we want to see with respect to reward-to-pain as the outcome from the many backtests, we can address what it is that the tradescape backtests actually study. To understand in depth why tradescapes represent a next generation technology for designing trading system signalers, we will first look at a real-world trading signal design that will point out why a tradescape is constructed as it is. We will start with a real-world optimization of a simple trading signaler. We will look at an RRt response surface as derived from varying the fast and slow lengths of an SMA crossover. We will optimize this signaler on QQQ for that same 10 year period used in the tradescape, again trading long only.
This plot is similar to the tradescape in terms of the Z-variable. We use the same RRt reward-to-pain ratio. Although we show only a subset of the optimization (zooming-in on one large region of tradability), the full optimization study actually consisted of thousands of backtests using different combinations of the two moving average lengths. Here the X-axis is the length of the slow SMA (MA 1) and the Y-axis is the length of the fast SMA (MA 2).
In this particular plot, because we are looking specifically at the outcome of trading systems, we plot delta RRt, the difference between the RRt from the trading system and the RRt from the underlying buy and hold of QQQ. We do this because we want a trading system to improve the reward-to-pain of the buy and hold. Otherwise, why trade it at all? For this signaler, the optimization shows a zone where this signaler offers an improvement of up to +1.0 in RRt. With the underlying having an RRt of 0.61, anything that doubles the reward-to-pain is significant.
Let's analyze what we are really doing in this optimization. The slow MA determines the central channel or equivalence point of the time series. It is a very smooth, very lagged picture of this central channel that uses a great deal of information to arrive at this estimate. The fast MA is a smooth representation of the more local movement of prices. It has lower lag and it uses less information to arrive at its estimate. When that local movement crosses above the central channel, a trend is assumed as having begun and a long position is taken. When the fast MA falls below this slowly changing central channel, the trend is assumed to have ended and the position is closed.
In such MA crossover optimizations, the effective parameter zones are often a bit precarious. Let's use a different 3D visualization to better see where robustness in the response surface, if any, can be had.
By looking nearly straight down from above at the surface, we see that the strongest performance rests with MA lengths in the vicinity of [75,60]. In general, we prefer sharper differences when crossing two MAs. The two SMAs, very close in length, suggest that 15 points, fully 60 bars in the past, will be the only difference in generating the signal. Even though the improvement in reward-to-pain is higher at these settings, an experienced system designer would probably choose the softer plateau. A good length might be [120,45]. The trading signal difference still rests with data well in the past, but here 75 points, starting 45 bars in the past, make the difference for the signaling.
There are those who lean toward minimal optimization who would argue that there is only one real zone of performance here, and the two (or three) "hot" regions are more from chance than anything deterministic likely to persist into the future. If we were to go along with that argument, we would choose the center of mass of the whole performance region, perhaps 100,50.
Let's assume that we take the necessary steps to certify the [120,45] setting as a viable signaling algorithm for trading QQQ. We thus conclude from different in-sample data periods that these settings have been close to the optimum across time, and from different reserved or walkforward periods that these settings have generally been effective.
At this point, it is fair to ask what have we learned in building this basic SMA-crossover signaling system for QQQ. It is also fair to ask what we don't know.
What We Do Know
We know that we chose a robust zone in the optimization of the parameters of an SMA crossover that signals QQQ. We also know the specific backtest results for each of the systems analyzed in the optimization matrix. The following is the equity curve for this SMA 120x45 trading system where the 2500 day in-sample data are at the end of the data sequence, and the reserved data consists of the wild Internet surge and bubble burst period at the start of the data.
The blue equity curve is the buy and hold. The red curve is the long-only traded equity curve. The in-market periods have a white graph background, the out-of-market gray backgrounds. The two equity curves have the same starting value 2500 bars back. This occurs around bar 900.
On first inspection, it would seem little was gained from the active trading, but if we look carefully we see that the red traded equity curve appears to have a good deal less pain across the nearly 14 years, especially when looking at the bubble burst and the financial meltdown periods.
What do we specifically know? We know the primary zone [120,45] that we assessed as the most appropriate for this signaling system results in very long duration trades. There are just 10 round trip (buy and sell) trades in the last 10 years, eight for wins, and these ten average 167 days in duration. We know for these ten years, the RRt of the underlying, 0.61, was more than doubled by the optimized trading system, to an RRt of 1.36. We know we were in the market 67% of the time across those 10 years. We know the robust annual trend or CAGR of the traded equity curve is +9.1% for the most recent 10 years. We know the robust annual trend of the buy and hold was +6.5% for this same period. Despite less time in the market, the trend estimate that accounts all points in the ten year period shows a +2.6% annual improvement from using the trading system.
We also observe that robust statistics are essential. The actual CAGR of the buy and hold for these ten years, using just the start and end dates, is +12.0%. For the traded equity it is +9.8%. The start date chosen just happens to be at the very bottom of the Internet bubble decline, the most opportune start date one could possibly choose to show an artificially high rate of return.
We most certainly know we don't like the 59% drawdown that takes place in the initial out-of-sample bubble period, even though it is far better than the 83% drawdown of the buy and hold during this same period.
What We Don't Know
If we step back, we can readily see there is a very great deal we do not know.
1. We have no idea if this traded equity curve of QQQ is a great achievement, something mediocre, or a system that doesn't really deserve be traded.
2. We have no idea if QQQ is an entity that fares well in terms of trading order, as is done with a trend-following smooth signal, such as this MA crossover. There may be a great many entities far more amenable to this type of signaling.
3. We have no idea if this nearly half-year average trade length is one that represents something close to optimum, or if it is far from optimum but the only thing that just happens to work within the constraints of the signaling algorithm.
4. We have no idea how accurately we catch the turns that would be seen by an ideal signaler that fully maps the order in the QQQ price movements.
5. We have no idea what the average lag is with our signaling systems. Since we use simple MAs, we know we have a (120-1)/2=59.5 bar lag in the central channel MA, and a (45-1)/2=22 bar lag in the signaling MA. We generally have no idea what the average lag of the crossover happens to be, if there is any difference between the lag of the entries versus that of the exits, nor what the scatter of those lags at the turns of the signal actually looks like.
6. We have no firm certainty that random chance did not generate the performance zones. The parameters chosen from the optimization were not in a wide, robust, sharply-defined zone of stability and the performance of the selected system was based on just ten trades. We have no idea how vulnerable any of these systems might be for suddenly ceasing to work. Because both signaling components are very smoothed moving averages, performance arising from pure luck or random chance can appear to have some breadth in an optimization response surface.
7. We have no estimate of the actual growth-based order in the QQQ price movements at the various time horizons possible for trading it.
There are clearly many items not covered here. From the example, we don't know if the various types of stops can significantly improve the RRt that comes with trading QQQ. We don't know if pyramiding or position sizing algorithms for entries might make sense. We don't know if a partial booking scheme or reverse pyramiding positions for exits make sense. While tradescapes do not address these issues, extensions to tradescapes can address two other major unknowns that are often critical to trading individual securities.
8. We don't know if trend or sentiment filters, procedures that limit trades in certain periods to long only or short only, based on sentiment or prevailing trending, can improve the RRt of trading QQQ. We could run the backtest with a wide selection of real-world sentiment filters and targets to see if there was an improvement, and as such the answer can be inferred without tradescapes, but it is something that can be immediately known, within a few seconds, by generating what we call a sentimentscape. In general, the sentiment target is not the entity being traded, but is rather an overall market or segment index. The fact that one must generally address two entities instead of one can dramatically increase the degree of difficulty of adding sentiment-augmented signaling to a trading system. Sentimentscapes simplify the process in answering the sentiment target and its settings before beginning any trading system design.
9. While in this case QQQ should represent an average of many Nasdaq entities, we don't know if there is a surrogate, such as the Nasdaq 100 index proper, or another entity that contains greater order than QQQ and which would represent a better target for generating the trading signals for QQQ. While we could again run the backtest with a full selection of potential surrogates, including indices which are not themselves traded, an extension to tradescapes also makes this question something swiftly answered, assuming one has sufficient market knowledge to pose a set of reasonable potential surrogates. This is done using what we call referential tradescapes.
How Tradescapes Work
The problem with real-world signalers can be put very succinctly. Most of the time, with most entities, with most settings of parameters, they simply don't work. They do not improve upon the buy and hold of the underlying entity. In fact, they can result in very significant losses. Even if one uses a signaling system that uses some measure of price smoothing, so as to presumably trade order, in many instances there is no clear answer to any of the questions we posed above.
The problem thus needs to be redefined. In order to assess how much ordered tradable growth there is for a given entity, for each time horizon or information content used for the signal, we first need an idealized set of signals that contain zero-lag. In other words, we seek perfect trading signals that mine as much of the order within a time series as is possible. That alone tells us a great deal about the ordered growth, resilience and tradability of the various entities we might choose to trade.
To carry that assessment into the real world and render it meaningful, we must maintain that accuracy, but degrade the signal by known shifts in time, lags that are very precisely known, are exact at every signal transition, and which tell us how good our signaler must be in terms of lag in order to achieve, under the best case scenario of full accuracy with respect to the order within the system, a reward-to-pain we find acceptable.
Why Tradescapes Matter
In general, even a wide matrix of parameter ranges in any trading signal algorithm optimization will map only a modest portion of the time horizon and lag covered by a tradescape, and much of that which is covered will generally be useless, outside regions where genuine performance is to be had. Further, at every one of those points in the optimization, the accuracy of the signal will be in question. That is just the way of it. If it were easy, every signal design would yield clear and consistent performers. We suspect that most experienced trading signal designers know quite well that a great deal of time is spent wandering about algorithm parameter space that has absolutely nothing useful to offer.
A tradescape, because it uses an ideal signaler, isn't limited by the constraints of what can be achieved with any particular style or class of real-world signaling algorithm. The signal used in the tradescape cannot be used for real-world signaling because of its look-ahead, but it is able to do something remarkable. It is able to map the entirety of all tradable order within an entity, from the fastest to the slowest of signaling, and from immediate to the most delayed of responsiveness, and it shows the trader exactly what a full accuracy signaler trading just the order within the time series would have achieved historically at each time horizon and signal lag.
Unlike that which is observed with so many trading signal design experiments, a tradescape never fails. It will show you precisely where an entity is tradable in terms of ordered growth and where it is not. It does this for you in a few seconds, and it is applicable to every potential signaler that trades order in some manner. The challenge that remains is the one that should actually remain, designing the real-world algorithm that realizes the needed signaling. With a tradescape, however, there is a head start. We know what we can potentially achieve and we know what we are trying to build. We don't begin in the dark.
Time Horizon, Lag, and Full Accuracy
So far, all we've shown is that we set the Z-variable to the robust RRt, something that very cleanly addresses the reward-to-pain in trading systems. In order to make tradescapes possible, we map what we regard as the two principal variables in the trading system that we care most about, that we deem most important, and we fix the third in the only useful place it can have.
Let us begin by a simple logical progression. Let's again look at QQQ. Across 2500 bars of data, the 120x45 SMA produces 21 crossings. This is the same thing as saying the signal is the difference between the two MAs, and that it has 21 sign changes across that time. That is our reference point. We can then generate an ideal signal that also has this same 21 signal transitions across this same time when processing QQQ. For any number of reasons, it may not be possible to exactly match the transition count since two different algorithms are involved, but we will mathematically come as close as we can. When that is done we find that our ideal signaller achieves the same transition count with a time horizon or information content of 45.8 bars of past and present information. We now have a standard reference for the length of our real-world MA crossover signal. Not surprisingly, it is very close to the length of the fast signaling component in the MA pair.
To achieve the zero lag in the ideal signal, we use future information, but that is not used in the value we assign for the time horizon-it is obviously not available to real-world signalers. That additional future information is used to secure a full accuracy in the signal, not for additional smoothing.
We now have an ideal signal with an information content of about 46 bars and the real world signal which we assign this same time horizon since it is at this length we match the signal transitions or trade count. We can now compare the two signals and for each of the transitions in the ideal signal, we can look and see if we have a corresponding transition in the real world signal, and further we can measure how many bars that response lags the ideal. For every entry and exit in the ideal signal, we can first check and see if it exists in the real world signal and for each we can measure the amount of lag that is present.
The first thing we will do is a reality check where we look for the maxima in the correlation between the two signals. This correlation lag looks at every point in both signals, not just those that trigger the entries and exits. When we do that for this example, we see a correlation lag of 51.2 bars. This is generally very accurate, though not directly related to trading performance.
We noted that the 120 length central channel SMA has a mathematical lag of 59.5 bars and the 45 length signaling SMA a lag of 22 bars. Clearly the lag of the crossover is between these two values, but closer to the lag of the slower component.
Since we care about the lag at the actual signal transitions corresponding to the entries and exits, we measure those as well. We find that robust estimates of these lags are 51.3 bars for the entries and 52.3 bars for the exits. We have good agreement, and what would be expected from the correlation lag and the symmetry of an SMA crossover. We can now use the 45.8 time horizon and the 51.8 lag and compute a lag fraction of 1.13. The lag is 1.13x the time horizon.
We have now dealt with the three factors we regard as most important in trading system signalers. We computed a time horizon or information content that fits will with the fractal nature of markets. We have estimates for the lag at the entries and exits. And we signal with full accuracy, or at least as much as the best designed EM (expectation modeling) science makes possible.
The accuracy in the 120x45 SMA signaller is 100%. It caught all nine of the trades we were able to generate with the ideal signaler. The MA crossover added a tenth, however, which was not present in the ideal signal.
The Real-World Signal and Tradescape Results
Because we know the real world signal tested with an EM length of 45.8 and a lag fraction of 1.13, we can now generate an ideal signal not only with this information content that matches the transition count as closely as possible, but we can now deliberately lag this ideal signal those 51.8 bars to replicate the lag in the MA crossover.
QQQ System Tradescape
EM Length 45.84 45.84
Lag 51.79 51.79
Lag Fraction 1.13 1.13
RT Trades 10 9
Av RT Len 167 181
Win % 80.0 77.8
Wrong Side % 27.16 28.11
Robust CAGR 6.48 8.75 9.94
r^2 Fitted Trend 0.61 0.91 0.92
Robust Sharpe 0.27 0.54 0.63
R^3 0.23 1.17 1.27
Av Retrace % 10.37 6.49 6.14
RRt 0.61 1.29 1.54
Here we compare the reward-to-pain metrics of the underlying, the MA crossover system, and the corresponding tradescape signal with the same time horizon and lag. The system does well, and it approaches the tradescape results. They are close.
There are some fine points associated with accuracy and lag. The lags are uniform for the tradescape signals, but they will have a certain scatter in the real-world trading signal. Those entries and exits with the higher lags may occur at inopportune times or at more harmless ones. That impacts the results. Also, a little error or spurious signals relative to the ideal reference can go a long way toward degrading performance. Each trade consists of both an entry transition and an exit transition, so losses can follow when just one side of a trade fails to signal properly.
In this example, we did well for such a high lag fraction. And in what should be very reassuring to a serious trader, we know precisely why.
Tradescape Signal Analysis
We will now look at the QQQ tradescape with a contour plot. This type of visualization makes it easier to see trading signals that are plotted atop tradescapes. The 120x45 MA crossover is plotted as system 1 below. It is circled to make it easier to see. It is colored by the tradescape scale, and since the RRt of the trading system is reasonably close to the color of the tradescape at this region of the surface, the point tends to blend in with the background. That is precisely what one hopes to see.
In this case, we see that "knuckle" we observed in the original 3D visualization that extended out to high lag fractions. The system we chose happens to rest in a very good place atop it.
How does one explain luck in the trading world? This is an example of a very simplistic signaler that works well with a hard to trade entity and it does so because it happens to rest in a time horizon-lag region that is very forgiving. We could build far more responsive (lower lag) signalers, and see little benefit, at least in this historical analysis. Of course, if we had a 0.9 lag signaller with the same accuracy, we might see only a modest benefit in additional return, but such a signaller would be far likelier to survive any major sea-change in the market dynamics of how QQQ is traded.
In general, we can say at this point, with some confidence, that the trading system design was sound. We have done very well with a technologically weak signaling algorithm, but one that trades order very effectively. And by good fortune. there just happens to be a zone where this entity behaves in an orderly fashion, one that forgives a great deal of delay in entering and exiting positions.
Progressive Tradescapes are simply a set of tradescapes that are sequential in time. Each represents a subset or segment of the overall historical period studied. We can use progressive tradescapes to see if this knuckle has been persistent in a wide sense across the 13 years of QQQ data that is available.
The Z scale was changed to >0 instead of >P. This allows us to see all tradescape activity where a profit occurred rather than only where the reward exceeded pain. This is more practical for smaller time periods since there will be market periods where even the most clever of long trading systems will not realize more return than pain, but may still eke out a profit. The first of the progressive tradescapes is the tech bubble collapse period. If that is forgiven (ie., assumed to be unlikely to occur again walking forward), we can look at the next three progressive tradescapes that cover 2003 to current time. In all three of these, the [45.8,1.13] tradescape setting corresponding with the trading system is profitable. It is also rather striking that these zones have moved around across time. The third panel of the series includes the financial meltdown. QQQ did reasonably well through that difficult time, at least when viewed in this wider sense.
And so we come back to our original question:
For trading a given entity, how good do I have to be in the entries and exits to realize more reward than pain in my trading or investing?
For this QQQ example, we can say we do not need a technological wonder in terms of a signaler in order to realize an effective reward-to-pain. We do need to trade the right time horizon however. And we see a very fast trading zone with a lag fraction between 0.7-0.8 which certainly teases.
Trading Order and Chaos
In general signals that trade using only smoothed entities, such as our crossover between two MAs, will primarily trade order. The smoothing removes much of the influence of chaotic movements.
Signals that trade using the actual price against some reference process a good measure of the chaos in the price movements. If one has an envelope or bands around a central channel, or bands in general, and a one day chaotic or fat-tail event moves the price outside those bands, a trade is immediately signaled. A stop is nothing more than the addition of such a threshold on one side of the price.
The obvious question is to what extent a technology that analyzes the order within the price movements, such as tradescapes, has value when there is a chaotic aspect to the real-world signaling system.
While it is possible to purely trade order (and there may be much to recommend such for those entities amenable to such), most trading systems combine a response to both the orderly and chaotic movements in price.
The simplest form of disorder or chaos could be easily perceived to be fat tail events, large movements that occur at a frequency or magnitude that would be little expected from the overall volatility or standard deviation of single bar price changes. The large unexpected movements in the opposite direction of a trade can be captured simply by reacting directly to price instead of a smoothed representation of price. While this can be as simple as signaling price above and below a single moving average, a blended system typically involves some kind of price or density band to prevent the whipsaws that would come from the lack of smoothing in the values triggering the trades. Basically, if you want to use price directly in an effective way, you must put some kind of vertical band in play that separates the entry and exit price thresholds.
Those bands can be derived from a variety of sources and can consist of many different algorithms. The bands can be determined by a density or scatter of one price, such as the close, or by two as in using the high and low. One can use the the mathematical error or uncertainty associated with estimating that central channel at any given point. A volatility band, like the Bollinger, uses two bands constructed using the standard deviation type of volatility. They bands could also originate with different moving averages, and their different lags could be addressed by displacements in time.
The more one focuses any trading to the tails of the density or extreme events of the scatter, the more the estimate(s) of that scatter will reflect the chaos in the price movements. By the time one is using range-based estimates, such as the ATR (average true range) type of volatility, one is maximizing the measure of chaos which is included in the computation of the bands surrounding a smooth central channel.
The simplest density bands are the breakout bands consisting of the range formed by the highest high and lowest low each for some specified count of bars. These are also the most chaotic of the band-based signalers since these trading bands move very abruptly across time and there is no central channel at all since these bands are formed of absolute prices and no computation of a smooth central price channel occurs.
To see the value of tradescapes in trading systems that seek to process chaos, we will optimize a breakout system for QQQ.
A BreakOut Example
In breakouts, the highest high in so many bars determines the upper threshold and the lowest low in a separate count of bars determines the lower threshold. There is no scatter per se, only the absolute bounds in price, the worst and best cases of that scatter. Breakouts actually shun order, since there is no central channel. They seek to react to jumps to new trading ranges, abrupt or otherwise. In a sense they really map the chaos insofar as the extent of each of the bands is determined by all of the fat tail events that happen to occur within the two windows.
Of course, breakouts process order as well. They don't tend to do so particularly efficiently, and they are sometimes ill regarded for that reason. Further, certain breakout settings are absolutely necessary to meaningfully process the orderly aspect of the price movements. If one ponders this closely, there is a sense that any optimization of the lengths of the highest high and lowest low windows deals mostly with a memory effect that is closely related to the order that is present in the price movements.
These widely used n-day breakouts for entry and exit points are sometimes called turtle signaling. When the highest high of prior days is exceeded by the high on the current day, one assumes that an upward trend has begun. When the low of the current day falls below the lowest low of prior days, one assumes any upward trend has ended and that the reverse has begun. By their very nature, a breakout is likely to have a much wider scatter of lag at the signal turns than the MA crossover example we first covered.
In terms of implementation, we will trade at the close on whatever day such a highest high or lowest low threshold is crossed intraday and as in the first example, all returns will be reinvested.
First, let's return to QQQ for 2500 days of long only trading and optimize the n-day length for the highest high (HH) as one parameter, and the n-day length for the lowest low (LL) as the second parameter. Will will again use delta RRt as the reward-to-pain performance metric.
This optimization shows a potentially higher benefit from turtle signaling for QQQ, but it also evidences some very narrow peaks. Our experience is that this is not uncommon. We often see sharp peaks near those values traders traditionally use for breakouts, especially around the values of 10 and 20 for the lengths. This is clearly suggestive of cause and effect. The response surface for breakouts is sometimes a picture of what a lot of professional traders are doing.
We will explore the value of tradescapes in anticipating an effective turtle signaling system. Breakouts by their nature respond to sharp, abrupt, fat-tail events, but they do not always respond swiftly. If one is in a position where the price has recently risen sharply, the running lowest low or LL may be quite far removed from the current price. It might have to fall long and hard before an exit is triggered, even to the point where all gains are lost.
Let's look at the optimization in a patch plot that shows every backtest in the optimization:
What we would ideally love to see is what happens near that peak we observed for fast trading in the tradescape. As is typically of most breakout optimizations, there is no single zone that is a clear optimum. Let's pick four that make some sense:
 nHH=9 nLL=18
 nHH=11 nLL=18
 nHH=10 nLL=41
 nHH=16 nLL=41
Let's analyze the performance of these breakout settings in conjunction with the QQQ tradescape:
Here we look at the QQQ tradescape using a contour plot that we've zoomed in. Even though the system trades both order and chaos, we would like to see if we get a better results near that fast trading zone peak in the original tradescape. Here we plot the four breakout trading systems on the tradescape, each at their equivalent EM lengths and lags, and we color each system with the RRt scale used for the overall tradescape. We have also changed the lag fraction scale to accommodate the lower lag signaling. Here especially, at these lower lags, we would clearly love to see one of these systems colored close to the tradescape surface around it.
This we do not achieve, but we did fare well once again:
QQQ [9,18] Trndscp [11,18] Trndscp [10,41] Trndscp [16,41] Trndscp
EM Length 9.83 9.83 10.0 10.0 19.0 19.0 25.0 25.0
Lag 7.10 7.10 7.79 7.79 13.0 13.0 15.12 15.12
Lag Fraction 0.72 0.72 0.78 0.78 0.68 0.68 0.60 0.60
RT Trades 47 46 46 46 25 25 21 21
Av RT Len 37.2
Win % 51.1 71.7 52.2 67.4 56.0 68.0 57.1 76.2
Wrong Side % 12.22 20.63 15.09 21.85 5.33 19.85 9.51 19.7
Robust CAGR 6.48 10.76 19.75 9.56 15.64 11.28 16.09 10.91 18.28
r^2 Fitted Trend 0.61
0.96 0.99 0.94
Robust Sharpe 0.27 0.61 1.22 0.56 0.98 0.58 1.00 0.59 1.09
R^3 0.23 1.38 10.45 1.04 4.16 0.99 4.32 1.00 6.98
Av Retrace % 10.37 5.27 2.53 5.46 3.23 5.54 3.67 5.44 3.57
1.94 7.12 1.67
Covg Error % 17.39 14.13 30.0 21.43
As expected, we are looking at two drastically different signaling paradigms. Still, there is much that can be learned by comparing the real-world breakout signaler with the order-based tradescape signals. We can say from the start that the order in QQQ is important, and the breakout signals suffer from their inability to exploit that fully. The lags with the breakout signals are great, but the coverage errors for the signal turns range from 15-30%. Given that it only takes one error on an entry-exit pair to break a trade, it is possible that as many as half the trades fail to track the order represented by the tradescape for these four signals. The win rates are weak on the breakouts, and the RRt reward-pain is much weaker, though still better than the very lagged MA crossover example. Interestingly, the breakouts were on the right side of the ideal trades considerably more than the lagged ideal signals, but that did not result in greater returns, wins, or reward-to-pain. In fairness, it is rare than a trading system so effectively manages both the order and the chaos in the price movements that the real-world signal results in better performance than the tradescape at the same time horizon and lag, and that is especially true at lower lags where accuracy is so much harder to come by.
There is very good news here. The tradescape can only address the ordered aspect of the signaling, but doing so suggests that all four of these systems are in a very stable place. The historical reward-to-pain has been increased by a factor of 3, where we saw only a factor of two with the MA crossover system.
Comparing Different Entities
Part of the edge one realizes by using tradescapes is something that applies to general investment science. Which securities make sense for a portfolio? Which securities are forgiving in terms of this tolerance for delay in making productive entry and exit decisions? For the signal designer who has become familiar with what he or she can expect from their trusted signaling systems, it can take just a few seconds to know what one can trade effectively and what is unlikely to do well, and that can be inferred with no design studies.
For example, the following is a set of tradescapes for the US overseas market ETFs:
If one knows, for example, that their favorite signaling methodology typically generates about a lag fraction of 1.0 with time horizons varying from an EM length of 25-40, then these ten year histories don't bode well for trading SPY, EWT, or EWJ in such a way as to realize more reward than pain. That may be due to lack of growth in the entity, the lack of ordered movement, or more likely some measure of both. In this example, there is good promise for EWZ, EWM, EWS, EWH, EWC, and EWW. EWA, EWG, and EWY are shown as tradable in terms of order, but they require a faster signaling, an EM length of 15-20, in order to realize this greater reward than pain. For these past ten years, EWZ, EWM, EWS, EWC, and EWW are particularly forgiving of lagged entries and exits.
Why does a system work so very well for one entity and so very poorly for another? Some of that answer lay in the nature of the order in each of the entities as shown in the plots above.
The Tradescape Difference
Without an expectation model (EM) reference, the best we can typically do in a trading system design is an optimization surface with two adjustable parameters. We can plot those two variables relative to a reward-to-pain metric. We know what we can expect in terms of reward-to-pain for those two variables, but we won't have any measure of consistent information content or time horizon, nor will we have any estimate of lag or accuracy at any of the adjustable parameter pairs.
What if we used just a portion of the EM technology to transform the plotted variables from the two adjustable parameters to an EM-based estimate of time horizon and lag? Is a full tradescape needed?
The plot on the left is the transformed RRt response surface for the breakout optimization. We had 3136 pairs in the test, the nHH and NLL each ranging from 5 to 60 in unit increments. For each of those points, the EM length and Lag Fraction were computed and plotted on a delta RRt surface. On the right, we see the traditional QQQ tradescape with the same scaling, the points that are generated for every tradescape clearly shown. Even though we tested essentially the entire practical range of breakouts, we covered only a small portion of the trading landscape that is plotted in a tradescape. Further, for each of the points in the plot on the left, the accuracy varies as no doubt does the measure of order and the measure of chaos addressed by the signaling at each point. The scatter in the lag represented at each point may also vary considerably across the backtests. The nature of the trading algorithm also results in major holes in the landscape that is actually addressed. The tradescape, on the other hand, covers the entire trading landscape, has the full accuracy for the trading of the ordered movements at every point, and every meaningful time horizon and lag is represented.
The Promise of Understanding Why
For a trader or investor, the very idea that one might be able to understand why something works, or conversely does not work, is generally unexpected. Movements in prices have such a human side, the greed, the panic, the manipulation, the reaction, the uncertainty, that any grasp as to "why" is often seen as beyond reach. By using EM science to generate tradescapes, however, we can know in a historical sense where successful trading of the order exists and what one can realistically hope to achieve by trading that order.
Let's look at a second breakout optimization for QQQ:
In this case, we generated a comprehensive matrix of 6500 HH and LL breakout values and for each parameter pair we again compute the delta RRt, the improvement in reward-to-pain from the trading system. In this instance, we added the slower breakouts for both entries and exits. To better visualize the nuances in the surface, we now use a 250x250 shaded surface plot and a C1 interpolant. The microstructure in the response surface is apparent. Also clear is why so many dislike optimizations. Choose parameters on any one of those sharper peaks and the safe bet would be substantially less performance walking forward. Let's say one's goal in a QQQ trading system signaler is to realize 2x the underlying's 0.61 RRt. We thus want a dRRt walking forward of +0.61.
In our initial optimization, we looked at the traditional breakout ranges used by experienced traders. If we now look at this full range optimization, we see the same entry optima for the nHH, but we now see a very strong region with very long exit breakouts. The quick to enter but slow to exit optimum holds to even another level.
Now, in order to achieve the desired result, what parameters do we choose? Can a tradescape help in this situation where the entries and exits are so disparate?
If we use EM technology to convert each nHH,nLL pair to a time horizon and lag at the signaled turns, we can generate a tradescape-like picture from the actual breakout algorithm. For this plot, the gray points produce an inferior reward-to-pain. The blue points produce anywhere from 0-0.5 improvement. The yellow points produce a 0.5-1.0 improvement. The red points produce a 1.0-1.5 improvement. With most real-world algorithm optimizations, we cannot plot a surface. Two pairs with close to the same estimated time horizon and estimated signaling lag can produce very different reward-to-pain as a consequence of accuracy differences.
The EM lengths represent a single estimate of a time horizon. This will be an average value for a signaling algorithm that favors fast entries but slow exits. Still, we see something striking. Most of the red and yellow points in the real-world signaling algorithm track the tradescape pattern. Most are at the lowest lag fraction realized for each time horizon.
Why should we select a parameter pair along this boundary? The simple answer is that this is where the ordered trading is found. Clearly there are certain yellow and red points at high lag fractions. How can they look as good as they do? Recall that the real-world signals offer only about a +1.5 delta RRt maximum. With a 0.61 for the underlying, the real world algorithms achieve up to an RRt of about 2.0. The tradescape goes far higher, but it represents full accuracy at every point. Where there should be a turn, there is a turn, however lagged. At high lag fractions, the signaling algorithm has a lot of leeway to get it right. The accuracy is often high. For breakouts, where the order is largely traded by the memory effect represented by the lengths, high lag fractions don't necessarily map to good performance.
Because the EM length maps to the fractal time horizon or information content utilization of the algorithm, and the lag fraction measures the lag at the signal turns relative to this time horizon, we can assess the two factors most important in signaling. In a tradescape, the accuracy is set as close to optimal as can be realized with EM technology. The real-world algorithms will rarely exceed such. When they do, the additional information can sometimes help us understand why.
If we return briefly to the [75,60] SMA crossover that we initially dismissed, we find that it has an exceptionally high lag fraction (>2). For a given count of signal transitions (trades), it has a very high amount of lag relative to the EM reference with this same trade count. And yet we saw that it offered the highest delta RRt in the SMA crossover optimization. The EM technology tells us the effective time horizon, the lag, and the reward-to-pain we would see with full accuracy when we lag the optimum signal accordingly. How do we account the higher performance? Are we successfully trading the chaos?
We know that would be unlikely. In an SMA crossover we respond only sluggishly to price movements. Again, we come back to the difference between a 75 point and a 60 point simple average is just 15 bars a full 60 bars in the past. The lag should be bad and is. Why then does it produce any benefit at all in trading? Why does it so exceed what the tradescape with its ideal trading of order offers at this same high lag? Here we can use our understanding of MA crossovers. Those with close to the same lengths don't signal on price so much as to estimate a principal cyclic oscillation. We could regard the MA crossover in this instance as a harmonic estimation algorithm whose lengths set an effective period and whose positions at each point in time lock in the phase. Given that such cycles are well known, we have to assume we are somehow extracting the principle cycle information at each point using this basic procedure. Is this order or chaos? The lag in the actual algorithm is real and it is very high. If this means of indirectly estimating the principle cyclic component in the data were to suddenly cease working, it might be some time before such was detected.
We thus understand that the accurate signaling of ordered price movements is not a singular process. There are many ways to achieve that accuracy. A tradescape does so by directly reacting to price. That is what we did with the breakout signals and with the SMA crossover with disparate lengths where the premise is a smooth central channel and fast signaler crossing it. In the course of building trading systems with tradescape and EM technologies, you will find systems that look out of place relative to tradescapes. When that occurs, you now have an edge. You know you are looking at something outside this orderly signaling on price. That opens the way to explore what one is actually doing and with an understanding as to why any given system works, we can know how vulnerable we are to having that signaler cease working.