Signal Analysis (White Paper)
Signal Analysis in Trading Systems
To realize a grounded and practical approach for evaluating the effectiveness of the binary signals often used in trading systems, we seek to answer the crucial questions. Which entities have some prospect of working with a given signaling system? Which signaling systems have a good prospect of trading a specific financial instrument? How good does one have to be in the signaling for a given security in order to historically realize more reward than pain? For a given lag and information time horizon, what might one expect as a realistic upper limit in terms of performance, at least with respect to trading the order in the price movements? What is the lag and accuracy one actually realizes in one's real world signaling? Is there a way to refine that signaling in order to improve upon the risk adjusted return?
Most trading signals can be reduced to a time series of +1, 0, and -1 values. Unless there is some form of position sizing, leveraging, or portfolio allocation adjustment arising from a continuous signal, these three states can describe most trading scenarios.
We will consider any trading system that signals on 1, 0, and -1 states a binary signaling system, even though it may not be that in the strictest sense.
In the simplest system, a trading signal will be +1 for a long position, and 0 for an out-of-market state. The trader opens a long position with a buy order when the signal first changes to a +1, and that position is closed as soon as the signal ceases to be 1.
A short side signal would be similar, consisting of -1 and 0 values. One would enter a short position when the signal first changes to a -1 and close out that position as soon as the signal ceases to be -1.
If one is to draw both long and short positions from the same signal, it is convenient to specify 0 for out of market, +1 for a long position, and -1 for a short position. If one wishes a long only evaluation, the -1 is treated the same as 0. If one wishes a short side test, the +1 is treated as a 0.
A Unique Approach for Trading Signal Analysis
Our paradigm for analyzing the efficacy of a binary trading signal uses risk-adjusted returns. We prefer the RRt because of its stability, simplicity, continuity in optimizations, and because its components can be regarded as fundamental properties of any time series. We recognize an improvement from a trading signal only if there is an improvement in the reward to pain ratio used in the analysis.
We also use an EM (expectation model) reference as the gold standard by which any given trading system is measured. This is done against the backdrop of what we call a tradescape, a contour or 3D surface visualization that consists of a large number of backtests that use a set of these ideal EM reference signals at various levels of lag.
The tradescape is the canvas upon which the trading signal analysis results are presented. When a binary signal is analyzed, an EM reference is constructed that matches the zero crossing count in the one-bar differenced signal. It uses the same signaling target as the trading signal, which may be a different entity if a referential surrogate is used to generate the signal.
If the trading signal has 200 transitions in 2500 days of trading AAPL, for example, we adjust the information content parameter (the EM length) in the EM algorithm so that our ideal reference signal contains this same number of transitions. If there are 100 trades that occur from the trading signal (100 entries + 100 exits), then we build an EM reference with as close to this same count of trades as possible.
At this point we have apples and apples and we can compare the trading system to this ideal reference. The EM signal is a benchmark that we assume is 100% accurate in terms of detecting the turns that should exist when trading order for a signal with this count of transitions. While we do not say that these EM entry and exit positions are the only possible ones that generate strong profits, we can say they are very close to the optimum, and very nearly the ones you would humanly select if for each window in the data, you hid (and fully forgot) all other data and smoothed just those data points using the human eye, and you ended up with the same count of sign changes in the slope of your smoothed curve. Intuitively, you would be slow to allow a transition to happen if the resultant slope was very near zero, but very quick to put that transition in if the slope would be significant in the opposite direction.
Once we have the trading signal and its EM reference counterpart, we can perform a signal analysis to determine the lag that is present at each transition. The lag at the actual turns is that which makes or breaks the trading. It is what really matters. When we search for the expected transition within some tolerance in order to log each lag, we also measure how often the signaling algorithm succeeds in mapping the transitions in the reference. This is what we call signal accuracy.
With this information in hand, we can then plot the performance result from each of the trading signals on the tradescape. We do so at the measured lag and at the information content length drawn from the matching EM reference. The difference between the tradescape's reward to pain and that which is computed from the trading signal will be the key to understanding the performance of the signaling. In general, the loss of accuracy relative to the EM signal results in a reward to pain that rests below, often substantially below, the tradescape's surface.
A Moving Average Example
To see how all of this works, let's look at directly trading AAPL using the slope of moving averages. If the slope is positive, we assign a +1 to the trading signal, and if negative, we assign a 0. We will look only at long trading positions. We will analyze the following signals:
Signal = 2 passes, SMA, length 5
Signal = 2 passes, SMA, length 10
Signal = 2 passes, SMA, length 20
Signal = 2 passes, SMA, length 40
Signal = 2 passes, WMA, length 5
Signal = 2 passes, WMA, length 10
Signal = 2 passes, WMA, length 20
Signal = 2 passes, WMA, length 40
Signal = 2 passes, EMA, length 5
Signal = 2 passes, EMA, length 10
Signal = 2 passes, EMA, length 20
Signal = 2 passes, EMA, length 40
For this example, we use 2 passes on the MAs to prevent the need for any kind of signal conditioning or thresholding to deal with whipsaws in the signal. We understand in advance that this approach for increasing the smoothness will double the lag in the signal. We use this signal for this example since it is especially accurate, especially with the SMA, but in most instances, the lag will be too high for successful trading.
The analysis consists of the 10-year long EOD tradescape for directly signaling on the AAPL closing prices. All 12 systems are analyzed. The plot is scaled to show all regions where the reward is greater than the pain. For AAPL, that is everywhere in the range shown in the tradescape graph.
The trading system points are colored using the RRt color gradient. If you look closely, you will see that, in general, these signals produce substantially less reward-to-pain than their EM references. There is one exception that we will explore further in a moment.
If we use a 3D view angle that allows us to look beneath the tradescape surface, the drop lines to the trading signal points illustrate the extent to which the signals failed to produce the same return as the EM reference, the value of the RRt tradescape surface at this same value of lag and information content.
If we use a different view, we see trading system  (2 pass WMA length 5) that slightly exceeds the tradescape surface (the drop line is blue and the trading system point is above the tradescape surface. The most profitable tradescape zone is nearby and as such, we assume that this is unlikely to result from random chance.
The SMA Signals
Just as the signal point colors reveal in the contour plot, the SMAs may have high lag, but they are quite accurate, approaching the value of the EM references. For the three faster SMAs that appear in the plot, the accuracy is anywhere from 96.5-100%. The measured lag for each is exactly the theoretical 2 passes * (n/2-1). For signal 1, the 2 passes of an SMA of length 5, the measured lag is 4.0. The lag fraction where the point is plotted, however, is based on the EM length of 3.71. This is the length used to create the EM reference that matches the 126 trades in the trading signal. The lag fraction is 1.08, a typical value for a fast 2-pass SMA.
Note that signal 2 had a weak reward to pain, but this makes sense when contrasted with the tradescape at this same lag and trade density. This is not a favorable zone in the AAPL tradescape to be signaling. We expect to do poorly not because of any failing of the signaler, but rather because this is an unwise place to be for generating AAPL signals. Despite its high lag, system 3, the 2pass 20 length MA does very well. It actually falls into a rather forgiving tradescape region. For AAPL, which is itself quite forgiving because of its strong long term trend, we see that a simple SMA can effectively trade AAPL for more reward than pain.
The WMA Signals
These are signals 5-8, all of which plot in a very favorable place for lag. The measured accuracy will generally be less than the SMA. It is not near 100, as is true of the SMAs, but between 87-96%. Keep in mind that a 5% error at the turns means that 1 in every 20 transitions will fail to match the reference. With one transition needed for an entry and one for an exit, this means that as many as 10% of the trades will not synch-up with the reference. That is significant, but not nearly the impact of the WMAs with a transition accuracy closer to 90%. That 10% error at the turns could well equate to 20% of the trades being questionable in terms of entries or exits.
The EMA Signals
These are signals 9-12. EMAs are single coefficient IIR filters, and in general they can be expected to produce very different behaviors than the multiple coefficient FIR-type filters that include the linearly weighted WMA.
An IIR filter processes only the most recent point at each bar, and is therefore quite susceptible to the measure of disorderly behavior in a time series. A large impulse takes longer to wash out of the moving average than a smaller one. For AAPL, the EMAs exhibit the familiar inaccuracy. The accuracy at signaling the turns is weak, 72-86% across the four signals. Because of how an impulse is processed in an EMA, it is adaptive. Greater disturbances are given greater influence. Any kind of adaptive procedure generally means that the density of the lag observed at the turns will be more widely distributed. Some of the transitions will be faster than the WMA, others will be slower. That can dramatically impact the reward-pain. In every way, IIR filters and FIR filters are apples and oranges.
Here we see that the EMA signals 9 and 11 do reasonably well.
The Power of Signal Evaluation Using Tradescapes
In this example, we see a 10-year tradescape analysis with 700 backtests. Atop that, we see twelve trading signals characterized for lag and trade density. The real world signals are backtested in the same way as the EM reference signals in the tradescape. For each trading signal, we see the real-world reward-pain as well the the signal quality as compared to an ideal expectation. An analysis such as this, realized from a TradeStation analysis procedure, executes in just a few seconds.
The tradescape for an entity tells us exactly what we can expect from an ideal signaling signaler, and it does for every lag and trade density we might wish to explore. Further, the backtest results from the the different signals are placed on the tradescape at the lag and EM lengths that allow the ideal and the real world to be directly compared. We further use signal analysis methods to report the average lag and accuracy at the turns for the trading signal, at those transition points that make a difference in terms of the quality of entries and exits.
To sum up:
(1) From the tradescape, we know how favorable the entity is to price-based signaling in general-the issues of long term trend, the risk or pain, and the measure of order in the time series are all addressed as one would address them in a professional trading system design.
(2) From the tradescape, we know how good we have to be in terms of lag in order to signal, and we know the average trade length where we will be most successful-we know what we are up against before we even get started with any system design.
(3) From the signals we sent along with the entity data for the tradescape, we see the real world with respect to what is attainable in the ideal-we see the reward-pain that we actually realize.
(4) From the signal analysis that links the real world systems with the tradescape, the theoretical ideal of a full accuracy signaler, we see what we have achieved in terms of lag and information content, and we can see how close the signal is to the most favorable signaling zones in the tradescape.
(5) From the independent signal analysis, we use the gold standard EM reference to estimate the accuracy in the signal-the signaling can be tweaked for maximum accuracy on the actual instrument to be traded.