TSCI.pngBacktest Engine


8617.png The Customize Backtest Engine option in a tradescape plot offers the means to alter how a tradescape is generated in the internal backtest engine. These alter the global settings for the product and these will remain in place across instances and sessions until changed.

(Professional Edition)

The Tradescape Paradigm

The computations for a tradescape could be done in a pure mathematical algorithm. In effect, each of the EM signals would produce a set of active segments in the time series and the redefined price movement just from those segments would determine the signaled price curve. For greater flexibility, and to subject the EM signals to the same process as real-world signals, the program uses a custom backtest engine for all computations.

The default settings are designed to use the backtest engine in this pure mathematical approach. There is no trading cost. There is no slippage or spread. One assumes a closing price and the ability to execute at that same price. There are no stops, and the EM signal is based completely on the closing price.

We feel that this design is the closest one can come to a universal trade landscape analysis. Given that the EM is itself an idealized signal, designed to be fully accurate in terms of catching all of the ordered movement in the time series, we feel it makes little sense to introduce real-world non-idealities into the basic tradescape analysis.

The main exception comes mainly when you want to do a 'what-if' analysis using the ideal EM signals. In such an instance, you may wish to adjust the backtest to explore these real-world trading factors.

Adjust Cautiously

While we make this customization easy, we also want to discourage this for general tradescape and signal analysis. The product will be most useful and meaningful if the plots always represent this ideal trading landscape for the order that is present in the time series. The program is designed to compare apples and apples across every key variable such as the financial instrument, the sampling rate of the data, the asymmetry in the signaler, different analysis periods, and what occurs in the special instances of referential and two-stage signalers. To keep everything in a single general framework where comparisons are fully valid, the defaults are strongly recommended for all analyses except those where a specific study is desired for how a real-world trading factor impacts EM signals.

Lag and Accuracy in Signal Analysis

Please note that the trading signal analysis that generates estimates for the lag and for the accuracy at the turns is not impacted by these settings. Since this type of analysis requires an ideal reference or gold standard, we generate an ideal EM signal with the same count of turns as the trading signal. We thus attempt to match the zero crossings of the differenced signals as closely as possible. With this reference in place, we can then search the turns in the signals to see where there is a match, and how much lag is present when there is a match. This type of analysis would be little served by any distortion in the reference signal.

Backtest Parameters that Do Not Alter the EM Signals

Certain backtest customizations do not impact the EM signals used in the tradescapes. They only impact the return that is realized from processing those signals. Examples would include the equity model, the trading costs, and the trade execution timing. The same EM signal is generated regardless of these settings. If one nets out less return because of trading costs or the pain being greater as a consequence of how trades are executed, the only difference rests in the output variable or z-gradient of the plot. The reward-to-pain will certainly diminish to the extent these real-world non-idealities impact the tradescapes.

Backtest Parameters that Change the EM Signals

There are customizations that can have a profound impact on the EM signal. We include these as a matter of convenience for a narrow purpose. These should definitely not be used to routinely generate tradescapes.

Stops are an example of a customization that dramatically alters when the optimum signal generates the turns. If one has stopped out with a certain cooling period, there is an exit attributable just to the stop, and nothing happens for the count of bars one waits out a period of better market equilibrium. Such a system is a blended signaler, one that trades order but also includes a chaotic component, this defense against unexpected events. The utility here is primarily to see if the inclusion of stops can produce any marked improvement in the reward-pain that follows from the optimal signal without such stops.

In general, you should construct your real-world trading signals with all possible variations and pass each of these signals into the tradescape-based signal analyses. They should thus be implemented on your real-world signals, not on the idealized ones. You should never base the use of any stop on their value with idealized EM signals. If there is a benefit from a stop with the EM signals, there may well be a greater benefit with a well-crafted real-world signaling system. If there is no benefit or a diminished performance, one has likely learned nothing. The stop might still be of value with one's real-world signaling system even if there is no benefit from its addition to the EM signals.

The signal variable is the other obvious example of an adjustment that will impact the EM signal.

You should add stops or modify the signal variable only for a "what-if" analysis and remove them afterward.

Persistence of Backtest Settings

Note that all settings are persistent across analyses and sessions. It is also recommended that you adjust these backtest settings when only a single tradescape applet is active.

backtest01.png

Starting Equity

The default is 100,000 in equity. This is sufficient to insure that most of the equity is invested. The backtest engine does not allow fractional shares, so this number needs to be merely large enough to insure that nearly all of the amount can flow into shares. If the entity is something like the DJ index, with a value greater than 10,000, you may wish to set this amount to 1,000,000 to diminish the granularity of the trades and the proportion of the equity that is not invested due to a fractional share amount.

This amount is the starting equity unless the equity model specifies a fixed count of shares.

Equity Model

Far and away this will be the most important factor impacting the reward-pain estimates.

Amt is Initial Equity-All Equity Invested in Each Trade - This is the default. At every point in the analysis or walkforward period of the tradescape, all available equity is invested down to a unit share. If trades produce wins, a greater amount is available in successive trades. If losses are incurred, there is less equity to trade. Since this is assumed to be the default, we can estimate robust trends or CAGRs looking for the extent to which compound growth is realized. The trend values and the reward-pain estimates are designed for this specific equity model. The trends and reward-pain estimates will be underestimated, especially when there is exponential growth in price across time, if the other equity models are used.

Amt is Fixed Equity Invested in Each Trade - Here the starting equity amount is used in every trade. The share count in each trade can vary widely across time, but the total equity invested will be constant. This makes the obvious assumption that initial losses will be met by some form of borrowing; that is, there is an infinite pot of money from which to draw this fixed amount to be used in each trade. Because there is no reinvestment of gains, the robust trend computation will underestimate the actual growth. There is no skew in the equity to earlier or later time periods since the same fixed equity is invested in every trade.

Amt is Fixed Shares Invested in Each Trade - In this case, a constant share count is specified. This is definitely not what you want except for very short term trading analyses. In this case, the Starting Equity is the share count, not an equity amount. For all trades, the same count of shares is invested. In a entity that grows in price, the skew will be to later periods as is true of the default option. Because the skew is different from what would be seen with reinvestment (the entity's price and one's equity don't synch identically), the robust trend estimates can be either understated or overstated.

You can use this option as a what-if with respect to an ideal signal. The equity curves that you see when you double click any of the tradescape points will reflect these equity models. Simply bear in mind that certain numeric metrics may not be designed for a given model. Look at the equity curve. The equity plots can report an annualized simple return and an average retracement, which may be the reward and pain you actually want for your study, not the R or RRt metrics which measure the robust compound growth.

Trading Costs

Cost Per Trade - If you an analyzing a period where the data reflect no splits, and your pricing model for trades is a fixed cost per trade, you can add this absolute deletion of equity from every entry and exit. Be cautious of using this across many years. Check the price across the range of the analysis. If the price is $1 at the start of the analysis and $100 at the end, and you know splits took place in those years, 100 shares at $1 each and a $10 trading cost would have the trading costs at 10% on each side of the trade. Whatever took place in those times, it is unlikely one saw anywhere close to that. On the other hand, those same 100 shares at the $100 price with that same $10 trading cost would have the trading costs at just 0.1% on each side. If a lengthy period of time is being analyzed, and you want to incorporate a trading cost into the EM engine, you should do so by lumping together everything (trading costs, slippage, spread) into a single percentage to be subtracted on each trade.

Spread/Slippage (%) - This is the general way to account for the inefficiency of trading. Specify a % that will be lost due to spread and execution slippage, and if you choose, commissions, and this will offer a real world sense for the execution costs.

Note that these are simply the fixed and percentage deductions for the inefficiency in systems.

Trade Timing

This is also another way to account inefficiency in execution.

Trade Close - Value and Execution Taken Just Before Close - This is the default. It assumes perfect efficiency. You execute a trade at the closing price. In actual practice, you take a price just prior to the close and execute just prior to the close, and assume that price will signal and execute at the closing price. This is the timing that makes sense for a tradescape as a mathematical object.

Trade Close at Open of Next Bar - Value at Close, Execution Next Bar's Open - This is an option where the execution will not occur at the close, bur rather at the open price of the next bar. In our experience, since a tradescape reflects trading with the trend, a signal at the close will reflect momentum warranting that entry or exit. Waiting longer for the execution tends to result in losing some of the benefit of those entries or exits, especially if there is an overnight gap between.

Trade Close at HL of Next Bar - Value at Close, Execution Midpoint of Next Bar - This is a worst case scenario. One executes at the midpoint price of the next bar. The entry or exit is even further delayed, and if the price action continues, it will work against the trade. In our experience, this will generally produce the lowest reward-pain values,

Trade Stops

Unlike the previous customizations, this option alters the actual EM signal. In effect one generates a hybrid EM-stop signal. The purpose is to do a what-if in terms of seeing what happens with a given type of stop on this ideal signal. This may be of limited value since a stop is also a defense for the inaccuracies in a signaler. If a signaler has high accuracy, a stop possibly may be less effective.

The Risk % for Stop specifies the percentage delta to trigger the the stop. The default is 0.0 (there are no stops).

The Cooling Period (bars) specifies the cooling-off period following a stop out before trading can assume. The default is 4 bars.

The Stop Type dropdown offers six different types of stops:

Bar % Intrabar-An adverse % from Bar Open Signals Intrabar Stop - this tests for chaotic activity within a bar. If greater than a given % change takes place in an adverse direction, the trade exits at the intrabar price.

Static % Intrabar-An adverse % from Entry Price Signals Intrabar Stop - this tests for adverse movement from the point of entering a position. If greater than a given % change takes place in an adverse direction, the trade exits at the intrabar price.

Trailing % Intrabar-An adverse % from HH or LL Since Entry Signals Intrabar Stop - this tests for adverse movement from the best point achieved after entering a position. If greater than a given % change takes place in an adverse direction from this high or low water mark, the trade exits at the intrabar price.

Bar %-An adverse % from Bar Open to Bar Close Signals Exit at Close of Bar - this tests for chaotic activity within a bar. If greater than a given % change takes place in an adverse direction within the bar, the trade exits at the bar's closing price.

Static %-An adverse % from Entry Price to Bar Close Signals Exit at Close of Bar - this tests for adverse movement from the point of entering a position. If greater than a given % change takes place in an adverse direction, the trade exits at the bar's closing price.

Trailing %-An adverse % from HH or LL Since Entry to Bar Close Signals Exit at Close of Bar - this tests for adverse movement from the best point achieved after entering a position. If greater than a given % change takes place in an adverse direction from this high or low water mark, the trade exits at the bar's closing price.

Trade Signaling

This option also alters the actual EM signal since the signal is based on something other than the price one would actually trade. Traders sometimes signal on the HL or HLC price to seek to smooth out some of the whipsaws arising from the greater variability in the C or closing prices. In general, you will see the EM signal's response degrade with these alternate signal sources since the EM algorithm has been designed to be free of whipsaws. Trading the HL or HLC price may be far more favorable in situations where the real-world signaler is vulnerable to these whipsaws or undesirable turns. This what-if basically allows one to see how much one gives up in potential reward-pain by using a less direct signal source. That can be more than made up by the absence of whipsaws in the real-world signal.

Signal is Based on Close of Bar - The generated EM signal is based on the closing price time series. This is the default.

Signal is Based on HL of Bar - The generated EM signal is based on the (high + low)/2 price time series.

Signal is Based on HLC of Bar - The generated EM signal is based on the (high + low + close)/3 price time series.

Portfolio Option

Whenever incoming daily (EOD) data contains multiple entities, a equal weight portfolio of all entities (independent reference and sentiment entities excluded) can be automatically generated and made available to the backtest engine. This will be made available in all procedures that support EOD data. You may specifically select the portfolio, or for those options that generate a panel of different entities, the portfolio will be included as the last panel.

A portfolio is constructed using equal weighting by price, and is done using the most recent price in each of the data streams. Note that the portfolio is balanced for equal investment from each entity only at this most recent point in time. If a portfolio option is desired, it can be added by checking the Add Automatic Portfolio Generation when Multiple EOD Data Streams Present option.

Restoring the Backtest System Defaults

The Reset button restores all of the backtest system defaults. You should use this if you make changes and for a given analysis you want to be certain the defaults are fully back in place.

Exporting Custom Backtest Settings

Use the Save button to write a custom backtest test configuration to disk. These are internal files with BFS extensions.

Importing Custom Backtest Settings

Use the Read button to import a custom backtest test configuration saved to disk.

Persistence

All settings are persistent across analyses and sessions. The current settings are saved on program exit and are used for all subsequent analyses. This is true for all versions, including the standalone.

If you are using TradeStation, you should adjust these backtest settings when only a single tradescape applet is active. In an environment such as TradeStation, you can have dozens of tradescapes open at the same time. Each of these is an independent process and each will save its settings to a single set of global defaults. In such an instance, the last of the tradescape processes closed will determine what the defaults look like for the next launch. If you are uncertain, check the customizations and use the Reset option. This insures the standard settings are intact.