TSCI.pngSignal Encoding


The Trading Sciences platform uses ASCII data files that contain encoded trading signals. These are generated transparently within the specific platform, such as TradeStation, and require no additional processing when using the platform in standalone mode. If you wish to manually add trading signals to an existing data file, create one independently, or of you wish to extract the trading data from an ASCII file containing encoded signals, the following information may be of use.

Binary Signal States

The Trading Sciences trading signal evaluation routines analyze "binary" signals. For our purpose, binary means for each day there is a market state for the entity being traded:

    +1      for in market long
    -1      for in market short
     0      for out-of-market

Since a tradescape and signal analysis is either long or short, but not both, only one side of the signal is used:
    For a long tradescape, the +1 signal is in a position (long), otherwise out of market
    For a short tradescape, the -1 signal is in a position (short), otherwise out of market

ASCII Data Format

The Trading Sciences ASCII data files use the Yahoo data format that you will find from many of the different financial sites that offer the means to save historical information to ASCII files. These are CSV (comma separated value values). The Trading Sciences format adds two additional columns, Time and Flags:

Symbol,Date,Open,High,Low,Volume,Close,Adj-Close,Time,Flags

SPY,7/6/1998,114.78,115.88,114.56,3.0906e+006,115.88,115.88,1300,0

SPY,7/7/1998,115.98,116.13,115.28,4.8595e+006,115.66,115.66,1300,0

SPY,7/8/1998,115.88,116.94,115.75,6.7833e+006,116.75,116.75,1300,0

SPY,7/9/1998,116.28,116.72,115.63,6.9133e+006,115.81,115.81,1300,0

SPY,7/10/1998,116.03,116.91,115.06,7.7921e+006,116.38,116.38,1300,65793

SPY,7/13/1998,116.56,116.84,116.06,6.5551e+006,116.5,116.5,1300,65793

SPY,7/14/1998,116.94,118.16,116.94,6.9314e+006,117.81,117.81,1300,65793

SPY,7/15/1998,118.06,118.28,117.53,4.9264e+006,117.61,117.61,1300,65793

SPY,7/16/1998,117.69,118.59,117.06,6.2792e+006,118.53,118.53,1300,65793

SPY,7/17/1998,118.63,119,118.31,3.5096e+006,118.69,118.69,1300,328965

...

 

The Flags Field

The Flags field will contain up to 15 encoded signals stored in a single integer value. The first signal is stored in bits 1 and bits 2. If the signal is out of market, neither bit is set. If the signal is long (+1), the first bit is set. This bit contributes 1 to the value of the Flags integer. If the signal is short (-1), the second bit is set. This bit contributes 2 to the overall integer. It there is only one signal encoded, the value of Flags will be 0, 1, or 2 depending on whether the state is out of market, long, or short. If you want to manually add a single signal, you can follow this convention.

When there is just a single signal being manually added, you can optionally take advantage of the fact we process the sign bit as an alternative to the 0, 1, 2 encoding. As such, you can also directly set Flags to 0, 1, -1 for out of market, long, and short.

As with bitwise encoding in general, things get more complicated when there are multiple signals. The second signal is stored in bits 3 and 4 using the same convention as bits 1 and 2. These two bits contribute 4 or 8 to the overall integer value of Flags. The third signal is stored in bits 5 and 6 and these will contribute 16 or 32 to the value in Flags. And so on. If you wish to extract the signal

Signal Extraction - An Excel Example

(1) Import the CSV ASCII file written by one of the signal evaluation procedures in the Tradescape application into Excel. The Flags field with the encoded signals be in column I.

(2) In row 1 for columns K through Y, enter the numeric values 1 through 15. These columns will contain the 15 extracted signals.

(3) Enter =MOD(INT($J2/2^(2*K$1-2)),2)-MOD(INT($J2/2^(2*K$1-1)),2) in cell K2. Copy K2 from L2 to Y2. Copy the block K2-Y2 from K3 to the last row of data.

All 15 signals are extracted in the original -1,0,1 format in which they were encoded. There will be all zeros if no signal was encoded in that position.

excel01.png

Signal Reconstruction - An Excel Example

If we assume that columns K through Y are repopulated with up to 15 externally written signal values, the Flags field can be reconstructed with the following Excel formula:

=IF(K2<>0,IF(K2=1,2^(2*K$1-2),2^(2*K$1-1)),0)+IF(L2<>0,IF(L2=1,2^(2*L$1-2),2^(2*L$1-1)),0)+IF(M2<>0,IF(M2=1,2^(2*M$1-2),2^(2*M$1-1)),0)+
IF(N2<>0,IF(N2=1,2^(2*N$1-2),2^(2*N$1-1)),0)+IF(O2<>0,IF(O2=1,2^(2*O$1-2),2^(2*O$1-1)),0)+IF(P2<>0,IF(P2=1,2^(2*P$1-2),2^(2*P$1-1)),0)+
IF(Q2<>0,IF(Q2=1,2^(2*Q$1-2),2^(2*Q$1-1)),0)+IF(R2<>0,IF(R2=1,2^(2*R$1-2),2^(2*R$1-1)),0)+IF(S2<>0,IF(S2=1,2^(2*S$1-2),2^(2*S$1-1)),0)+
IF(T2<>0,IF(T2=1,2^(2*T$1-2),2^(2*T$1-1)),0)+IF(U2<>0,IF(U2=1,2^(2*U$1-2),2^(2*U$1-1)),0)+IF(V2<>0,IF(V2=1,2^(2*V$1-2),2^(2*V$1-1)),0)+
IF(W2<>0,IF(W2=1,2^(2*W$1-2),2^(2*W$1-1)),0)+IF(X2<>0,IF(X2=1,2^(2*X$1-2),2^(2*X$1-1)),0)+IF(Y2<>0,IF(Y2=1,2^(2*Y$1-2),2^(2*Y$1-1)),0)

If you place this formula in cell Z2 and copy it from Z3 to the end of the data, you will fully reconstruct the Flags encoding in column J.

While you may copy the pertinent information to a separate worksheet and save that sheet as a new CSV, you can also save the Excel file with all of the signal and reconstruction information present and it will be properly imported. You must remember, however, to paste (as values) the reconstructed values in column Z into the original Flags cells in column J. The name "Flags" must be preserved in cell J1. All of the information beyond column J is ignored in the import procedure.

Directly Importing Signals from an Excel Sheet

Please note that the signal encoding is a programming convenience used by the tradescape platform's ASCII files. All ASCII files that do not conform to this encoded format are imported in a general ASCII import procedure. Since that import allows the specification of individual signal columns, the encoding is not used. The reserved ASCII format with this encoding is identified by the strings shown in columns A-J of Row 1 in the extract above. As long as the strings in an ASCII file do not exactly match, the generic ASCII import option will be invoked and individual signal columns can be directly imported as they appear in columns K-Y in the example above.