What Level of Performance Can Computers Produce in a Neural Network Trading System?
An Introduction to BioComp Profit

Jeffrey Mishlove, PhD, CTA

HYPOTHETICAL PERFORMANCE RESULTS HAVE MANY INHERENT LIMITATIONS, SOME OF WHICH ARE DESCRIBED BELOW. NO REPRESENTATION IS BEING MADE THAT ANY ACCOUNT WILL OR IS LIKELY TO ACHIEVE PROFITS OR LOSSES SIMILAR TO THOSE SHOWN. IN FACT, THERE ARE FREQUENTLY SHARP DIFFERENCES BETWEEN HYPOTHETICAL PERFORMANCE RESULTS AND THE ACTUAL RESULTS SUBSEQUENTLY ACHIEVED BY ANY PARTICULAR TRADING PROGRAM.

ONE OF THE LIMITATIONS OF HYPOTHETICAL PERFORMANCE RESULTS IS THAT THEY ARE GENERALLY PREPARED WITH THE BENEFIT OF HINDSIGHT. IN ADDITION, HYPOTHETICAL TRADING DOES NOT INVOLVE FINANCIAL RISK, AND NO HYPOTHETICAL TRADING RECORD CAN COMPLETELY ACCOUNT FOR THE IMPACT OF FINANCIAL RISK IN ACTUAL TRADING. FOR EXAMPLE, THE ABILITY TO WITHSTAND LOSSES OR TO ADHERE TO A PARTICULAR TRADING PROGRAM IN SPITE OF TRADING LOSSES ARE MATERIAL POINTS WHICH CAN ALSO ADVERSELY AFFECT ACTUAL TRADING RESULTS. THERE ARE NUMEROUS OTHER FACTORS RELATED TO THE MARKETS IN GENERAL OR TO THE IMPLEMENTATION OF ANY SPECIFIC TRADING PROGRAM WHICH CANNOT BE FULLY ACCOUNTED FOR IN THE PREPARATION OF HYPOTHETICAL PERFORMANCE RESULTS AND ALL OF WHICH CAN ADVERSELY AFFECT ACTUAL TRADING RESULTS.

Neural network trading systems are designed to find and exploit nonlinear patterns in market price data.  I find that these systems are among the very most powerful tools available to forecasters. 

The daily neural network signals that I post here in the Forecasting Systems Letter are created by a "mega-system" containing 27 unique neural network systems that I developed using a program called Profit, written by BioComp Systems.  In order to convey the level of success that I believe is reasonably possible to achieve, I am posting data below that I can print out from just one of these 27 systems.  And, taking Murphy's Law into account, I chose a system that is underperforming most of the others. Nevertheless, this performance is hypothetical (i.e., done by computer) and not actual.  It is important to note the disclaimers presented at the top of this page regarding such hypothetical performance.

The statistics presented below show the "out-of-sample" performance of this system between January 1 and October 12, 2002.  The term "out-of-sample" is very important when evaluating trading systems.  It refers to a period of unbiased testing, free of any performance optimization or "cheating" of any sort.  The system was "trained" or developed during an earlier "in-sample" period.  Performance statistics during the "in-sample" period will naturally be biased toward a successful outcome. 

The statistics presented above, refer to trades involving a single S&P futures contract.  These days, one must deposit about $20,000 into a futures trading account to trade such a contract – although I would recommend putting about $40,000 into such an account to cover potential draw downs.  This is highly leveraged, high-risk trading.  The contract will gain or lose $250 for each point movement.  The statistics include a $25 commission per trade and a $25 allotment for "slippage."  One note of caution: these statistics do not incorporate the use of protective stops.  I like to set a stop loss of about 15 points per day.  The use of such stops will usually reduce both drawdown and overall profitability.

The neural network signal is an oscillator that moves above zero to indicate a long position and below zero to indicate when you should be short.  One does not reverse direction until the signal actually crosses over the zero line.  In the illustration below, the top chart is of the S&P futures contract itself.  The green and red dots are buy and sell signals.  The middle chart shows the neural network signal; and the bottom chart shows the accumulation of equity or profit in the trading account.  In this system, trades are always placed at the close of the trading session.

It is interesting to note that this particular system did not really begin to generate consistent profits until April 2002 when the market started its long decline.  Click here to see a trade by trade account. 

The system described above is actually a composite of 200 different neural network "models."  The neural network signal is the average signal from these models.  It is significant to note that even the poorest performer among these 200 models would have lost less than half as much money as a simple "buy and hold" strategy.  The statistics for this model are shown below.

On the other hand, the best model of the 200 showed a trading profit of more than twice that of the overall system (based, after all, upon the average signal).  The statistics for that model are shown below:
In general, one can say that the bell curve of out-of-sample, trading performance for all 200 models significantly exceeds chance expectation. 
 

Return to ForecastingSystems.com