The Forecasting Systems Letter
Jeffrey Mishlove

---

The story I am going to present in this article is among the most interesting I have encountered in my four-year exploration of computerized, financial forecasting systems.  As an anchor point, let me refer you to my Letter of October 19, 2002, in which I described "my best BioComp Profit neural network system."  Ironically, in the spirit of "Murphy's Law" it was just about at that time that this very same system began it's breakdown -- as is shown in the graph below:

As in all (or most) such diagrams from BioComp Profit, the top chart shows the S&P futures contract price, with red and green dots indicating buy and sell points.  The middle chart shows the yellow neural signal oscillating between -1 and +1.  The bottom chart shows the accumulation of equity from trading a single contract using this signal.  Clearly, what we see here is a "phase shift" in the market.  The very same nonlinear formulas that had been producing positive equity began yielding a similar, but opposite result.

I might mention here that my partner in Thinking Allowed Productions (see www.thinkingallowed.com) is none other than Arthur Bloch -- the author of the best-selling series of Murphy's Law books.  Because I have known Arthur for nearly a quarter century, I have had an opportunity to contemplate this principle -- that if something can go wrong, it will.  It has, indeed, made for a very successful series of humorous books.  I suppose Arthur Bloch's very success with Murphy's Law is proof enough that we can rely upon this law to operate only intermittently.  That is one reason why I remain hopeful and optimistic regarding neural forecasting systems.

The "Gift04" system, as I reported last October, was originally constructed by my colleague Ingvar Engelbrecht in Sweden.  Ingvar engaged in a very methodical study of input variables.  The ones he selected included special transformations, i.e., fast moving-averages, etc., designed by Jurik Research.  In developing this system, Ingvar made a point of avoiding all optimization or transformation of the input variables using the various routines available within BioComp Profit.

I decided to see whether this system could be resuscitated by retraining new neural "meshes" on more recent time-series data.  After all, as I have reported in the past three editions of this Letter, I have recently been able to create a number of promising systems (based on out-of-sample testing) in this manner.  So, following the same protocol that led to these earlier results, I ran the neural modeling routine with the "training" running from Jan 1, 1999, through Jan 26, 2003.  However, as the diagram below shows, the out-of-sample testing yielded decidedly uninteresting results:

I guess this is a good place to point out that I now have discovered that I had, for all of these months, misunderstood the nature of the input variables that Ingvar had so diligently researched and then so graciously provided to me.  I mention this now simply to highlight how easy it is for human error to creep into systems of this sort.  Ingvar's original system included an input based upon special indicators sold by BioComp Profit.  One of them was designed to forecast the VIX or volatility index.  I misread this and therefore substituted the volatility index, VIX, directly.  In this instance, that turned out to be a fortuitious mistake -- but it might just as easily have been costly.

In any case, it occurred to me that it might just be useful to launch a second retraining effort. This time, I decided, instead of working with Ingvar's original input variables, I would also transform them using the proprietary "Cook's Transformations" that are available in BioComp Profit Professional.  My best guess is that Cook's transformations optimize the input variables themselves, across the in-sample training data, in order to control three different factors: correlation with the target variable, movement similarity with the target variable and "mean crossing difference" with the target variable.  This strategy has resulted in what seems to be the most promising system I have created to date.  The following graph shows the out-of-sample test results:

The statistics that describe the out-of-sample performance of this system -- consising of 200 individual models whose individual signals are averaged using an equity-weighted voting method -- are presented in the graphic below:

When I saw these results, I honestly was not prepared to believe them.  They simply looked too good to be true.  So, I contacted Ingvar and asked him to attempt an independent replication.  He managed to succeed in creating a system of 200 models that performed at better than 30% of perfect.  I have subsequently also done two further replications with similar results.

Of course, there is no guarantee that this system will hold up and actually be useful for real trading.  To the extent that it is dependent upon certain training start and stop dates is it probably fragile and brittle.  If Murphy's Law were to hold, one might expect the system to break down right away -- now that I have reported on it in an optimistic manner.

On the other hand, I do remain confident -- because these same inputs performed quite well for the first ten months of 2002.  Perhaps this system, too, will have another six months or so of useful life.  Time will tell.  In the meantime, I intend to use it as one of about eight different systems that are now part of my neural network "mega-system."

Forecasting Systems Letter Archive
Return to ForecastingSystems.com