Masters (1G): "Back testing Basics" - The Basics of Systems Testing

Participants will learn how logical backtesting will not only assist the trader in the construction of a logical trading methodology, but also give him/her the confidence to "Trade the Plan" and thereby avoid the emotional pitfalls which plague most traders.

************************************************************

Why Backtest, Anyway??

Whether we want to realize it or not, emotions certainly play an extremely important role in our trading success, or lack thereof. Anyone who has placed a trade on an active market has noticed the effect this simple act can have on your level of attention to the market and your general state of tension when the trade is still open.

Unfortunately, this elevated state of attention, sweaty palms and a queasy stomach can cause us to make decisions and take actions that we would not consider in our normal state of mind.

Paper trading is a must to develop a system, but there is no way in the world that this simulated exercise can closely approximate the real deal.

I’m reminded of a trader I worked with a few years ago. Watching him paper trade one day I saw a completely relaxed individual on a couch in front of the monitor, a cup of coffee in his hand and a newspaper conveniently arranged within sight. Occasionally he would study the monitor, observing the indicator positions at regular intervals. When he saw the correct set up, he would announce, "Yes, I’ll take that one" and dutifully record the simulated trade in his market journal.

The previous routine then set in, with occasional glances at the monitor "just to see how we’re doing" and exiting the trade when the indicators dictated.

A few weeks later, when the same individual was trading his own, real money, the picture was totally changed. The coffee and newspaper were nowhere in sight, he was poised precariously on the edge of a chair, observing every tick on the chart with absolute intensity. Gone was the relaxed, normal state in which he was able to mentally assemble the total picture of the trade in progress and make logical decisions accordingly. Replacing his logical attitude was one of intense fear of losing even a small percentage of his trading, or risk, capital. Each tick of the market brought a totally new assessment of the market, each new scenario becoming less and less logical and more fear oriented. Finally the guy just liquidated the trade simply because "he just couldn’t take it any more" and took a small loss on the trade, because " it was only going to get worse!!". Naturally, following the rules of his trade, the transaction went on to produce the expected result, a modest profit.

Bottom line, this individual simply did not have the confidence to "Trade the Plan" and execute the trade in the exact same manner which he had done so successfully on paper.

Gaining Confidence with Statistics

Most traders, whether they realize it or not, possess a logical, innovative, scientific mind which allows them to assemble a series of observations into a logical sequence which leads to a specific action.

Unfortunately, the human being in all of us can frequently override our logical side when our own money is on the line and cause us, in retrospect, to make some moves which we would not make in our more logical mind.

This same logical, scientific mind can be trained to perform well under stress by providing it with logical, scientific evidence that the trade plan devised is indeed functional and profitable.

The only way I have found to quantify and display the above logic is a methodical backtesting program which will accurately describe how the selected methodology would have performed had it been accurately applied over a considerable period of time.

For instance, if we could prove, using probability tables or other mathematical tools, that if condition 1 and condition 2 and condition 3 occur within 5 minutes of each other that the market would tend to rise over the next 30 -60 minutes at least 75% of the time, our confidence level in placing the trade would be enhanced considerably.

Progressing further, we can also quantify the fact that given a certain set of market conditions, the market in question could be expected to rise $2.50 per share at least 70% of the time, rise $3.00 per share at least 60% of the time, but was able to rally $5.00 per share only 30% of the time.

In contrast to the fixed target strategy discussed above, we can establish self adaptive criteria which uses previous market data to establish high percentage targets for selected trades.

Armed with these solid facts generated from previous market data, we are more able to assess a logical method for exiting the trade with a reasonable profit.

Also, we can derive a set of conditions, when present, will result in the trade moving against our position in excess of 70% of the time. With this knowledge we will have a good idea of when the trade is in trouble and should be exited without delay.

Methods of Creating the Data

A trading method can be tested in a number of ways. All have their advantages and difficulties.

I actually began system backtesting by recording the hourly price changes in agricultural commodities as they were being broadcast by local radio stations in the 1970’s. I’d then continue to analyze patterns using standard engineering graph paper and a hand held calculator. Crude as it was, this basic method was able to uncover in my mind the two most basic principles which still govern my system development today, specifically daily price follow-through and intraday range expansion.

A backtesting routine does not necessarily have to be computerized to be effective as long as it is methodically applied in a consistent manner to a reliable array of market data.

That being said, there is a lot to be said for using one of the commercially available charting and systems analysis programs to test your strategy.

Obviously, an accurate program can save the trader countless hours of laborious calculations, thereby freeing up the individual for trading or other system development functions.

Computerized analysis and reporting is undoubtedly more accurate as well. The human eye will not pick up many trades which are activated by just one or two ticks in the price data. The computer is much more accurate at recording all trades which would have been taken had the system been trading real time. The human eye just isn’t as specific as a computerized analysis.

The human mind has a quite selective memory when it comes to picking which trades to include and which not to include in the test sample. It’s real easy to rationalize "Something else would have kept me out of that trade" or "I probably would have used a smaller target or larger stop here".

Always Check the Charts

It is of vital importance at this stage of the system testing process that the system developer / trader continually refer back to the actual price charts to observe what the system testing results are actually reporting. A set of variables which produce the highest profit return on the system may not be the set that produces the best system from a trading standpoint.

Frequently the best system settings will be the one with a better ratio of profit to drawdown, higher percentage of profitable trades and fewer consecutive losses. Usually the system settings producing these results will not be that much less profitable than the highest profit settings. A system which produces a higher average profit per trade, fewer consecutive losses and a lower drawdown will often be easier to trade than the most profitable system. This type of analysis will usually give the trader a version if his / her system which is much more adapted to your own trading style.

In order to give an example of a computerized optimization run report, I have included below a portion of an optimization report for my Spectrum System as applied to Ebay stock for the time period January 1, 1999 through September 4, 1999. The report shows the 15 most profitable results from the test. The data has been sorted by net profit in descending order.

Ebay Testing Results 1-4-99 through 9-4-99

The four left hand columns above represent the four inputs for the Spectrum System. The next column represents the net profit for the system for each of the input settings on the left. Subsequent columns detail the Maximum Drawdown, Percent Profitable Trades, Profit Factor, Maximum Consecutive Winning Trades, Maximum Consecutive Losing Trades, and Average Profit per Trade.

This report clearly demonstrates that the most profitable system settings may not always be the parameters one would want to trade. Carefully examining the second line of the report, note that although the net profit was a bit lower that the top sorted system, the maximum drawdown was considerably less, the percent of trades which were profitable was greater, as was the profit factor. Of particular note is that the number of consecutive losing trades was half of the number on the first line. Also note that the per trade average profit was the highest for this system setting among all the tests listed on this brief report. Overall, I would much prefer to trade this system than the one which received the top rating on net profit alone.

Conclusions

I can’t stress enough the importance of constant monitoring of the charts during the system testing phase. Many times I have discovered bugs in the actual system theory or programming just by observing the behavior of the system under varying input settings. Remember that you originally constructed your system by observing chart patterns and the manner in which the market behaved around selected indicators and selected price points. It only makes sense that you continually evaluate your testing using

the same criteria.

Accurate system backtesting is an essential ingredient in the development of a trading system.

Additionally, the information gleaned from this process can go a long way towards giving the trader the confidence to "Trade the Plan" after "Planning the Trade".

Contact Information:

Dr. John F. Clayburg

29568 Hwy 141

Coon Rapids, Iowa 50058-7178

5239. Voice and FAX

e mail: clayburg@pionet.net

Web Site: http://www.clayburg.com