 Feature
Historical Testing of Efficient Markets
Part II
by Van K. Tharp, Ph.D.
Historical testing is the primary method that some people rely on to develop confidence in trading but it is replete with pitfalls. For example, I talked about the data pitfall in last month�s article. As a result, my goal in the next few articles is to take a trading methodology that I feel confident in and determine what might be involved in doing historical testing with it. I�ll probably devote at least three articles to that topic, but the exact number will depend upon how the research develops.
First, I feel confident that if you buy stocks that show efficient uptrends (i.e., they are basically fairly straight lines going up), with a 25% trailing stop and risking 1% then you will make nice profits under most market conditions. I illustrated this in the Market Mastery newsletter in 2001 and in the recent evaluation of our portfolio. However, a number of questions remain.
1) Is it possible to automate this form of trading? In other words, how do you take something that�s fairly discretionary and turn it into something that�s
objective? Right now, it�s my subjective judgment that determines whether or not something is a good efficient stock. (This question, by the way, is probably the most difficult question for most trading systems.)
2) If the method can be automated, what is the formula that will totally define an efficient stock for us?
3) How can we overcome some of the data problems that are present in historical stock data?
4) What market conditions are favorable for this method and what market conditions should be avoided?
5) And what can we expect from this method long term?
I don�t expect to be able to answer all of those questions in these studies, but if you can begin to understand some of the problems involved in backtesting, then I�ve met my objective.
STUDY: First
an Attempt to Automate Efficiency
In this study I�ve been working with Bob Spear, the developer of Mechanica software, to develop an algorithm to buy efficient stocks automatically. We decided to work with the S&P
500 database, which presented our first problem because it basically represents today�s S&P
500 even though the database goes back to 1980. Wouldn�t you love to know what the S&P 500 of 25 years from now would consist of? What new stocks will be so strong that they will be considered one of America�s top 500
stocks? All you�d have to do is just buy and hold the stocks that are not on it now and you�d make a fortune. But that�s what we were dealing with. Furthermore, all of the stocks were split and dividend adjusted. That means that a stock like Microsoft, which came out in 1986 begins at a price of about 15 cents (which also isn�t very realistic). But this is taken care by virtue of the fact that we buy an outsized position (10s of thousands of shares) so the accounting arithmetic
works.
Automatic Entry
The problem with automatic entry is that my efficiency algorithm, which is a composite of four efficiency algorithms, is just a preliminary screening tool. You could probably use a channel breakout or a list of stocks making new highs and do a similar screening. It doesn�t matter because I still have to look at the charts to pick the stocks I want to buy. Our first solution to that was to do some sort of smoothing function. Bob programmed Mechanica to rank the S&P 500 in terms of smoothness for each day of the 25 year database and store that ranking for later use. Our first algorithm was to rank the inverse of the standard deviation of the change in daily prices. We took the inverse because we assumed that the smallest standard deviation would be the smoothest.
Mechanica, then did the following:
on the first trading day of each month, it calculated the composite efficiency of each of the stocks in S&P 500. The first thing it did was determine if the composite efficiency was greater than 8. If it was greater than 8, it then purchased the top 10% of them in terms of smoothness. That was it � our automated screening process. But did it work?
Exit
The exit in our model was very simple -- we used a 25% trailing stop. Thus, if the price dropped 25% from our entry, we were out. However, whenever the stock made a new closing high, the stop was adjusted, now being 25% away from that new high. It was always raised and never lowered. Thus, the stop became both
a reason to abort a trade and our profit taking exit. I used the 25% trailing stop because it seems to be an excellent substitute for buy and hold.
Position Sizing
We began each position with a 1% risk based upon the Total Cash available. Since our portfolio started out with $100,000, each position would risk $1000 and be a total investment of $4000. When our total cash changed, our 1% risk would always reflect that change.
Commissions and
Slippage
Some of our positions were huge. For example, if you were to risk $1000 with a 25% stop on a split adjusted MSFT starting out at say 16 cents, then our initial risk would be 4 cents. We�d be buying 25,000 shares. The cost of buying those shares, especially in the mid 1980s would be tremendous. However, we wouldn�t really be buying a split adjusted stock. Instead, we�d probably be buying a $25 stock. As a result, we elected to include a 1% cost per trade
(1% in and 1% out) for commissions and slippage.
Results
The results of this study made us look like geniuses. These results are summarized in Table 1. We are able to turn $100K into $265 million in 25 years.
| Table 1: Summary Results |
| Initial Balance |
100,000 |
|
$
Won |
330,803,321 |
| Net
Win Loss |
265,541,946 |
|
$
Lost |
65,261,375 |
| Ending
Equity |
265,641,946 |
|
Incentive
+ Fees |
0 |
| ROI |
265542% |
|
Other
Credits |
0 |
| Compound
Annual ROI |
37.83% |
|
Commission/slippage
netted |
0 |
| Max
Drawdown % |
38.65% |
|
Other
Debits |
0 |
| Max
Drawdown % Date |
19871204 |
|
|
|
| Longest
Drawdown in years |
1.49 |
|
Long
Wins |
382 |
| Longest
Drawdown Start Date |
20020410 |
|
Long
Losses |
292 |
| Longest
Drawdown End Date |
20031006 |
|
Short
Wins |
0 |
| MAR
Ratio |
0.98 |
|
Short
Losses |
0 |
| Sharpe
Ratio |
1.93 |
|
Long
$ Won |
330,803,321 |
| Return
Retracement Ratio |
5.5 |
|
Long
$ Lost |
65,261,375 |
| Sterling
Ratio |
0.54 |
|
Short
$ Won |
0 |
| Std.
Dev. Daily % Returns |
1.21% |
|
Short
$ Lost |
0 |
| Value
at Risk (99% confidence) |
3.33% |
|
Largest
Winning Trade |
22,969,429 |
| Average
Expectation Value |
33.27 |
|
Largest
Losing Trade |
2,209,232 |
| Expectation |
176.28% |
|
Transactions
netted at open |
0 |
| Kelly |
0.45 |
|
Average
Winning Trade |
865,977 |
| Sum
of Up % / Sum of Down % |
1.36 |
|
Average
Losing Trade |
223,498 |
| Percent
New Highs |
16.74% |
|
Max
Consecutive Wins |
16 |
| |
|
|
Max
Consecutive Losses |
13 |
| Trades |
674 |
|
Days
Winning |
3,465 |
| Trades
Rejected |
701 |
|
Days
Losing |
2,729 |
| Wins |
382 |
|
Average
Days in Winning Trade |
378 |
| Losses |
292 |
|
Average
Days in Losing Trade |
80 |
| Percent
Wins |
56.68% |
|
|
|
| Avg
$Win to Avg $Loss |
3.87 |
|
Number
of Margin Calls |
0 |
| |
|
|
$
Largest Margin Call |
0 |
| Start
Date |
19801001 |
|
|
|
| End
Date |
20050422 |
|
Size
Adjustments |
0 |
| Max
Items Held |
16,487,507 |
|
Size
Adjusted Items |
0 |
| Total
Items Traded |
117,196,789 |
|
|
|
| Total
Slippage + Commission |
9,000,400 |
|
Process
time (H:M:S) |
0:28:15 |
Notice that we made a compounded annual return of 38.8%. How many people did that from 1980 through 2005? Our worst drawdown was 38.7%, which occurred in 1987. It made 674 trades and rejected 701 trades (because we were fully invested). 56.7% of our trades made money and the average
win was 3.87 times bigger than the average loss. We also spent 378 days in a winning trade versus 80 days in a losing trade.
And we spent nine
million four hundred dollars in trading costs, so
you can�t say that low costs influenced our results.
The Sharpe ratio of this system is nearly two and the
System Quality Number is close to Holy Grail range.
Figure 1 shows the distribution of yearly returns. Notice that we only have two years of negative returns, 2002 and the last year of trading. Two losing years out of 25 is an exceptional, especially with no screening for bear markets. And during nine of the 25 years we made over 50%.
Yearly Returns from Our Efficiency System.
So what is your reaction to these results? Do you need more information or do you want to jump on this and trade it? Remember, I�ve already told you that I believe the efficiency model is an excellent long term trading system.
Perhaps you�d like more information. After all, the maximum drawdown is nearly as big as our compounded annual return. Let�s look at the drawdowns produced by this system.
These are shown
below.
Drawdowns in our Efficiency System
Notice that most drawdowns did not go over 15%, but twice we exceeded 35% and we exceeded 25% quite a few times. Was that due to our risk? Well,
the table below plots the risk against the equity curve.
Risk Versus the Equity Curve
Our maximum initial risk should only be 25%,
but since one position could get very large and have a significant
risk, you�ll notice that the total risk sometimes gets bigger than
40% (e.g., in 2002-3). Nevertheless,
risk is still about what we�d expect it to be in this system.
So would you trade it?
All you have to do is buy stocks that are going up in a
straight line and put a 25% trailing stop on them, risking 1% of
your portfolio with each. You
should be able to do that, but will you?
Why or why not?
Problems
with the Study
Many of you might be salivating to trade this
system. Based upon these
results I could probably
sell it to others.
However, I see a number of problems with the study, which
we�ll be addressing in future articles with additional studies.
Let me just list the problems here to see how many you might
have caught. (Oh, and if
you see some I didn�t mention, please let me know. Email van@iitm.com).
1)
Why are the worst performing years 2003 and 2005?
We had a bear market in 1980 through 1982.
We had the crash in 1987.
We had a nasty market in 1990-1.
What happened with those and why didn�t we have drawdowns
in those years? Some of
the problems below might explain that.
2)
We have the S&P 500 data problem.
We are basically trading a limited universe that includes the
very best performing stocks during that time period.
And those stocks were pre-selected.
I believe that the efficiency method is good enough to find
the S&P 500 that will exist 20 years from now, but that still
doesn�t negate the selection problem with these data.
To solve it, we�ll be looking at a database of stocks that
includes the real S&P 500, including those stocks which no longer
exist. Finding such a
database is very difficult, but we have succeeded in creating it!.
3)
Perhaps any trend-following algorithm would produce
similar results with this database.
We�ll soon test a 180 day breakout system to see how it
compares. At least with
that system, I�ll know that the stock is doing what I expect it to
be doing when we buy it, making a new 180 day high.
4)
Our smoothing algorithm is flawed.
Remember that we are basically ranking stocks according to
the standard deviation of daily changes in price.
The smaller the standard deviation the greater the ranking,
which is quite interesting because it tends to favor the low-priced
split adjusted stocks in our database.
Thus, it is biasing us to buy the very stocks that will go up
the most.
5)
Third, one of my Super Traders looked over a list of
the trades made during the study.
He concluded that the stocks being purchased were anything
but efficient. Thus, our
algorithm that we automated doesn�t seem to be doing what we
wanted it to do.
6)
Fourth, we will have the most cash during the short
bear markets of this 25 year period.
Thus we should be situated to accelerate out of each
drawdown. Now that�s
good, since most of the period under consideration was a secular
BULL market. But today
we�re in a secular bear market.
What if we have a 3-4 year down period?
We�d be buying stocks as the market continues to go down.
We could have a 25% loss on top of a 25% loss on top of a 25%
loss. That would be a
disaster. Can we filter
out the bear market periods and make this perform better?
During the next few months I�ll be presenting
one or two additional studies each month to address these issues.
But the important lesson for you was if you saw the flaws in
our backtesting. Or did
the result make you just want to jump on the system and trade it?
These types of flaws occur all the time and that�s one of
the things I�d like to point out.
By the way, if you have some interest in
Mechanica, which we are using in these tests, then
go to the Mechanica web site -- http://www.mechanicasoftware.com.
Mechanica is the new windows version of Trading Recipes.
About Van Tharp: Trading
coach, and author Dr. Van K. Tharp, is widely recognized for his
best-selling book Trade Your Way to Financial Fre-edom and
his outstanding Peak Performance Home Study program - a highly
regarded classic that is suitable for all levels of traders and
investors. You can learn more about Van Tharp at www.iitm.com.
|
