Analysis Paralysis
Over Analysis, Over Simplification and Somewhere in Between
By D.R. Barton Jr.
October 16, 2018
A note to readers: While much of this article’s content is timeless, it is from a past publication and may contain outdated information, missing links or images.
The more I trade and interact with other traders (both old and new), the more convinced I become that the markets can best be approached not as a problem to be solved but as a game of understanding group psychology. Since all of my formal training (chemical engineer, MBA, yada, yada, yada…) is in problem-solving, a market that acts more like a psychological process poses ongoing challenges for me. I tend to step on my head a lot.
Let me explain. When people do something that impedes their own progress we say that they are “tripping over themselves.” I don’t think that such a lackluster metaphor captures the essence of the barrier that most traders face. It seems to me more like “stepping on one’s head.” While physiologically difficult (if not impossible), I think this metaphor strikes a clear picture in my mind of what’s really going on when people are stuck trying to explain the markets as a pure numbers game. I say this because the problem that I’m describing here is that of thinking too much. You can call it over-analyzing, or trying to act smart and say intelligent things. But the bottom line is that most traders seem to try to understand the markets in a purely technical/analytical/logical/linear fashion. When the markets don’t act in accordance with this analytical model, they think harder, overanalyze, pull out new (or old) indicators and get stuck in the mud of numbers, numbers, and more numbers. They step on their head.
Am I saying there is no place for technical analysis? Absolutely not! Technical analysis forms the basis for most actions that good traders take in the markets. Should we resort to relying on fundamental analysis? Fundamentals have their place, especially as set-ups, but it’s tough to use fundamentals to time things. No, what I’m advocating here is interjecting common sense into our trading. Instead of thinking harder and crunching more numbers there are situations that call for a good understanding of crowd psychology and how people’s expectations, hopes, and fears are moving the markets.
Let’s look at some examples of over-analysis, and at some characteristics of traders that strike a good balance between technical analysis and common sense.
Over-Analysis
“A hundred wagon loads of thoughts will not pay a single ounce of debt.”
—Italian Proverb
Does a good engineer, doctor, accountant, computer programmer or other analytically trained professional make a bad trader? As with most complex questions, a good answer is “It depends.” On the one hand, folks that have this type of background are usually very quick to grasp the concepts of technical analysis and can be very controlled practitioners of systematic trading. On the other hand, those of us who have had years of education and practical experience in fields that reward exactness, attention to detail and “being right” have real barriers to overcome as traders. Analytically trained professionals tend to have the most problem accepting that sometimes our models don’t work. Sometimes the markets just don’t make sense and lots of times we’re just plain wrong about the markets!
When it turns out they’ve made an incorrect prognostication in the markets, most new traders (especially the technically trained ones) tend to add complexity in order to understand where they went wrong: extra indicators, more filters, optimization. It’s time to apply years of training and practice in fields that specialize in problem-solving to this special problem called the markets. But instead of “solving the problem,” we step on our heads.
Here’s the tough part: sometimes we can’t solve the problem. In many areas of life, (trading and investing being the one we’re focused on here) there is too much uncertainty or variability involved to definitively solve the problem. Sometimes the best we can do is to find an edge and take advantage of it. Now, before all of my tech buddies freak out, that edge can usually be quantified and put into statistical language. (Whew. For a minute there it looked like this was going to be about psychology and learning “the rhythm of the markets” and all that soft stuff…)
Oversimplifying
“From naive simplicity, we arrive at more profound simplicity.”
—Albert Schweitzer
So far we have concentrated on the undesirable effects of over-analyzing the markets to the point that common sense is thrown out the window. This type of over-analysis has been poignantly dubbed “stepping on your head.” Now let’s take a look at the exact opposite of stepping on your head—the act of oversimplifying. I’ve decided not to poignantly dub this one.
We’ve been concentrating on the behavior of adding complexity to a system or to the analysis of a situation to the point that indecision is almost unavoidable. Let’s look at the flip side of that coin. That would be the case where a trader tries to take an inherently complex environment (like the markets) and simplify the model to the point where the model is meaningless.
How do people do this? One common way is when folks reduce their understanding of the market to one indicator. “If price is above the 50-day moving average (MA), we’re going up and if it’s below that MA, we’re heading down.” People who fall into this trap love the unambiguous results, the ease of understanding the system or model, and the minimal time required to arrive at a market or trading opinion. So what’s the problem with this simplified approach?
It doesn’t work. But don’t take my word for it. My good friend and system savant Chuck LeBeau did perhaps the definitive study on simple entries in his book Computer Analysis of the Futures Market. Chuck and his co-author David Lucas looked at a myriad of single indicator systems as entry signals that might give an indication of future market direction. These ranged from moving averages to stochastics to volatility breakouts. What Chuck found was that none of them were statistically better than a random entry or a coin flip! Oversimplifying has some alluring aspects that I mentioned in the previous paragraph, but don’t be taken in by the promise of getting something for next-to-nothing.
Complex systems and models (like the stock, commodity and currency markets) cannot easily be reduced to one or two simple variables. The same can be said for other complex models like human relationships, economics, etc. When one tries to predict the response of such a complex system to a stimulus, there comes a point where simplifying the model just makes your prediction ability regress toward a 50 / 50 probability.
So what’s a trader to do? Overanalyzing leads to poor decisions or no decisions at all. Oversimplifying causes us to have a useless model that is no better than a coin toss.
Let’s look at several ways that any trader can find a happy median between these two extremes and some starting points for how you can construct systems that mimic those used by top traders.
A Happy Median
Oversimplification is the polar opposite of “stepping on your head,” or over-analysis to the point where common sense is tossed out the window. The main problem is that it leads to poor decision making, or that very common subset of poor decision making: NO decision. This “paralysis by analysis” is a prevalent result of stepping on your head. In our discussion on oversimplifying, we found that reducing complex systems to simplistic models takes us to the point where we’re basically flipping a coin—our decision-making becomes a 50/50 proposition with no edge. Now, let’s take a look at how we can position ourselves between the extremes.
Let’s start out with the basics. In trading, one needs an edge. If your profits and losses are equal, then over time you will lose money after paying for transaction costs and the “hidden” costs of trading (data, software, subscriptions, etc.). This means that to just break even, you have to have an edge! And to be profitable, you need to have a significant edge. In seeking out that edge, people go to the extremes of over-analysis (if these 27 technical indicators and these 11 macro-economic trends line up, then I’ll think about entering…) or of oversimplifying (whenever the price crosses above the 20 period moving average I’ll buy; when it crosses below, I’ll sell).
We’re looking for a nice place in between the extremes. What should you look for when capturing an edge that will keep you from oversimplifying or stepping on your head? Let’s look at four key areas:
Degrees of freedom: This is the mathematical term for how much complexity you build into your system. Every new variable that you insert adds at least one degree of freedom. You have to be careful because some individual indicators add multiple degrees of freedom. In Jack Schwager’s book New Market Wizards, he interviews systems guru and Turtles co-founder William Eckhardt. Eckhardt suggests that no fewer than three and no more than eight degrees of freedom (or variables) should be used in a system. So there is a good rule of thumb for being someplace in the middle.
Complex set-ups, simple entries. In general, if you have any complexity in the entry side of your system, you should build that into the set-up (the part of your system that tells you to get ready to trade) and leave the entry signal simple and unambiguous. (The entry is the part of your system that tells you to “pull the trigger.”) The reason for this is straightforward; you don’t want to give yourself any reason to waiver or be timid when you get an entry signal. It should be clear and simple so that you can act without hesitation.
Profit-taking stops can be more complex. This article doesn’t allow me the space to write adequately about stops and their relationship to set-up and entry techniques. But in general, SPEND MORE TIME DEVELOPING PROFIT-TAKING STOPS than you do on your set-ups and entries. Decision-making is more difficult for most people when you are already in a trade. In addition, devising a process that captures the biggest part of the move after you’re in is generally more difficult than identifying when that move will happen. You can have several different stop algorithms in place at once. Complexity here is not necessarily a bad thing, although it is possible to go overboard.
Use tiny position sizes to protect you. As with all new trading systems, trade tiny sizes until you are sure that you can be consistently profitable with the system. Then you can ramp up your size. You may be thinking, “What if I miss that big move while I’m still ‘practicing’?” All I can say here is that I’ve had many people thank me for suggesting that they start small, and I have yet to hear someone say that this common sense approach has cost them money. Do the right thing when starting out; it’s easy to rationalize trading big lot sizes, but my suggestion is this—don’t step on your head in this arena either!
Models to Follow
“Experience tells you what to do; confidence allows you to do it.” —Stan Smith
Now, let’s close out by looking at a trader who has found a great balance that works well for him.
I’ve known Brad Martin since 2000. Brad spent 14 years as a floor trader at the CME and CBOT and now has been trading six years as an electronic day and swing trader. I’ve had the opportunity to trade with and model Brad’s trading strategies since 2004. The fruits of our efforts have resulted in two really incredible workshops that we teach together. Brad is the type of trader who almost never steps on his head. I’ve never met another trader who acts with more confidence and ease at his decision points. There are several good reasons for this, so let’s explore them a bit.
Brad knows exactly where he’ll get out if he’s wrong before he ever enters a trade. This may seem like a simple thing, but Brad has an unwavering sense of capital preservation and he uses strict risk control on every trade. His dedication to preserving his capital is so ingrained that he never has to think or analyze anything at his decision points. The decision was made before he entered the trade. This absolute “get out point” is actually very liberating. While Brad doesn’t like being on the wrong side of a trade, he knows that it’s part of his business and he does a great job of making his losses impersonal, neutral events.
Brad has a plan. When Brad enters a trade, he knows exactly what factors will cause him to get out of a trade early, what things will tell him to stick around because the move could continue in his direction, and more importantly, what things are not going to influence him one way or the other. One insightful workshop attendee told the group that the amazing thing about Brad’s trading style is not the indicators and factors that he looks at, but rather all of the things that he chooses to completely ignore! Brad knows the factors that give him an edge; he pays close attention to them, and doesn’t let outside distractions influence him.
Brad has confidence that comes from knowing his strategies work. For many people, this seems like a simplistic statement. But I believe that most people who struggle with their trading do so because they don’t believe 100 percent in what they’re doing. Brad makes decisions look easy because they are easy for him. Tiger Woods consistently drains tough putts in pressure situations because he’s done it before. He has a mental and physical routine that works and is proven. The same can be said for Brad’s trading. Tiger doesn’t sink every putt, and Brad doesn’t make money on every trade. But what they both do works, and because of their confidence, neither of them has to over-analyze the situation when it happens in the heat of battle.
To learn more, volume 5 of Dr. Tharp’s Peak Performance course is dedicated to decision making and how factors such as analysis paralysis can be overcome. (Available on Amazon starting in 2024.)
