自由式国际象棋的启示：融合基本面与量化分析

GLOBAL FINANCIAL STRATEGIES www.credit-suisse.com

Lessons from Freestyle Chess Merging Fundamental and Quantitative Analysis September 10, 2014

Authors

Michael J. Mauboussin [email protected]

Dan Callahan, CFA [email protected]

“Weak human + machine + superior process was greater than a strong computer and, remarkably, greater than a strong human + machine with an inferior process.”

Garry Kasparov1

In the late 1990s, machine beat man in the game of chess. Software programs can now outplay humans in most board and card games, with the exception of poker and Go.

In freestyle chess, humans are allowed to use computers to augment their play. Currently, man plus machine is better than man or machine.

While chess and investing have important differences, they also have useful similarities.

The question is whether a melding of fundamental and quantitative methods can improve on either approach by itself.

Fundamental analysts can leverage the computer’s ability to gather data and crunch numbers.

Quantitative analysts can leverage the analyst’s ability to sort causality and detect patterns.

Machine + Man > Machine or Man

You can mark May 11, 1997 as the date that machine beat man in chess. On that Sunday, Garry Kasparov, the world champion, lost the decisive last game to Deep Blue, a computer that IBM built. With that, Deep Blue defeated Kasparov in the six-game match 3 ½ to 2 ½. Kasparov, who was the number one player for an astounding 20 years and is perhaps the greatest player of all time, called that final showdown “the worst game of my career.”

Kasparov’s willingness to face IBM’s best demonstrated that he embraced machine play, and he has been a great ambassador for the game. But he has lingering misgivings about Deep Blue’s victory. “I don’t have any proof of foul play,” he wrote, but “I live in doubt.”2 There’s little doubt that the win gave IBM a boost: The stock’s advance the next day, net of the market’s move, added $1.7 billion to the company’s market capitalization.

Notwithstanding Kasparov’s reservations about IBM’s tactics in that match, it is now well established that machines can beat humans in chess. One way to measure the progress of computers is with the Elo rating system, which is a method to calculate the relative skill of players in head-to-head competition. Today’s best computer programs have Elo ratings of about 3,200, more than 300 points higher than the world’s greatest players. That advantage suggests that the stronger player is expected to win close to 90 percent of the points in a match.3 To add some context, a bright beginner would have a rating of about 600 and a grandmaster needs to achieve the level of 2,500.

Chess, which the renowned German writer Goethe reportedly called “a touchstone of the intellect,” was the gold standard for machine intelligence from an early date.4 But computers were beating humans in other games well before Deep Blue’s success. Exhibit 1 shows the date at which computers achieved superhuman status in a number of games over the past couple of decades. Most of these games are largely computational, which plays to the computer’s strength.

Exhibit 1: Machine versus Man in Various Games

Zen series of programs attains rank 6 dan in fast games; programs improving at Go 2012 Very strong amateur rate of about 1 dan per year, may surpass world champion in about a decade.

Source: Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford: Oxford University Press, 2014), 12-13.

The victory of Watson, a “cognitive technology” also created by IBM, over champions of the game of Jeopardy! was especially striking because Watson had to be able to handle complex language as well as vast amounts of information.5

Go is also notable in that software programs have yet to beat the best players. Go has different features than chess, including a larger board, fewer restrictions on moves, and the fact that pieces get added, not removed, as the game progresses. Still, artificial intelligence researchers expect computer programs to beat the world champion in about a decade’s time.

Shortly after his loss to Deep Blue, Kasparov introduced a new form of playing called “advanced chess,” or, as it is more commonly known today, “freestyle chess.” (The concept of using computers to augment play had been around for a long time.) In freestyle chess, humans are allowed to use input from chess programs to select their moves. It’s no longer man versus machine, but rather man plus machine versus all comers.

In 2005, a team called ZackS won a freestyle tournament by beating an opponent that included Vladimir Dobrov, a grandmaster, his highly-rated teammate, and their computer programs.6 There was some speculation that ZackS was actually Kasparov’s team, but in fact it was two twenty-something-year-old guys in New Hampshire named Zackary Stephen and Steven Cramton. Stephen has a master’s degree in statistics and spent his days as a database administrator. Cramton was a soccer coach in the fall and ran a snowboarding program in the winter. They used four chess software engines in all but relied primarily on two of them. They also developed their own database for research and opening analysis.7

Freestyle teams are currently better than the best machines, although the gap is likely to narrow over time. So for now, man plus machine beats man or machine. A recent estimate places the advantage of the freestyle players over the best programs at 100-150 rating points, which suggests they are expected to win about two-thirds of the points in a match.8 Freestyle teams appear to be melding the strengths of humans and computers while mitigating the weaknesses.

There’s a surprising fact about ZackS’s story. Stephen and Cramton are not great chess players. Stephen’s rating was 1,381 and Cramton’s 1,685. Were Cramton, the higher rated player, to go head-to-head with Dobrov, the grandmaster, Dobrov would be expected to win 99 percent of the points. No contest. This raises an essential question: What exact skill, or skills, did ZackS have that allowed the team to be so effective?

Tyler Cowen, a professor of economics at George Mason University, dedicates a chapter to freestyle chess in his terrific book, Average Is Over. He draws four lessons from the success of freestyle chess:9

1. Human-computer teams are the best teams.

2. The person working the smart machine doesn’t have to be an expert in the task at hand.

3. Below some critical level of skill, adding a man to the machine will make the team less effective than the machine working alone.

4. Knowing one’s own limits is more important than it used to be.

There is one other fascinating aspect of the current chess scene. When the 22-year-old Magnus Carlsen won the world chess championship in 2013 by defeating the 43-year-old Viswanathan Anand, he was the first player to come of age in a time when computers were always better than humans. When Deep Blue beat Kasparov, Carlsen was only six years old.

So as he developed as a player, Carlsen learned not only from other players and coaching but also by observing how the software programs played the game. Indeed, analysis of his game in the qualifying tournament suggested that “he played more like a computer than any of his opponents.”10

The goal of this report is to explore the applicability of freestyle chess to the world of investing, where fundamental analysts are “man” and quantitative analysts are “machine.” More pointedly, might there be a way that investors can combine the strengths of fundamental and quantitative analysis while sidestepping the weaknesses?

Chess and Investing: What’s Different and What’s the Same

Let’s start with the obvious and relevant point that chess and investing are different in important ways. To begin, a chessboard has 64 squares (8x8) and the moves of each piece are set. So while there are a massive number of possible outcomes, the game itself is played in a stable and linear environment. Markets are much less stable and exhibit non-linear properties. In chess, the board and pieces don’t care about what you think. In markets, the beliefs of participants feed back onto the market itself. In finance, the models of the world shape and reshape the world itself.

As each player can see all the pieces on the board, chess is a game with perfect information. But in investing the information each investor has is partial, not perfect. As a result, a chess player can use substantial computational power to his or her advantage, whereas an investor does not have a similar source of edge.

Further, chess games have a beginning, middle, and end. Markets are effectively perpetual.

In a game of chess, players compete head to head. In markets, investors compete with the aggregate of many investors, or the crowd. Individual mistakes do not cancel out in head-to-head matchups but they can cancel out in a group. Indeed, diversity is one of the underpinnings of the “wisdom of crowds.” On the other hand, crowds also make collective mistakes from time to time, allowing for the “madness of crowds” and investment opportunity.

Finally, chess is a game largely of skill. Elo ratings measure skill and are a reasonably reliable predictor of which player is likely to win. Importantly, differential skill is relevant. Investing is a game largely of luck. The reason is not that investors are not skillful. By any reasonable measure they are more skillful than ever. Rather, the distribution of skill has narrowed, leaving more to luck. This is another way of saying that it is more difficult today to gain an investment edge, although by no means impossible.11

Still, there are similarities between the two activities that are worth noting. Both realms are subject to biases and mistakes induced by stress. For example, Kasparov admits that he was “in no condition to play chess” as he faced Deep Blue in game six and that his loss came from an “infantile blunder in the opening.” If even a great champion can get “exhausted and confused” in the game he normally dominates, it is easy to see how investors may also make mistakes in judgment.

In chess and investing, new information arrives that should allow you to update your beliefs. As a result, you cannot fully anticipate the next, best move. John Holland, a professor of computer science, engineering, and psychology at the University of Michigan, says, “Strategy in complex systems must resemble strategy in board games. You develop a small and useful tree of options that is continuously revised based on the arrangement of pieces and the actions of your opponent. It is critical to keep the number of options open. It is important to develop a theory of what kinds of options you want to have open.”12

Process is also at the core of success in both fields. Chess players assess moves and attempt to skillfully select those that offer an advantage over a competitor. Because skill predominately determines outcomes, small deviations from a proper process can be very costly. Process in investing is about finding an edge, or mispricings, and building a portfolio that takes advantage of the mispricings. Because luck looms large in investing, short-term outcomes are an unreliable indicator of skill. But over time, good process wins.

Fundamental and Quantitative Analysts – Can We Freestyle?

Truth be told, fundamental and quantitative approaches to active investing tend to work mostly independently. There are certainly organizations that have attempted to meld the two, but one approach tends to dominate.

Further, neither camp is fully convinced that the blend leads to better outcomes.

For example, quantitative investment managers were asked in a recent survey, “Does [a] fundamental overlay add value to the quantitative process?” More than two-thirds of the respondents disagreed that the most effective process combines the two.13 Expressing clear skepticism about the value of a fundamental overlay, one manager quipped, “the fundamental analyst is a costly business monitor compared to a $15,000 computer.”

The cultural divide runs the other way as well. In the same survey, one money manager said this, “Can a firm with a fundamental culture go quant? It is doable, but the odds of success are slim. Fundamental managers have a different outlook.” That fundamental and quantitative analysts have different personalities and training reinforces the intellectual and practical divide.

Notwithstanding this cultural divide, here are some ideas about how an investment firm can take steps toward freestyle investing.

What Fundamental Analysts Can Take from Quants

Computers are really good at examining lots of data and crunching numbers. These are two activities that humans aren’t so good at. So it’s natural that the quant overlay will feature these abilities:

Methods to offset limited recall or experience. The key to generating excess returns in the market is to have a point of view that is different than what the market is expressing. There needs to be a gap between fundamentals—for example, what a company’s future financial results will be—and expectations, what the market expects the results to be.

Implicit in these expectations gaps is a forecast of the future. Your forecast need not be as precise as a single point estimate, but you must see the distribution of outcomes and their associated probabilities differently than the market does. The challenge for fundamental analysts is that they are generally poor at making forecasts. Here are two areas where quantitative thinking can be very helpful.

The first is what Daniel Kahneman, the eminent psychologist, calls the “inside” versus the “outside” view.14 The basic idea is that when we face a problem, our natural approach is to gather information, combine the information with our own input, and project into the future. Kahneman calls this the inside view and it often leads to forecasts that are poorly calibrated because we do not take into consideration all of the information that is relevant to the problem.

The outside view considers a problem as an instance of a larger reference class. It asks a simple question: “What happened when others were in this situation before?” Using the outside view allows a fundamental analyst to make a more informed forecast. For example, consider the case of an analyst who is trying to forecast sales for a company that currently has $20 billion in revenue. An analyst using the inside view would look at each business line and aggregate them. An analyst using the outside view would consider the distribution of growth rates for all companies that at one point had sales of $20 billion. A proper blend of the two approaches yields a better forecast than a simple reliance on the inside view.

Reversion toward the mean is a closely related concept.15 Reversion toward the mean says that an outcome that is far from average will be followed by an outcome with an expected value closer to the average. Reversion toward the mean occurs any time the measure of the same metric over two time periods has a correlation of less than one. Indeed, the correlation coefficient is a good proxy for the rate of reversion toward the mean, with low correlations implying rapid reversion.

In our experience, few fundamental analysts properly combine the inside/outside view and reversion toward the mean in making their forecasts. A quantitative approach would aid them in this task.

Let the computers crunch numbers. Humans are much better at seeing certain patterns than computers but are much worse at doing calculations. So any time there is an aspect of fundamental analysis or portfolio construction that can benefit from number crunching, let the computer do its thing.

One of a fundamental analyst’s challenging chores is to update his or her point of view as new information comes in. Similar to a chess player, an analyst’s view on a position is necessarily subject to revision as additional information is revealed. There is a formal and mathematical way to do this through Bayes’s Theorem.16 The theorem tells you the probability that a belief is true conditional on some event happening.

Most fundamental analysts struggle with incorporating new information for all but that with the most obvious implications. One of the main reasons is confirmation bias, a tendency to seek information that substantiates a prior point of view and to discount, or dismiss, information that disconfirms a point of view. And even analysts who incorporate new information struggle to adjust their beliefs sufficiently.

Another area where number crunching can be helpful is in portfolio construction. A quantitative take on a portfolio can reveal exposures to factors or biases that are hard to identify otherwise. Even if fundamental analysis provides the raw material, in the form of ideas with edge, quantitative analysis can allow for some guidance in putting those ideas together so as to come up with an effective finished good.

Let the computers cast the net wide. Fundamental analysts generally have a much smaller universe of investable securities than quantitative analysts do because they add the constraint of research coverage. Let’s look at equities as a case in point. You start with the whole equity market, refine it to the investable universe (winnowed by style, geography, or other constraints), select companies to cover, and then construct a portfolio. A quantitative approach has no need for coverage and hence can work with a larger universe.

A fundamental analyst can use a quantitative approach for screening. Indeed, this is the area where quantitative analysis is already used most often by fundamental investors. This combination works if computers are better than humans at generating alternatives and humans are better than computers at winnowing them down.

One of the ways that the humans in freestyle chess added value was by examining how the different chess software programs disagreed.17 This allowed the humans to compare and contrast approaches and to carefully weigh the best move. Likewise, multiple quantitative screens yield different ideas, which provide a fundamental analyst with the ability to add value as he or she prunes the variations to decipher value gaps.

What Quants Can Take from Fundamental Analysts

The power of algorithms, and hence the strength of quantitative analysis, is that they faithfully allow you to reach your goal. But this is only true if the algorithm tightly matches the environment. Slippage between the model and the world increases as change occurs. An example is the correlation between asset prices. Those correlations may be stable over an extended period, but a regime change can alter relationships rapidly and in some cases violently. Such changes render past relationships, and the models that are built on them, useless.

On a panel discussing behavioral finance at an investment conference, one discussant asserted, “I have never seen a situation where overriding the quantitative model has improved results.” That may be true in his firm. But just as humans still add some value to freestyle chess, there are some ways that fundamental analysis can add value to quants:

Separate circumstances (causality) from attributes (correlations). This is one of the hottest debates in the discussion of big data. Some big data enthusiasts have suggested that we no longer need to consider causality at all. The authors of a best-selling book on the topic claim, “Society will need to shed some of its obsession for causality in exchange for simple correlations: not knowing why but only what.”18 If you believe that in finance and build a quantitative approach consistent with it, you will fail. Correlations are not sufficiently reliable.

Fundamental analysts can help with the question of causality. In stable environments, correlations can be very effective at revealing causality. For example, retailers know that a customer who buys a certain basket of goods has a certain probability of buying a related basket of goods. So far, so good. But in unstable environments, such as markets, correlations without theory to explain the relationships are very dangerous either because the results are spurious or the correlations themselves change for other reasons.

Naturally, every quantitative strategy that will be presented to you will have done well in backtesting. But there should be some underlying theory to explain why the mispricing occurred and how the model exploited it. A fundamental mindset can help identify such theories. One illustration is the momentum effect, where investments show persistent relative results for a short time. Academics explain this effect by analyzing how investors react to new information and move collectively, called “bandwagon effects.”

Naturally, all quantitative models are built by humans. So the use of some fundamental analytical insights may be very useful in building better algorithms.

Dealing with regime changes. Quantitative strategies are essentially a set of rules to select securities and to build a portfolio. Consistent with the prior point, rules are context dependent. When mismatches between the rules and the environment occur, the outcomes can be poor.

We have witnessed a handful of such cases in recent years. For instance, in August 2007 a number of quantitative funds had sharp losses. One reconstruction of the events suggested that concerns over the subprime mortgage market spilled over to equities and caused certain factors popular among quants to work poorly.19 Because many of these strategies were enhanced with leverage, liquidation by some funds caused a feedback loop—asset drop, margin call, asset sale, asset drop, margin call, etc.—that generated the stinging losses.

Another example is the “flash crash” in May of 2010, when the Dow Jones Industrials Average plunged six percent in minutes only to largely recover moments later. A seemingly innocuous trade in S&P 500

futures contracts led to sharp moves in a handful of securities as trading algorithms fed off of one another. In both cases, quantitative approaches got caught in runaway feedback loops that distorted the market.

A fundamental investor who is close to the market may be able to see these types of developments, recognize the mismatch between model and market, and either slow things down or shut them down altogether.

Tyler Cowen, the economist who discusses freestyle chess in his recent book, describes playing chess with a program called Shredder against Shredder itself (so it’s Cowen plus Shredder versus Shredder alone). He notes that at a handful of crucial junctures in the game he overrides the strategic judgment of the program, which he reckons adds value in four out of five instances.

More granular information. Most quantitative strategies use factors to build portfolios that seek to generate excess returns, adjusted for risk. These factors include small capitalization versus large capitalization stocks, cheap stocks versus expensive stocks, or low-risk stocks versus high-risk stocks.

The strength of a quantitative model is that it can consider lots of securities. The weakness of the model is that it doesn’t know much about any particular security.

Here is where a fundamental approach can be handy. There may be obvious and logical reasons why a certain stock or industry looks cheap that the quantitative model, which relies on rules, cannot appreciate. A fundamental analyst can frequently see these mistakes easily. One quantitative manager put it this way, “a fundamental view can be of help—for example, if a model suggests to buy shares in a German bank when the fundamental analyst knows that German banks are heavily invested in subprime.”20

Conclusion

There is a battle between qualitative and quantitative approaches in many fields today including investing, sports, business, and politics. The rapid gains in computing power and vast amounts of data have only made the battle line more acute. The central question is whether these camps can work together to be more effective than either of them individually.

The advent of freestyle chess provides hope for such a possibility. As of now, the combination of man and machine is better than either man or machine. The essential human skill in combining the two is not aptitude in the activity itself, as we saw with ZackS, but rather knowing how and when to appeal to the strength of each approach.

An essential question is whether the lessons of freestyle chess apply to investing. Chess is a game of perfect information that is stable and linear. Because of the countless possible states the game is extremely complicated, but it is not a complex adaptive system. Markets are complex adaptive systems, with imperfect information, instability, and non-linearity.

Still, we believe that some of the lessons of freestyle chess are useful. Properly done, a melding of fundamental and quantitative methods may well yield better results than either of them on their own. If we had to say which camp had the most to gain from the other, we’d say that the fundamental analysts have more to learn from the quants than the other way around.

No matter which approach you use, we hope that thinking about the success of freestyle chess will provoke some ideas that may ultimately make you more effective.

Endnotes 1 Garry Kasparov, How Life Imitates Chess: Making the Right Moves—from the Board to the Boardroom (New York: Bloomsbury, 2007), 166.

2 Kasparov, 163.

3 In a chess match, a player gets 1 point for a win and ½ of a point for a draw. So in a six-game match, a 300 point rating edge suggests an expectation of winning about 5.2 of 6.0 possible points.

4 Goethe did not actually say this himself. Rather, Adelheid, a character from Goethe’s play Götz von Berlichingen, says the phrase while in the midst of a chess match.

5 Stephen Baker, Final Jeopardy: Man vs. Machine and the Quest to Know Everything (Boston, MA: Houghton Mifflin Harcourt, 2011). Also John E. Kelly III and Steve Hamm, Smart Machines: IBM’s Watson and the Era of Cognitive Computing (New York: Columbia Business School Publishing, 2013).

6 Tyler Cowen, Average Is Over: Powering America Beyond the Age of the Great Stagnation (New York: Dutton, 2013). For additional discussion of freestyle chess, see Erik Brynjolfsson and Andrew McAfee, The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies (New York: W.W.

Norton & Company, 2014).

7“ PAL/CSS Report from the Dark Horse’s Mouth,” ChessBase.com, June 22, 2005. See http://en.chessbase.com/post/pal-c-report-from-the-dark-horse-s-mouth.

8 Cowen, 80.

9 Cowen, 93.

10 Christopher Chabris and David Goodman, “Chess-Championship Results Show Powerful Role of Computers,” Wall Street Journal, November 22, 2013.

11 Michael J. Mauboussin and Dan Callahan, “Alpha and the Paradox of Skill: Results Reflect Your Skill and the Game You Are Playing,” Credit Suisse Global Financial Strategies, July 15, 2013.

12 Michael J. Mauboussin, More Than You Know: Finding Financial Wisdom in Unconventional Places – Updated and Expanded (New York: Columbia Business School Publishing, 2008), 148.

13 Frank J. Fabozzi, Sergio M. Focardi, and Caroline Jonas, “Challenges in Quantitative Equity Management,” Research Foundation of the CFA Institute, July 2008.

14 Michael J. Mauboussin, Think Twice: Harnessing the Power of Counterintuition (Boston, MA: Harvard Business Press, 2009), 1-16.

15 Michael J. Mauboussin and Dan Callahan, “How to Model Reversion to the Mean: Determining How Fast, and to What Mean, Results Revert,” Credit Suisse Global Financial Strategies, September 17, 2013. 16 Michael J. Mauboussin and Dan Callahan, “Cultivating Your Judgment Skills: A Framework for Improving the Quality of Decisions,” Credit Suisse Global Financial Strategies, May 7, 2013.

17 Tyler Cowen, “What are humans still good for? The turning point in Freestyle chess may be approaching,” Marginal Revolution Blog, November 5, 2013.

18 Viktor Mayer-Schönberger and Kenneth Cukier, Big Data: A Revolution That Will Transform How We Live, Work, and Think (New York: Houghton Mifflin Harcourt, 2013), 7.

19 Amir Khandani and Andrew W. Lo, “What Happened to the Quants in August 2007? Evidence from Factors and Transaction Data,” SSRN Working Paper, October 28, 2008.

20 Fabozzi, Focardi, and Jonas, 27.