三十年：伟大投资者的十大特质反思

GLOBAL FINANCIAL STRATEGIES www.credit-suisse.com

Thirty Years Reflections on the Ten Attributes of Great Investors August 4, 2016

Authors

Michael J. Mauboussin [email protected]

Dan Callahan, CFA [email protected]

Darius Majd [email protected]

Source: iStockphoto.

“Perhaps the single greatest error in the investment business is a failure to distinguish between the knowledge of a company’s fundamentals and the expectations implied by the market price.”

The world of investing and business has seen a great deal of change in the past 30 years.

This report shares thoughts on the ten attributes of great fundamental investors.

Accounting is the language of business and you need to understand it to appreciate economic value and to assess competitive positioning.

Investors face a slew of psychological challenges. Perhaps the most difficult is updating beliefs when new information arrives.

Position sizing and portfolio construction still do not get the attention they warrant.

The substantial shift from active to passive management has profound implications for the investment industry.

Introduction

I started on Wall Street 30 years ago today. It has been a fascinating three decades, which is undoubtedly true of all periods of similar length. Notable events include the stock market crash of 1987, the fall of the Berlin Wall and unwinding of communism, the introduction of the Internet and the ensuing dot-com bubble, and a painful financial crisis and a subsequent recovery.

Exhibit 1 shows a contrast between then and now. The indices for equity markets are roughly 10 times higher than they were in 1986, unadjusted for inflation and dividends, and the yield on the U.S. 10-year Treasury note is one-fifth of what it was. Of the top 10 companies by market capitalization in 1986, only General Electric, AT&T, and Exxon Mobil remain in the group today. Google, Amazon.com, and Facebook were not even dreams. In 1986, Microsoft, Oracle, Adobe, and Sun Microsystems went public.

I have spent roughly two-thirds of my career on the sell-side and a third on the buy-side. Both experiences have been deeply gratifying and incredible learning opportunities. In this report, I offer what I believe to be the ten attributes of a great fundamental investor, as well as some thoughts on where we go from here. Before I do so, I provide some personal background and note the influences on my thinking. Feel free to skip directly to the attributes section on page six.

Exhibit 1: August 4: 1986 and 2016 (based on the closing price the previous business day)

1986 (inflation adjusted) 2016

Exxon Alphabet (Google) General Electric AT&T Exxon Mobil Royal Dutch Amazon.com General Motors DuPont Johnson & Johnson BellSouth Philip Morris Merck AT&T Source: FactSet, U.S. Energy Information Administration, Robert Shiller (see: www.econ.yale.edu/~shiller/data.htm), http://ns1758.ca/winch/winchest.html (cost of storage).

Note: Gold=NYMEX:GC in dollars per ounce; Oil=Cushing, OK WTI spot price in dollars per barrel; CAPE ratio as of July 2016.

Background

My first job was as a member of a training program at Drexel Burnham Lambert. (How I got the job is a funny story, which I recount in The Success Equation.1) Drexel was a hot firm at the time in large part because of Michael Milken. The program was ideal for a novice. We spent the first few months in classroom training and then rotated through more than a dozen departments, including various trading desks, equity research, investment banking, and operations. If you didn’t know who you were professionally, the exposure to each of the firm’s businesses was an excellent way to find out.

We were being trained to become retail brokers, or what we call “financial advisors” today, and we launched in January 1988. I was an abject failure. Granted, the environment was not ideal in the wake of the stock market crash of 1987 and Drexel’s legal woes. But I was not cut out for that work and it was apparent from the start. Fortunately, I learned a great deal at Drexel, not the least of which is what I am bad at.

I was a government major in college so came to Wall Street unencumbered with any business knowledge. (My senior year I did take “Principles of Accounting” for non-business majors at my father’s urging and earned a “C+” out of the generosity of the professor’s heart.) I was comfortable with numbers—my college counselor in high school encouraged me to apply to engineering schools—but had little or no exposure to business and finance. But a liberal arts background did expose me to lots of disciplines and cultivated critical thinking.

I believe my lack of business education was an asset because it encouraged me to ask a lot of questions and to think from first principles. I recall going to an equity research morning call and hearing the utility industry analyst suggest the slow-growing companies under his coverage deserved price-earnings (P/E) multiples in the high teens and the tobacco industry analyst imply that his fast-growing companies should trade at P/E’s in the mid-teens. How does that make sense? I was dropped into a world of rules-of-thumb, old wives’ tales, and intuitions. William James, the famous professor of psychology, suggested that upon entering the world a baby “feels it all as one great blooming, buzzing confusion.”2 That captured it pretty well.

My first breakthrough occurred when a classmate in my training program handed me a copy of Creating Shareholder Value by Alfred Rappaport.3 Reading that book was a professional epiphany. Rappaport made three points that immediately comprised the centerpiece of my thinking. The first is that the ability of accounting numbers to represent economic value is severely limited. Next, he emphasized that competitive strategy analysis and valuation should be joined at the hip. The litmus test of a successful strategy is that it creates value, and you can’t properly value a company without a thoughtful assessment of its competitive position.

The final point is that stock prices reflect a set of expectations for future financial performance. A company’s stock doesn’t generate excess returns solely by the company creating value. The company’s results have to exceed the expectations embedded in the stock market.

With my failure as a broker and with Creating Shareholder Value as my guiding light, I set off to get a job in equity research. My first real opportunity came as a junior analyst supporting two senior analysts, one following food, beverage, and tobacco companies and the other capital goods. I worked hard and read extensively during that period. For example, I read Competitive Strategy and Competitive Advantage by Michael Porter.

Bennett Stewart’s The Quest for Value, as well as McKinsey’s Valuation, came out then as well. The wonderful thing about that time was I could learn ideas on the weekend and apply them on Monday morning.

The opportunity to meld theory and practice was crucial.

The consumer analyst basically told me that I was to be available to support him during business hours but that if I wanted to launch coverage of any new companies—with his name on the top of the report—I was free to work nights and weekends to do so. I leapt at the opportunity, and decided to write about Ralston Purina. I was very familiar with Ralston because it had been covered by one of Drexel’s best analysts. The report I wrote was full-blown Rappaport, with a focus on strategy and economic value and a dismissal of superficial accounting measures.

The senior analyst, cut from a more traditional cloth, didn’t like the report much but didn’t prevent me from publishing it. Shortly thereafter, I received a call from the office of the chairman and chief executive officer, Bill Stiritz, indicating that he appreciated the approach and that he would like to discuss it in more detail. Stiritz, one of the executives featured in Will Thorndike’s excellent book The Outsiders, was considered one of the shrewdest executives in the consumer staples industry. Receiving his imprimatur was a huge boost.4

I landed the job as the senior packaged food analyst at First Boston (which later merged with Credit Suisse) not too long after that. After a few months, one of our technology analysts, Charlie Wolf, knocked on my door and asked if we could discuss brands. Charlie, beloved by all who knew him, was a tenured professor at Columbia Business School who took a sabbatical year on Wall Street in the early 1980s and never went back.

We chatted and I shared some of my research.

The next day he returned to my office and suggested that I teach at Columbia Business School. Columbia and First Boston had a close association, and a number of the analysts had taught as adjunct professors. For most, it was a course or two and they were done. I had guest lectured for another one of our analysts, and agreed to teach Security Analysis in the summer of 1993.

I have now taught that course for 24 years in a row (in 1995 I started teaching in the spring term). That experience has been a deep influence. When people ask me about what it is like to teach, I suggest they think about what it would be like to deliver 20 hours of lectures on what they do all day. At first you might think that is a pretty easy task, until you realize that articulating what you do forces you to think about what you do. As a natural consequence, you are likely to question whether there are better ways to do what you do.

Teaching imposes a discipline of understanding and communication that few other activities can—save perhaps writing. Inspirational teachers are also diligent students. So a commitment to teaching at a high level demands constant learning and consolidation of knowledge. There is the additional benefit of being around young people who are bright and challenging.

In 1996, I was sitting at a Baltimore Orioles baseball game with a hot dog in one hand and a beer in the other.

My host was the well-known money manager, Bill Miller. That night, he suggested I make a trek to New Mexico to visit the Santa Fe Institute. I had been reading widely, especially at the intersection of economics and biology, so I was primed for his message.

The Santa Fe Institute (SFI) was founded in 1984 by a handful of eminent scientists, including a number of winners of the Nobel Prize, who believed that many of the most vexing problems in the world lie at the intersection of disciplines. Universities, on the other hand, are largely structured in silos based on academic discipline. That means that the biologists talk to the biologists, the physicists to the physicists, and the economists to the economists. But there’s little discussion across boundaries. At SFI, researchers from all disciplines work together to tackle problems. One of the early seminars brought together economists, physicists, biologists, and others to discuss the economy as an evolving complex system.

I followed Bill’s advice and went to a meeting in the fall of 1996 and was immediately enthralled. The people drawn to the institute are naturally intellectually curious and expansive. And given that many of the institute’s founders came from the hard sciences, there is an insistence on rigor. I have learned many lessons from the researchers at SFI, including those about markets as complex adaptive systems, network theory, increasing returns, power laws, and diversity. The exposure to substantive and non-mainstream work has deeply animated my thinking.

For the last 20 years or so, I have spent most of my time thinking about investment process. This includes topics such as market efficiency (where and how pockets of inefficiency manifest), valuation, competitive strategy analysis, and decision making. What we know about each of these areas today is substantially greater than what we did in 1986, and yet we have an enormous amount to learn. As I like to tell my students, this is an exciting time to be an investor because much of what we teach in business schools is a work-in-progress.

As a strategist and a student of investing, I have had the opportunity to meet or study many great investors. I will now share the attributes that I believe distinguish them from the rest. The reason for the background is to reveal my bias. I believe in economic value, the importance of constant learning and teaching, and that diverse input combined with rigor can lead to insight. This is all very consistent with what Charlie Munger, the vice chairman of Berkshire Hathaway, calls the “mental models” approach.5

Ten Attributes of Great Fundamental Investors

Here are the top ten attributes that I believe great fundamental investors share:

1. Be numerate (and understand accounting). To be a successful investor, you have to be comfortable with numbers. There are rarely complicated calculations but a feel for figures, percentages, and probabilities is essential.

One of the main ways numeracy comes up is in financial statement analysis. Accounting is one of the main languages of business. Great investors are adept at financial statement analysis, which allows them to understand how a business has done in the past and gives some sense of how it will do in the future.

There are a handful of excellent books on accounting that are worth reading carefully.6 The goals of financial statement analysis are twofold. The first is to translate financial statements into free cash flow, the lifeblood of corporate value.

You calculate free cash flow by starting with cash earnings and subtracting the investments a company makes to generate future earnings. Investments include increases in working capital, capital expenditures above and beyond what is needed to maintain plant and equipment, and acquisitions. Free cash flow is what is left over after investments have been subtracted from cash earnings. It is also the sum that a company can return to its claimholders in the form of interest payments, dividends, and share buybacks, without jeopardizing a company’s value creation prospects.

Earnings are the most widely used metric of corporate performance. But it is easy to show that growth in earnings and growth in value are distinct. Companies can increase earnings and simultaneously destroy value if the investments the company makes don’t earn an appropriate rate of return. By focusing on the present value of future free cash flow, a thoughtful investor translates financial statements into what determines value.

The second goal of financial statement analysis is to make a link between a company’s strategy and how it creates value. One simple way to do this is to compare, line by line, two companies that are in the same business. Are their expenses comparable? Do they use capital in a similar way? Noticing differences in how companies spend money and allocate investment resources offers insight into their competitive positions.

You can do a simpler analysis just by looking at the path to return on invested capital (ROIC). You can break ROIC into two components: profitability (net operating profit after tax/sales) and capital velocity (sales/invested capital). Companies with high operating profit margins and low capital velocity are generally pursuing what Michael Porter, a professor of economics, calls a “differentiation” strategy.

Companies with low operating profit margins and high capital velocity are following a “cost leadership” strategy.7 The analysis of how a company makes money spills directly into an assessment of how long the company can sustain its advantage (if ROICs are attractive) or what the company has to do to improve its economic profits.

Understanding a business requires understanding the numbers. The task has become more challenging in recent years as companies are investing more in intangible assets and less in tangible assets. For example, in fiscal 2016 Microsoft spent about one-and-a-half times as much on research and development, which it expensed on the income statement, as it did on capital expenditures, which were capitalized on the

balance sheet. This shift means that traditional accounting measures are less useful but does not abdicate the responsibility to understand a business’s current economics and prospects.8

2. Understand value (the present value of free cash flow). The landscape of investing has changed a great deal in the past three decades. It is interesting to consider what about investing is mutable and what is immutable. The truth is that much is mutable. The average half-life of a public company is about a decade, which means that the investable universe is in flux.9 Conditions are always shifting because of unknowns including technological change, consumer preferences, and competition. But one concept that is close to immutable for an investor is that the present value of future free cash flow determines the value of a financial asset. This is true for stocks, bonds, and real estate. Valuation is challenging for equity investors because each driver of value—cash flows, timing, and risk—are based on expectations whereas two of the three drivers are contractual for bond investors.

Great fundamental investors focus on understanding the magnitude and sustainability of free cash flow.

Factors that an investor must consider include where the industry is in its life cycle, a company’s competitive position within its industry, barriers to entry, the economics of the business, and management’s skill at allocating capital.

A corollary to this attribute is that great investors understand the limitations of valuation approaches such as price/earnings and enterprise value/EBITDA multiples.10 Indeed, multiples are not valuation but a shorthand for the valuation process. No thoughtful investor ever forgets that. Shorthands are useful because they save you time, but they also come with blind spots. As Al Rappaport says, “Remember, cash is a fact, profit is an opinion.”11

3. Properly assess strategy (or how a business makes money). This attribute has two dimensions. The first is a fundamental understanding of how a company makes money. The idea is to distill the business to the basic unit of analysis. For example, the basic unit of analysis for a retailer is store economics. How much does it cost to build a store and fill it with inventory? What revenues will it generate? What are the profit margins? Answers to these and other questions should allow an investor to assess the economic profitability of a store, which he or she can then roll up to understand the overall company.

The basic unit of analysis varies by industry. What you need to understand to assess a subscription business, customer lifetime value, is different than a business dependent on research and development such as a biotechnology company. Great investors can explain clearly how a company makes money, have a grasp on the changes in the drivers of profitability, and never own the stock of a company if they do not understand how it makes money. You can think of this as the micro dimension of understanding strategy.

The second dimension is gaining a grasp of a company’s sustainable competitive advantage. A company has a competitive advantage when it earns a return on investment above the opportunity cost of capital and earns a higher return than its competitors. The classic approach is to analyze the industry and how the company fits in, and the common tools include the five forces that shape industry attractiveness, value chain analysis, assessment of the threat from disruptive innovation, and firm-specific sources of advantage.12

The key to strategy is to understand trade-offs. Michael Porter makes a distinction between strategy and operational effectiveness that executives and investors commonly blur. Strategy is about deliberately being different than competitors and requires making tough choices about what activities to do and not do.

Operational effectiveness relates to the activities that all businesses need to do and hence does not entail choice.

Great investors can appreciate what differentiates a company that allows it to build an economic moat around its franchise that protects the business from competitors. The size and longevity of the moat are significant inputs into any thoughtful valuation. As Al Rappaport emphasized years ago, strategy and valuation need to go together. This is the macro dimension of understanding strategy.

4. Compare effectively (expectations versus fundamentals). Comparing is a critical element of investing. Investors compare all day: stocks versus bonds, active versus passive, value versus growth, stock A versus stock B, and now versus later. Humans are quick to compare but not very good at it.

Perhaps the most important comparison an investor must make, and one that distinguishes average from great investors, is between fundamentals and expectations. Fundamentals capture a sense of a company’s future financial performance. Value drivers including sales growth, operating profit margins, investment needs, and return on investment shape fundamentals. Expectations reflect the financial performance implied by the stock price.

Making money in markets requires having a point of view that is different than what the current price suggests. Michael Steinhardt called this a “variant perception.”13 Most investors fail to distinguish between fundamentals and expectations. When fundamentals are good they want to buy and when they are poor they want to sell. But great investors always distinguish between the two.

One vivid analogy is pari-mutuel betting. Horse racing is a good example. The amount bet on a horse gets reflected in the horse’s odds, or probability, of winning the race. The goal is not to figure out which horse will win but rather which horse has odds that are mispriced relative to how it will likely run the race.14 Fundamentals are how fast the horse will run, and expectations are the odds. You need to consider those elements separately.

One of the basic challenges in comparing well is a concept that psychologists call “coherent arbitrariness,” which says that we struggle to understand the absolute value of a good but are effective at understanding the relative value.15 You may not know the proper value of a pharmaceutical company, but you can rank them in order of your preference. Unless you solely engage in arbitrage, absolute values ultimately matter.

Humans tend to think by analogy, which can create some cognitive trouble. One issue is that a single analogy, or even a handful of analogies, may fail to reflect a full reference class of relevant cases. For example, rather than asking whether this turnaround is similar to a prior turnaround, it is useful to ask for the base rate of success for all turnarounds. Psychologists have shown that properly integrating the outcomes from an appropriate reference class improves the quality of forecasts.

Another challenge with using analogies is that we see similarities when we focus on similarities and see differences when we focus on differences. The emphasis of the comparison colors the outcome. For example, Amos Tversky, a psychologist known for his collaboration with Daniel Kahneman, asked subjects which pair of countries they deemed more similar, West Germany and East Germany or Nepal and Ceylon (the study was done in the early 1970s and Ceylon changed its name to Sri Lanka in 1972). Two-thirds of the subjects selected West Germany and East Germany.

Tversky then asked subjects which pair of countries they deemed more different. Logic suggests an answer that is the complement of the first response, hence two-thirds finding Nepal and Ceylon more different. But that’s not what Tversky found. Seventy percent of the subjects rated West Germany and East Germany more different than the other pair. What you are looking for dictates what you see.16

A final challenge is considering causality from the point of view of attributes versus circumstances.

Attributes are features that allow for categorization. For instance, animals with wings and feathers can fly.

Circumstances capture causal mechanisms. Since the physics of lift causes flight, animals or objects that can create lift will fly, including most birds and airplanes, and those that can’t create lift won’t fly. To learn from history, you need to understand causality.17 We commonly limit our comparisons to attributes and hence miss essential insights. Great investors compare well without falling for the common traps.

5. Think probabilistically (there are few sure things). Investing is an activity where you must constantly consider the probabilities of various outcomes. This requires a certain mindset. To begin, you must constantly seek an edge, where the price for an asset misrepresents either the probabilities or the outcomes. Successful operators in all probabilistic fields dwell on finding edge, from the general managers of sports franchises to professional bettors.

When probability plays a large role in outcomes, it makes sense to focus on the process of making decisions rather than the outcome alone. The reason is that a particular outcome may not be indicative of the quality of the decision. Good decisions sometimes result in bad outcomes and bad decisions lead to good outcomes. Over the long haul, however, good decisions portend favorable outcomes even if you will be wrong from time to time. Time horizon and sample size are also vital considerations. Learning to focus on process and accept the periodic and inevitable bad outcomes is crucial.

Great investors recognize another uncomfortable reality about probability: the frequency of correctness does not really matter (batting average), what matters is how much money you make when you are right versus how much money you lose when you are wrong (slugging percentage). This concept is very difficult to put into operation because of loss aversion, the idea that we suffer losses roughly twice as much as we enjoy comparably sized gains. In other words, we like to be right a lot more than to be wrong. But if the goal is grow the value of a portfolio, slugging percentage is what matters.

There are three ways of coming up with probabilities. The first is subjective probability, a figure that corresponds with a state of knowledge or belief. For instance, you might assign a subjective probability to the likelihood that two countries go to war. Second is propensity, which is generally based on physical properties of the system. For example, you would estimate the probability of a die coming up four in a single roll as one out of six based on the fact that the die is a perfect cube and the toss is without bias. The final approach is frequency, which considers the outcomes of a proper reference class given the situation under deliberation.

You are well served to use the frequency approach to anticipate measures of corporate performance such as sales and profit growth rates. Subjective probabilities, which require frequent updating, can also be very useful. But there are realms where attaching probabilities to outcomes can be treacherous.18 Here, both the probabilities and outcomes are largely unknowable. This is the territory of “black swans,” outcomes that are “outside the realm of expectations,” have a large impact, and are explained after the fact. If you are going to participate in this area, the goal here is to gain exposure to positive black swans.

Warren Buffett, chairman and chief executive officer of Berkshire Hathaway, summed up this attribute well: “Take the probability of loss times the amount of possible loss from the probability of gain times the amount of possible gain. That is what we’re trying to do. It’s imperfect, but that’s what it’s all about.”19

6. Update your views effectively (beliefs are hypotheses to be tested, not treasures to be protected).20 Most people prefer to maintain consistent beliefs over time, even when the facts reveal their beliefs to be wrong. Further, we commonly expect others to be consistent. For example, politicians who

change their views are derisively called “flip-floppers.”21 The need for consistency tends to grow with age. The idea that many older people are set in their ways is grounded in truth.22

We all walk around with views of the world that we believe are correct. You are compelled to change your mind only when you confront reality that disconfirms your beliefs. The easiest way to avoid the sensation of being wrong is to fall for the confirmation bias. With confirmation bias, you seek information that confirms your view and interpret ambiguous information in a way that is favorable to your belief.

Consistency allows you to stop thinking about an issue and to avoid change as a consequence of reason.

But great investors do two things that most of us do not. They seek information or views that are different than their own and they update their beliefs when the evidence suggests they should. Neither task is easy.

The trait of seeking alternative views is called being “actively open-minded,” a term coined by a professor of psychology at the University of Pennsylvania named Jonathan Baron. Actively open-minded is defined as “the willingness to search actively for evidence against one’s favored beliefs, plans or goals and to weigh such evidence fairly when it is available.”23 Research shows that actively open-minded people perform well in forecasting tasks that demand collecting information. The trait of being actively-open minded can offset confirmation bias.24

Great investors also update their views as new information arrives. The idea is that you can represent your degree of belief about something by a probability. When new information arrives, you update that probability. The formal way to do this is to use Bayes’s Theorem, which tells you the probability that a theory or belief is true conditional on some event happening.25

The practical challenges are to avoid overreacting to information that appears to explain causality on the surface but in fact does not, as well as to detect information that does matter but that does not appear to be causal. In addition, while it is often clear to shift your degree of belief up or down, the magnitude of the shift is also important.

The best investors among us recognize that the world changes constantly and that all of the views that we hold are tenuous. They actively seek varied points of view and update their beliefs as new information dictates. The consequence of updated views can be action: changing a portfolio stance or weightings within a portfolio. Others, including your clients, may view this mental flexibility as unsettling. But good thinking requires maintaining as accurate a view of the world as possible.

7. Beware of behavioral biases (minimizing constraints to good thinking). Keith Stanovich, a professor of psychology, likes to distinguish between intelligence quotient (IQ), which measures mental skills that are real and helpful in cognitive tasks, and rationality quotient (RQ), the ability to make good decisions. His claim is that the overlap between these abilities is much lower than most people think.

Importantly, you can cultivate your RQ.

Warren Buffett captures this idea when he distinguishes between an engine’s horsepower (IQ) and output (RQ):

“I always look at IQ and talent as representing the horsepower of the motor, but that the output—the efficiency with which that motor works—depends on rationality. A lot of people start out with 400- horsepower motors but only get a hundred horsepower of output. It’s way better to have a 200- horsepower motor and get it all into output.”26

Two areas of research, heuristics and biases and prospect theory, merged economics and psychology in the last half century. While many of the ideas had been around for a long time, Daniel Kahneman and Amos Tversky did a great deal to formalize the thinking.

The heuristics and biases literature notes that we tend to operate with rules of thumb (heuristics), which are generally correct and save us lots of time. But these heuristics have associated biases that can lead to departures from logic or probability. Examples of heuristics include availability (rely on information that is available rather than relevant), representativeness (placing people or objects in categories that are inaccurate), and anchoring (placing too much weight on an anchor figure). There is now a long list of heuristics and biases, and great investors are those who not only understand these concepts but take steps to manage or mitigate behavioral biases in their investment process.

Prospect theory shows how the decisions by individuals depart from normative economic approaches in risky situations. Loss aversion, which says that individuals tend to suffer more from losses than comparable gains, is a good example. While there may be a persuasive evolutionary explanation for loss aversion, it is not good for money management.27

Epistemic rationality, or the degree to which your beliefs map accurately to the world, is an essential ingredient to RQ. People who score well on RQ are well calibrated, which means that the probabilities they assign to particular outcomes tend to be accurate over a large sample of judgments. One way to improve calibration is to keep score. You can track your forecasts and grade them based on outcomes.

The ability to sidestep behavioral biases is likely part disposition, part training, and part environment. Great investors are those who are generally less affected by cognitive bias than the general population, learn about biases and how to cope with them, and put themselves in a work environment that allows them to think well.

8. Know the difference between information and influence. In classic markets for goods or services, prices are a highly informative mechanism. In microeconomics, the equilibrium price is one that balances supply and demand. A higher price creates more supply than demand, and hence excess supply. A lower price creates more demand than supply, and hence excess demand. The equilibrium price is also known as the market clearing price because it clears away excess supply or demand. Supply and demand curves may shift, but price is a very useful source of information about the market.

Prices also provide useful information in capital markets. The main value is as an indication about expectations for future financial performance. (This is less true for derivatives, which may be efficiently priced even if the asset from which its price is derived is inefficiently priced.) As we saw in a prior point, great investors are adept at translating between expectations and fundamentals, and keep them separate in decision making. Further, arbitrageurs serve to close aberrant price gaps. Arbitrage is negative feedback, pushing prices that have drifted from fair value back toward their correct level.

Yet investing is an inherently social exercise. As a result, prices can go from being a source of information to a source of influence. This has happened many times in the history of markets. Take the dot-com boom as an example. As internet stocks rose, investors who owned the shares got rich on paper. This exerted influence on those who did not own the shares and many of them ended up suspending belief and buying as well. This fed the process. The rapid rise of the dot-com sector was less about grounded expectations about how the Internet would change business and more about getting on board. Negative feedback ceded to positive feedback, which pushes a system away from its prior state.

One of the best models for thinking about this type of behavior is the threshold model from Mark Granovetter, a professor of sociology at Stanford University.28 Imagine 100 potential rioters milling around in a public square. Each individual has a “riot threshold,” the number of rioters that person would have to see in order to join the riot. Say one person has a threshold of 0 (the instigator), one has a threshold of 1, one has a threshold of 2, and so on up to 99. This uniform distribution of thresholds creates a domino effect and ensures that a riot will happen. The instigator breaks a window with a rock, person one joins in, and then each individual piles on once the size of the riot reaches his or her threshold. Substitute “buy dot-com stocks” for “join the riot” and you get the idea.

The point is that very few of the individuals, save the instigator, think that rioting is a good idea. Most would probably shun rioting. But once the number of others rioting reaches a threshold, they will jump in.

This is how the informational value of stocks is set aside and the influential component takes over.

Great investors don’t get sucked into the vortex of influence. This requires the trait of not caring what others think of you, which is not natural for humans. Indeed, many successful investors have a skill that is very valuable in investing but not so valuable in life: a blatant disregard for the views of others. Success entails considering various points of view but ultimately shaping a thesis that is thoughtful and away from the consensus. The crowd is often right, but when it is wrong you need the psychological fortitude to go against the grain. This is much easier said than done, especially if it entails career risk (which is often the case).

9. Position sizing (maximizing the payoff from edge). Puggy Pearson was a cigar-chomping gambling legend who won the World Series of Poker and was one of the world’s best pool players. When asked about his success, Pearson said, “Ain’t only three things to gambling: Knowin’ the 60-40 end of a proposition, money management, and knowin’ yourself.”29 Great investors take to heart all three of Pearson’s points, but money management is the one that gets the least attention in the discourse on investment practice.

The book Bringing Down the House by Ben Mezrich tells the story of a half dozen students from MIT who deployed a card counting system to make lots of money in Las Vegas. Their system had two parts. The first was the method for counting cards. Here, members of the team fanned out to different tables and developed a signal to indicate when the odds looked good. But the second part of the system is commonly overlooked. The team members knew exactly how much to bet given the odds at the table and the size of their bankroll.

Similarly, success in investing has two parts: finding edge and fully taking advantage of it through proper position sizing. Almost all investment firms focus on edge, while position sizing generally gets much less attention.

Proper portfolio construction requires specifying a goal (maximize sum for one period or parlayed bets), identifying an opportunity set (lots of small edge or lumpy but large edge), and considering constraints (liquidity, drawdowns, leverage). Answers to these questions suggest an appropriate policy regarding position sizing and portfolio construction.

The most common approaches are based on mean-variance (maximize return for a given level of risk) and the Kelly Criterion (maximize a portfolio’s geometric mean return).30 Which approach makes most sense for you depends a great deal on how you answer the questions about goals, opportunities, and constraints. But the broad point is that most investors do a poor job with position sizing and great investors are more effective at it.31

Ed Thorp is one of the best investors of the last half century. In the early 1960s, he wrote the book, Beat the Dealer, which explained card counting in blackjack as a means to gain edge as well as a betting strategy based on the Kelly Criterion in order to take advantage of that edge. The MIT team was his intellectual progeny.

His returns as an investor are extraordinary. For the nearly 30 years ended in the late 1990s, Thorp’s investments had grown at a 20 percent annual rate with a 6 percent standard deviation.32 Thorp has been among the most thoughtful and thorough in explaining the benefits, drawbacks, and practical considerations of the Kelly Criterion.33

Astute investors understand that finding edge and betting on it appropriately are both essential to long-term success.

10. Read (and keep an open mind). Every year, Columbia Business School sends a group of students to Omaha, Nebraska to meet with Todd Combs, a graduate of the school (and former student), and Warren Buffett (also a graduate), at Berkshire Hathaway. After one of those trips, I asked the students for their impression of the meetings. The students, in somewhat of a state of disbelief, said that Combs suggested that they read 500 pages a day. In a world of endless meetings, blinking Bloomberg terminals, and demanding emails, this goal seems inconceivable.

Berkshire Hathaway’s Charlie Munger said that he really liked Albert Einstein’s point that “success comes from curiosity, concentration, perseverance and self-criticism. And by self-criticism, he meant the ability to change his mind so that he destroyed his own best-loved ideas.”34 Reading is an activity that tends to foster all of those qualities.

Munger has also said, “In my whole life, I have known no wise people (over a broad subject matter area) who didn’t read all the time–none, zero.”35 This may be hyperbolic, but seems to be true in the investment world as well.

Great investors generally practice a few habits with regard to their reading. First, they allocate time to it. Warren Buffett has suggested that he dedicates 80 percent of his working day to reading.36 Note that if you are spending time reading, you are not doing something else. There are trade-offs. But many successful people are willing to make reading a high priority.

Second, good readers tend to take on material across a wide spectrum of disciplines. Don’t just read in business or finance. Expand the scope into new domains or fields. Follow your curiosity. It is hard to know when an idea from an apparently disparate field may come in handy.

Finally, make a point of reading material you do not necessarily agree with. Find a thoughtful person who holds a view different than yours, and then read his or her case carefully. This contributes to being actively open-minded.

Research shows that successful people read a lot and do so more for education than for entertainment.37 Reading is their primary means to continue their education. This habit is particularly important for investors, who must synthesize a huge number of inputs into actionable ideas.

What Next?

It is harder than ever to generate excess returns in the investment management business. The main explanation for the rising difficulty is an idea we call the “paradox of skill.” The paradox says that in some activities, as skill increases luck becomes more important in determining outcomes. The key insight is differentiating between absolute and relative skill.

Absolute skill in investing, which includes bright investors working hard using vast amounts of data and incorporating ideas from the latest research, has never been higher. Relative skill, on the other hand, appears to be narrowing. The difference between the best and the average is less today than it was a generation or two ago. We see this clearly when we examine the standard deviation of excess returns for mutual funds, which has declined steadily for a half century.

The massive shift in asset allocation away from active investing toward passive investing exacerbates this effect. Thirty years ago, index funds were less than one percent of assets under management, and today they (along with other passive vehicles such as exchange-traded funds) are about one-third. Think of it this way: For you to have positive alpha, the industry’s term for risk-adjusted excess return, someone has to have negative alpha of the same amount. By definition, alpha for the market must equal zero (before fees).

So you want to compete against less-skilled investors because they are your source of alpha. It is disadvantageous for you if the weak players flee the market (selling their stocks and buying index funds), or if the least capable professional investors lose assets to passive funds, because it means that only the smartest investors remain in the active game. The truth is that weak players, whom the strong players require to generate excess returns, are fleeing at a record pace.

Still, here are a couple of suggestions for active managers to contemplate.

The first is to consider carefully where your skill will be most valuable. More than 25 years ago, Richard Grinold spelled out the “fundamental law of active management.”38 The non-technical interpretation says that excess returns equal skill times opportunity. All the skill in the world is for naught unless you have an opportunity to apply it. Before figuring out how you will win the game, figure out which game to play.

Second, there is a still a lot of upside in properly building an investment firm. In the past 30 years, we have learned a substantial amount about how our minds work, the value of diversity, and the role of training.

Take a look at exhibit 2, which considers the factors that contribute to peak performance as an investor. The box on the left starts with selecting the right people to be analysts and portfolio managers. The next pair of boxes place emphasis on proper training through deliberate practice and the thoughtful application of technology. Then the organization has to manage cognitive bias. Only then can an individual in a firm reach peak performance.

Exhibit 2: Achieving Peak Performance in an Investment Organization

Deliberate Practice Managing Peak Selection Mental Bias Performance Technology/ Innovation

Source: Credit Suisse.

Here are some thoughts on each:

Selection. Both Keith Stanovich’s work on IQ and RQ and Phil Tetlock’s work on superforecasting strongly suggest that there’s more to good decision making than traditional smarts measured through IQ.

Qualities such as active-open mindedness, thinking probabilistically, an ability to update views frequently and accurately, and persistence are not well captured in tests. Even proxies for intelligence, such as graduating from an elite university, do not do the job.39

Stanovich, along with some colleagues, will publish a book on rational thinking in September 2016 that includes the CART (Comprehensive Assessment of Rational Thinking), a test analogous to the IQ test.40 The cognitive reflection test (CRT) and extended CRT are simple tests that likely capture the same dimensions of thinking.41

Progressive investment organizations will extend their selection process to including screening for rationality, which will likely improve results.

Deliberate practice. The next step toward peak performance is the application of deliberate practice, which has a number of elements.42 Done properly, deliberate practice is designed with a specific objective, requires timely and accurate feedback from a coach or teacher, entails substantial repetition, and is only viable for students who are motivated. Deliberate practice is not fun, but it is gratifying because the performance improvements are tangible.

There is no straightforward way to map the elements of deliberate practice to the world of investing.

Designed practice with a specific objective may involve exercises in probability judgments, including learning how to establish prior beliefs, how to update views, and how to distinguish between fundamentals and expectations.

Feedback is a challenge. Your first inclination is to use the market as a source of feedback, but the problem is that market moves, especially in the short term, are very noisy. A better approach is to break down an investment thesis in components, each of which you can measure and track. This allows you to keep score and provide feedback. The Brier score is an established way to assess the accuracy of probabilistic forecasts.43

The way to achieve repetition is to constantly work on similar types of problems that may not be directly related to the portfolio. Handicappers talk of “action bets” and “prime bets.” Action bets are small bets that serve to keep the handicapper engaged and sharp. The stakes are low but the handicapper gets feedback to help his bigger bets. Prime bets are the big bets. If action bets are the warm up, prime bets are the main event. In order to gain repetition, the investor should think about everything relevant in probability terms and sharpen his or her ability to do so.

Motivation in cognitive tasks shows up as active open-mindedness, a need for cognition, and competitiveness. When the best forecasters in a tournament were asked why they participated in the first place, they cited “wanting to be among the top forecasters” as the primary reason. Investing entails ongoing learning, so motivation to learn and stay involved is essential. Reading is probably the best indication of this motivation.

Technology and innovation. Here is a provocative thought: if you compete against a computer, you are highly likely to lose. If you compete with a computer to augment your performance, you are more likely to

win. The question is how this plays out in the investment business.

One model we can examine is chess. Machine beat man when Garry Kasparov lost to Deep Blue in 1997. Since then, a new type of play called freestyle chess has become more popular. In freestyle chess, humans make the moves but can avail themselves of whatever aids they want, including chess programs and other humans. While it is close, it appears that freestyle teams are the best chess players in the world, easily surpassing humans by themselves and narrowly beating the best chess programs.44

You need to sort what the computer is good at and what you can do. Then you have to let the computer do its thing while you do yours. Computers are good at applying base rates, crunching numbers, and casting a wide net. Humans can add value by understanding causality in a more nuanced way, recognizing regime changes, and applying more granular knowledge.

It is hard to be at the cutting edge of innovation, but there is a great deal to gain from simply applying available ideas or principles. For example, it is still surprising how few fundamental investors use simulation in their work. You can buy a Monte Carlo simulator off the shelf for a reasonable price, and it will generate substantial insight. Is your organization set up in the best possible way? For example, are your teams the proper size, of the ideal composition, and managed well? Finally, fundamental firms should keep an eye on academic research to see if any of the ideas apply.

Managing mental bias. At this point, most investors are familiar with the basic ideas from the literature on heuristics and biases and prospect theory. (If you are not, you should learn about the ideas right away). The challenge today is to implement methods to manage and mitigate behavioral biases.

There are a few techniques that can be helpful. One example is the use of checklists, which come in two forms. A DO-CONFIRM checklist allows an investor to do his or her job as they see fit but prompts pauses to make sure the job has been done thoroughly. Our experience is that checklists can be effective for the routinized aspects of investing, for instance making sure that the analyst has done a competitive strategy or return on invested capital analysis properly. A READ-DO checklist is useful in periods of high stress. One case is when a stock you own drops sharply versus the market. A checklist can lay out the protocol to ensure the best possible decision.45

Another technique is an explicit effort to improve choice architecture. Research in psychology demonstrates that how choices are presented influences behavior. Nudges are structures that encourage positive behaviors without limiting options.

Having an organization with a high level of cognitive diversity, people with different training experience, personalities, and skills, is perhaps the most powerful antidote to bias. But cultivating cognitive diversity is not enough. You must manage diversity. “Psychological safety” is the single factor that best predicts good performance for a team. This means that people feel free to voice their views and are willing to take appropriate risks.46

Some Influential Books along the Journey

1980s Michael E. Porter, Competitive Strategy: Techniques for Analyzing Industries and Competitors (New York: Free Press, 1980).

Robert Cialdini, Influence: The Psychology of Persuasion (New York: William Morrow, 1984).

Michael E. Porter, Competitive Advantage: Creating and Sustaining Superior Performance (New York: Free Press, 1985).

Alfred Rappaport, Creating Shareholder Value: The New Standard for Business Performance (New York: Free Press, 1986).

Richard Dawkins, The Blind Watchmaker (New York: W.W. Norton & Company, 1986).

Max H. Bazerman, Judgment in Managerial Decision Making (New York: John Wiley & Sons, 1986).

1990s Michael Rothschild, Bionomics: Economy as Ecosystem (New York: Henry Holt, 1990).

Tom Copeland, Tim Koller, and Jack Murrin, Valuation: Measuring and Managing the Value of Companies (New York: John Wiley & Sons, 1990).

G. Bennett Stewart III, The Quest for Value: A Guide for Senior Managers (New York: Harper Business, 1991).

Peter Bernstein, Capital Ideas: The Improbable Origins of Modern Wall Street (New York: John Wiley & Sons, 1992).

M. Mitchel Waldrop, Complexity: The Emerging Science at the Edge of Order and Chaos (New York: Simon & Schuster, 1992).

Kevin Kelly, Out of Control: The Rise of Neo-Biological Civilization (New York: Addison Wesley, 1994).

Daniel C. Dennett, Darwin’s Dangerous Idea: Evolution and the Meanings of Life (New York: Simon & Schuster, 1995).

Peter Bernstein, Against the Gods: The Remarkable Story of Risk (New York: John Wiley & Sons, 1996).

Steven Pinker, How the Mind Works (New York: W.W. Norton & Company, 1997).

E.O. Wilson, Consilience: The Unity of Knowledge (New York, Alfred A. Knopf, 1998).

Carl Shapiro and Hal R. Varian, Information Rules: A Strategic Guide to the Network Economy (Boston, MA: Harvard Business School Press, 1999).

2000s Michael Lewis, Moneyball: The Art of Winning an Unfair Game (New York: W.W. Norton, 2003).

James Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (New York: Random House, 2004).

Eric D. Beinhocker, The Origin of Wealth: Evolution, Complexity, and the Radical Remaking of Economics (Boston: Harvard Business School Press, 2006).

Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011).

Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (New York: Crown Publishers, 2015).

Endnotes 1 Michael J. Mauboussin, The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing (Boston, MA: Harvard Business Review Press, 2012), 1-2.

2 William James, The Principles of Psychology, Volume 1 (New York: Henry HOLT and Company, 1890), 488. 3 Alfred Rappaport, Creating Shareholder Value: The New Standard for Business Performance (New York: Free Press, 1986).

4 William N. Thorndike, Jr., The Outsiders: Eight Unconventional CEOs and Their Radically Rational Blueprint for Success (Boston, MA: Harvard Business Review Press, 2012).

5 Peter D. Kaufman, ed., Poor Charlie’s Almanack: The Wit and Wisdom of Charles T. Munger (Virginia Beach,VA: PCA Publication, 2005).

6 Some of my favorites include Howard M. Schilit and Jeremy Perler, Financial Shenanigans: How to Detect Accounting Gimmicks & Fraud in Financial Reports, 3rd Edition (New York: McGraw Hill, 2010); Charles W.

Mulford and Eugene E. Comiskey, Creative Cash Flow Reporting: Uncovering Sustainable Financial Performance (Hoboken, NJ: John Wiley & Sons, 2005); Charles W. Mulford and Eugene E. Comiskey, The Financial Numbers Game: Detecting Creative Accounting Practices (Hoboken, NJ: John Wiley & Sons, 2005); and Kathryn F. Staley, The Art of Short Selling (New York: John Wiley & Sons, 1997).

7 Michael E. Porter, Competitive Strategy: Techniques for Analyzing Industries and Competitors (New York: Free Press, 1980).

8 Baruch Lev and Feng Gu, The End of Accounting and the Path Forward for Investors and Managers (Hoboken, NJ: John Wiley & Sons, 2016).

9 Madeleine I. G. Daepp, Marcus J. Hamilton, Geoffrey B. West, and Luís M. A. Bettencourt, “The Mortality of Companies,” Royal Society Interface, Vol. 12, No. 106, May 6, 2015; Michael J. Mauboussin and Dan Callahan, “Why Corporate Longevity Matters: What Index Turnover Tells Us about Corporate Results,” Credit Suisse Global Financial Strategies, April 16, 2014.

10 Stanley Block, “Methods of Valuation: Myths vs. Reality,” The Journal of Investing, Winter 2010, 7-14.

EBITDA stands for earnings before interest, taxes, depreciation, and amortization.

11 Alfred Rappaport, Creating Shareholder Value: A Guide for Managers and Investors (New York: Free Press, 1998), 15.

12 Michael J. Mauboussin and Dan Callahan, “Measuring the Moat: Assessing the Magnitude and Sustainability of Value Creation,” Credit Suisse Global Financial Strategies, July 22, 2013.

13 Michael Steinhardt, No Bull: My Life In and Out of Markets (New York: John Wiley & Sons, 2001), 129. 14 Steven Crist, “Crist on Value,” in Beyer, et al., Bet with the Best (New York: Daily Racing Form Press, 2001).

15 Dan Ariely, George Loewenstein, and Drazen Prelec, “‘Coherent Arbitrariness’: Stable Demand Curves Without Stable Preferences,” Quarterly Journal of Economics, Vol. 118, No 1, February 2003, 73-106. 16 Amos Tversky, “Features of Similarity,” Psychological Review, Vol. 84, 1977, 327-352. Reprinted in Eldar Shafir, ed., Preference, Belief, and Similarity: Selected Writings, Amos Tversky (Cambridge, MA: MIT Press, 2004).

17 Paul R. Carlile and Clayton M. Christensen, “The Cycles of Theory Building in Management Research,”

Harvard Business School Working Paper Series, No. 05-057, 2005. Also, see the chapter, “History, The Fickle Teacher,” in Duncan J. Watts, Everything Is Obvious*: *Once You Know the Answer (New York: Crown Business, 2011), 108-134.

18 Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable, Second Edition (New York: Random House, 2010), 365.

19 Warren E. Buffett, Berkshire Hathaway Annual Meeting, 1989.

20 The phrase “beliefs are hypotheses to be tested, not treasures to be protected” comes from Philip E.

Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (New York: Crown Publishers, 2015), 191.

21 Robert B. Cialdini, Influence: The Psychology of Persuasion (New York: William Morrow, 1993). 22 Stephanie L. Brown, Terrilee Asher, and Robert B. Cialdini, “Evidence of a Positive Relationship Between Age and Preference for Consistency,” Journal of Research in Personality, Vol. 39, No. 5, October 2005, 517-533.

23 Christopher Peterson and Martin E. P. Seligman, Character Strengths and Virtues: A Handbook and Classification (Oxford: Oxford University Press, 2004), 144.

24 Uriel Haran, Ilana Ritov, Barbara A. Mellers, “The Role of Actively Open-Minded Thinking in Information Acquisition, Accuracy, and Calibration,” Judgment and Decision Making, Vol. 8, No 3, May 2013, 188-201. 25 For a layman’s discussion of Bayes’s Theorem, see Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: The Penguin Press, 2012), 243-248. To solve for the new probability, you need three quantities. First, you need a prior probability (x). Second, you need an estimate of the probability as a condition of the hypothesis being true (y). Finally, you need an estimate conditional on the hypothesis being false (z).

Bayes’s Theorem = xy xy + z(1-x)

For an example, see Michael J. Mauboussin and Dan Callahan, “Cultivating Your Judgment Skills: A Framework for Improving the Quality of Decisions,” Credit Suisse Global Financial Strategies, May 7, 2013. 26 Brent Schlender, “The Bill & Warren Show,” Fortune, July 20, 1998.

27 Rose McDermott, James H. Fowler, and Oleg Smirnov, “On the Evolutionary Origin of Prospect Theory Preferences,” Journal of Politics, Vol. 70, No. 2, April 2008, 335–350.

28 Mark Granovetter, “Threshold Models of Collective Behavior,” American Journal of Sociology, Vol. 83, No. 6, May, 1978, 1420-1443.

29 David Spanier, Easy Money: Inside the Gambler’s Mind (New York: Penguin, 1987).

30 Javier Estrada, “Geometric Mean Maximization: An Overlooked Portfolio Approach?” Working Paper, January 2010.

31 Leonard C. MacLean, Edward O. Thorp, Yonggan Zhao, and William T. Ziemba, “How Does the Fortune’s Formula Kelly Capital Growth Model Perform?” Journal of Portfolio Management, Summer 2011, 96-111. 32 William Poundstone, Fortune’s Formula: The Untold Story of the Scientific Betting System that Beat the Casinos and Wall Street (New York: Hill and Wang, 2005), 320.

33 Edward O. Thorp, “The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market,” in Handbook of Asset and Liability Management, Volume 1: Theory and Methodology, S.A. Zenios and W. T. Ziemba, eds., (Amsterdam: North-Holland, 2006), 385-428.

34 “Berkshire Hathaway’s Warren Buffett & Wesco Financial’s Charlie Munger,” Outstanding Investor Digest, Vol. 19, No. 3 &4, December 31, 2004, 52.

35 Berkshire Hathaway Annual Meeting, 2004 per Tren Griffin, Charlie Munger: The Complete Investor (New York: Columbia Business School Press, 2015), 104.

36 Drake Baer, “Nine Books Billionaire Warren Buffett Thinks Everyone Should Read,” Business Insider, September 2, 2014.

37 Libby Kane, “What Rich People Have Next to Their Beds,” Business Insider, June 17, 2014. 38 Richard C. Grinold, “The Fundamental Law of Active Management,” Journal of Portfolio Management, Vol. 15, No. 3, Spring 1989, 30-37; Richard C. Grinold and Ronald N. Kahn, Active Portfolio Management: A Quantitative Approach for Producing Superior Returns and Controlling Risk, Second Edition (New York: McGraw Hill, 2000), 147-169; Roger Clarke, Harindra de Silva, and Steven Thorley, “The Fundamental Law of Active Portfolio Management,” Journal of Investment Management, Vol. 4, No. 3, Third Quarter 2006, 54- 72.

39 Barbara Mellers, Eric Stone, Terry Murray, Angela Minster, Nick Rohrbaugh, Michael Bishop, Eva Chen, Joshua Baker, Yuan Hou, Michael Horowitz, Lyle Ungar, and Philip Tetlock, “Identifying and Cultivating

Superforecasting as a Method of Improving Probabilistic Predictions,” Perspectives on Psychological Science, Vol. 10, No. 3, May 2015, 267-281.

40 Keith E. Stanovich, Richard F. West, and Maggie E. Toplak, The Rationality Quotient: Toward a Test of Rational Thinking (Cambridge, MA: MIT Press, 2016).

41 Shane Frederick, “Cognitive Reflection and Decision Making,” Journal of Economic Perspectives, Vol. 19, No. 4, Fall 2005, 25-42.

42 Anders Ericsson and Robert Pool, Peak: Secrets from the New Science of Expertise (New York: Houghton Mifflin Harcourt, 2016).

43 Glenn W. Brier, “Verification of Forecasts Expressed in Terms of Probability,” Monthly Weather Review, Vol. 78, No. 1, January 1950, 1-3.

44 Kevin Kelly, “The Three Breakthroughs That Have Finally Unleashed AI On the World,” Wired, October 27, 2014.

45 Michael J. Mauboussin, Dan Callahan, David Rones, and Sean Burns, “Managing the Man Overboard Moment: Making an Informed Decision After a Large Price Drop,” Credit Suisse Global Financial Strategies, January 15, 2015; Michael J. Mauboussin, Dan Callahan, Darius Majd, Greg Williamson, and David Rones, “Celebrating the Summit: Making an Informed Decision After a Large Price Gain,” Credit Suisse Global Financial Strategies, January 11, 2016.

46 Julia Rozovsky, “The Five Keys to a Successful Google Team,” November 17, 2015. See https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/.