成功的真正衡量标准：大多数公司使用错误的绩效指标。不要成为其中之一。

HBR.ORG October 2012 reprinT R1210B

The Big Idea

The True Measures Of Success Most companies use the wrong performance metrics. Don’t be one of them. by Michael J. Mauboussin

A The Big Idea The True Measures of Success

About a dozen years the cheap. The book was published nearly a decade ago, when I was working for a large financial ser- ago, and its business implications have been thor-vices firm, one of the senior executives asked me to oughly dissected. Still, the key lesson hasn’t sunk in. take on a project to better understand the company’s Businesses continue to use the wrong statistics. profitability. I was in the equity division, which gen- Before the A’s adopted the methods Lewis deerated fees and commissions by catering to invest- scribes, the team relied on the opinion of talent ment managers and sought to maximize revenues by scouts, who assessed players primarily by looking providing high-quality research, responsive trading, at their ability to run, throw, field, hit, and hit with and coveted initial public offerings. While we had power. Most scouts had been around the game hundreds of clients, one mutual fund company was nearly all their lives and had developed an intuitive our largest. We shuttled our researchers to visit with sense of a player’s potential and of which statistics its analysts and portfolio managers, dedicated capi- mattered most. But their measures and intuition tal to ensure that its trades were executed smoothly, often failed to single out players who were effective and recognized its importance in the allocation of but didn’t look the role. Looks might have nothing IPOs. We were committed to keeping the 800-pound to do with the statistics that are actually important: gorilla happy. those that reliably predict performance.

Part of my charge was to understand the divi- Baseball managers used to focus on a basic num-sion’s profitability by customer. So we estimated the ber—team batting average—when they talked about cost we incurred servicing each major client. The scoring runs. But after doing a proper statistical results were striking and counterintuitive: Our larg- analysis, the A’s front office recognized that a play-est customer was among our least profitable. Indeed, er’s ability to get on base was a much better predictor customers in the middle of the pack, which didn’t of how many runs he would score. Moreover, ondemand substantial resources, were more profitable base percentage was underpriced relative to other than the giant we fawned over. abilities in the market for talent. So the A’s looked What happened? We made a mistake that’s ex- for players with high on-base percentages, paid less ceedingly common in business: We measured the attention to batting averages, and discounted their wrong thing. The statistic we relied on to assess gut sense. This allowed the team to recruit winning our performance—revenues—was disconnected players without breaking the bank.

from our overall objective of profitability. As a re- Many business executives seeking to create sult, our strategic and resource allocation decisions shareholder value also rely on intuition in selecting didn’t support that goal. This article will reveal how statistics. The metrics companies use most often to this mistake permeates businesses—probably even measure, manage, and communicate results—often yours—driving poor decisions and undermining per- called key performance indicators—include finan-formance. And it will show you how to choose the cial measures such as sales growth and earnings best statistics for your business goals. per share (EPS) growth in addition to nonfinancial measures such as loyalty and product quality. Yet, Ignoring Moneyball’s Message as we’ll see, these have only a loose connection to Moneyball, the best seller by Michael Lewis, de- the objective of creating value. Most executives con-scribes how the Oakland Athletics used carefully tinue to lean heavily on poorly chosen statistics, the chosen statistics to build a winning baseball team on equivalent of using batting averages to predict runs.

4 Harvard Business Review October 2012 Copyright © 2012 Harvard Business School Publishing Corporation. All rights reserved.

Idea in Brief The statistics that companies use most Useful statistics have two qualities. They 1 Define your governing objective.

often to track and communicate perfor- are persistent, showing that the outcome

creating shareholder value. Choosing the right statistics—metrics that will allow you to understand, track, 4 Regularly reevaluate the chosen and manage the cause-and-effect rela- statistics to ensure that they tionships that determine the value of your continue to link employee activities company—is a four-step process. with the governing objective.

Like leather-skinned baseball scouts, they have a gut sense of what metrics are most relevant to their businesses, but they don’t realize that their intuition may be flawed and their decision making may be skewed by cognitive biases. Through Only through my work, teaching, and research on these biases, I proper statistical have identified three that seem particularly relevant analysis of a host of fac-in this context: the overconfidence bias, the avail- tors that could drive customer ability heuristic, and the status quo bias. satisfaction did the company discover Overconfidence. People’s deep confidence in that turnover among store managers, not in the their judgments and abilities is often at odds with overall employee population, made the difference. reality. Most people, for example, regard them- As a result, the firm shifted its focus to retaining selves as better-than-average drivers. The tendency managers, a tactic that ultimately boosted satisfac-toward overconfidence readily extends to business. tion and profits.

Consider this case from Stanford professors David Availability. The availability heuristic is a strat-Larcker and Brian Tayan: The managers of a fast- egy we use to assess the cause or probability of an food chain, recognizing that customer satisfaction event on the basis of how readily similar examples was important to profitability, believed that low em- come to mind—that is, how “available” they are to ployee turnover would keep customers happy. “We us. One consequence is that we tend to overestimate just know this is the key driver,” one executive ex- the importance of information that we’ve encoun-plained. Confident in their intuition, the executives tered recently, that is frequently repeated, or that is focused on reducing turnover as a way to improve top of mind for other reasons. For example, execu-customer satisfaction and, presumably, profitability. tives generally believe that EPS is the most impor-As the turnover data rolled in, the executives tant measure of value creation in large part because were surprised to discover that they were wrong: of vivid examples of companies whose stock rose Some stores with high turnover were extremely after they exceeded EPS estimates or fell abruptly profitable, while others with low turnover struggled. after coming up short. To many executives, earnings

People’s deep confidence in their judgments and abilities is often at odds with reality.

The Problem with Popular Measures EPS growth and sales growth are

eps compound annual growth rate another time) and predictive (they link sales growth in the two periods is cause and effect, predicting the out- unrelated (there is zero correlation), 40 come being measured). Statisticians r = 0—a random pattern. If increases assess a measure’s persistence and in one period match decreases in the its predictive value by examining the -50% 50 100 150 200 other (a perfect inverse correlation), coefficient of correlation: the degree r = –1.00—also a straight line. Even

2008–2010 of the linear relationship between -40 a quick glance can tell you whether variables in a pair of distributions. Put there is a high correlation between simply, if there is a strong relationship -80 the variables (the points are tightly between two sets of variables (say a eps compound annual growth rate clustered and linear) or a low correla- 2005–2007 group of companies’ sales growth in tion (they’re randomly scattered). two different periods), plotting the 50% The closer to 1.00 or –1.00 the points on a graph like the ones shown 40 r = 0.28 co­efficient of correlation is, the more here produces a straight line. If there’s persistent and predictive the statistic.

SALES compound annual growth rate 30 no relationship between the variables, The closer to zero, the less persistent 20 the points will appear to be randomly and predictive the statistic. 10 scattered, in this case showing that sales growth in the first period does -40% -20 20 40 60 80 100 Let’s examine the persistence not predict sales growth in the second. -10 of two popular measures: EPS In comparing the variable “sales -20 growth and sales growth. growth” in two periods, the coefficient The figures to the left show the coef-

2008–2010 -30 of correlation, r, falls in the range of -40 ficient of correlation for EPS growth 1.00 to –1.00. If each company’s sales -50 and sales growth for more than 300 growth is the same in both periods (a Sales compound annual growth rate large nonfinancial companies in the 2005–2007

growth seems like a reliable cause of stock-price in- inhibit such a shift, and so executives end up manag-creases because there seems to be so much evidence ing the business with stale statistics. to that effect. But, as we’ll see, the availability heuristic often leads to flawed intuition. Considering Cause and Effect Status quo. Finally, executives (like most peo- To determine which statistics are useful, you must

ple) would rather stay the course than face the risks ask two basic questions. First, what is your objec-that come with change. The status quo bias derives tive? In sports, it is to win games. In business, it’s in part from our well-documented tendency to usually to increase shareholder value. Second, what avoid a loss even if we could achieve a big gain. A factors will help you achieve that objective? If your business consequence of this bias is that even when goal is to increase shareholder value, which activi-performance drivers change—as they invariably do— ties lead to that outcome?

executives often resist abandoning existing metrics What you’re after, then, are statistics that reliably in favor of more-suitable ones. Take the case of a reveal cause and effect. These have two defining subscription business such as a wireless telephone characteristics: They are persistent, showing that the provider. For a new entrant to the market, the acqui- outcome of a given action at one time will be similar sition rate of new customers is the most important to the outcome of the same action at another time; performance metric. But as the company matures, and they are predictive—that is, there is a causal reits emphasis should probably shift from adding lationship between the action the statistic measures customers to better managing the ones it has by, for and the desired outcome.

instance, selling them additional services or reduc- Statistics that assess activities requiring skill are ing churn. The pull of the status quo, however, can persistent. For example, if you measured the perfor-

…and they don’t do a good job of United States. The compounded an- predicting value creation. growth shows a reasonably good cor-nual growth rates from 2005 to 2007, 200% relation with increasing shareholder on the horizontal axes, are compared r = 0.37 value (r = 0.37), so it is somewhat pre-with the rates from 2008 to 2010, on 150 dictive. The problem is that forecast-

RELATIVE TOTAL SHAREHOLDER RETURN the vertical axes. If EPS and sales ing earnings is difficult because, as 100 growth were highly persistent and, we saw in the previous analysis, EPS therefore, dependent on factors the growth in one period tells you little 50 company could control, the points about what will happen in another. would cluster tightly on a straight line. Earnings data may be moderately But in fact they’re widely scattered, -80% -40 40 80 120 predictive of shareholder returns,

2008–2010 revealing the important role of chance but they are not persistent. -50 or luck. The correlation is negative Using sales growth as a gauge of and relatively weak (r = –0.13) for -100 value creation falls short for a dif-EPS growth but somewhat higher eps compound annual growth rate ferent reason. While sales growth is 2008–2010 (r = 0.28) for sales growth. This is more persistent than EPS growth, it consistent with the results of large- 200% is less strongly correlated with rela-scale studies. r = 0.27 tive total returns to shareholders 150 (r = 0.27). In other words, sales-Next, we’ll look at the predictive growth statistics may be somewhat

RELATIVE TOTAL SHAREHOLDER RETURN 100 value of EPS growth and sales persistent, but they’re not very growth by examining the correlation predictive. 50 of each with shareholder returns. Thus the two most popular mea-In the figures to the right, adjusted sures of performance have limited EPS growth and sales growth are on -40 -30 -20 -10 10 20 30 40 50 value in predicting shareholder the horizontal axes. The vertical axes returns because neither is both per­

2008–2010 -50 are the total return to shareholders for sistent and predictive. each company’s stock less the total -100 return for the S&P 500. Adjusted EPS Sales compound annual growth rate 2008–2010

mance of a trained sprinter running 100 meters on is also more persistent than batting average because two consecutive days, you would expect to see simi- it incorporates more factors—including the ability lar times. Persistent statistics reflect performance to get walked—that reflect skill. So we can conclude that an individual or organization can reliably con- that a team’s on-base percentage is better for predict-trol through the application of skill, and so they ex- ing the performance of a team’s offense. pose causal relationships. All this seems like common sense, right? Yet com-It’s important to distinguish between skill and panies often rely on statistics that are neither very luck. Think of persistence as occurring on a contin- persistent nor predictive. Because these widely used

uum. At one extreme the outcome being measured metrics do not reveal cause and effect, they have lit-is the product of pure skill, as it was with the sprinter, tle bearing on strategy or even on the broader goal of and is very persistent. At the other, it is due to luck, earning a sufficient return on investment. so persistence is low. When you spin a roulette Consider this: Most corporations seek to maxi-wheel, the outcomes are random; what happens on mize the value of their shares over the long term. the first spin provides no clue about what will hap- Practically speaking, this means that every dollar pen on the next. a company invests should generate more than one To be useful, statistics must also predict the result dollar in value. What statistics, then, should exyou’re seeking. Recall the Oakland A’s recognition ecutives use to guide them in this value creation? that on-base percentage told more about a player’s As we’ve noted, EPS is the most popular. A survey likelihood of scoring runs than his batting average of executive compensation by Frederic W. Cook &

did. The former statistic reliably links a cause (the Company found that it is the most popular mea-ability to get on base) with an effect (scoring runs). It sure of corporate performance, used by about half

other companies adopt them to achieve the result of skill and overlook the influence of same great results. This formula is intuitive, luck. We connect cause and effect where includes some compelling narrative, and has there is no connection.

sold millions of books. The lesson is clear: When luck plays a part If causality were clear, this approach in determining the consequences of your would work. The trouble is that the perfor- actions—as is often the case in business— mance of a company almost always depends you don’t want to study success to identify on both skill and luck, which means that a good strategy but rather study strategy to given strategy will succeed only part of the see whether it consistently led to success.

time. Some companies using the strategy Statistics that are persistent and predictive, will succeed; others will fail. So attributing a and so reliably link cause and effect, are firm’s success to a specific strategy may be indispensable in that process.

wrong if you sample only the winners.

of all companies. Researchers at Stanford Graduate survey by Graham and his colleagues found that the School of Business came to the same conclusion. majority of companies were willing to sacrifice long-And a survey of 400 financial executives by finance term economic value in order to deliver short-term professors John Graham, Campbell Harvey, and earnings. Theory and empirical research tell us that Shiva Rajgopal found that nearly two-thirds of com- the causal relationship between EPS growth and panies placed EPS first in a ranking of the most im- value creation is tenuous at best. Similar research reportant performance measures reported to outsid- veals that sales growth also has a shaky connection

ers. Sales revenue and sales growth also rated highly to shareholder value. (For a detailed examination for measuring performance and for communicating of the relationship between earnings growth, sales externally. growth, and value, see the exhibit “The Problem But will EPS growth actually create value for with Popular Measures.”)

shareholders? Not necessarily. Earnings growth and Of course, companies also use nonfinancial value creation can coincide, but it is also possible to performance measures, such as product quality, increase EPS while destroying value. EPS growth is workplace safety, customer loyalty, employee satis-good for a company that earns high returns on in- faction, and a customer’s willingness to promote a vested capital, neutral for a company with returns product. In their 2003 HBR article, accounting pro-equal to the cost of capital, and bad for companies fessors Christopher Ittner and David Larcker wrote with returns below the cost of capital. Despite that “most companies have made little attempt to this, many companies slavishly seek to deliver EPS identify areas of nonfinancial performance that growth, even at the expense of value creation. The might advance their chosen strategy. Nor have they

demonstrated a cause-and-effect link between im- ford. Leave aside, for the moment, which metrics provements in those nonfinancial areas and in cash you currently use or which ones Wall Street analysts flow, profit, or stock price.” The authors’ survey of or bankers say you should. Start with a blank slate

1 157 companies showed that only 23% had done ex- and work through these four steps in sequence. tensive modeling to determine the causes of the effects they were measuring. The researchers suggest Define your governing objective. A clear that at least 70% of the companies they surveyed objective is essential to business success bedidn’t consider a nonfinancial measure’s persistence cause it guides the allocation of capital. Creat-or its predictive value. Nearly a decade later, most ing economic value is a logical governing objective companies still fail to link cause and effect in their for a company that operates in a free market system. choice of nonfinancial statistics. Companies may choose a different objective, such as But the news is not all bad. Ittner and Larcker maximizing the firm’s longevity. We will assume that

2 did find that companies that bothered to measure a the retail bank seeks to create economic value. nonfinancial factor—and to verify that it had some real effect—earned returns on equity that were about Develop a theory of cause and effect to 1.5 times greater than those of companies that didn’t assess presumed drivers of the objec-take those steps. Just as the fast-food chain boosted tive. The three commonly cited financial its performance by determining that its key metric drivers of value creation are sales, costs, and invest-was store manager turnover, not overall employee ments. More-specific financial drivers vary among turnover, companies that make proper links be- companies and can include earnings growth, cash tween nonfinancial measures and value creation flow growth, and return on invested capital. stand a better chance of improving results. Naturally, financial metrics can’t capture all value-creating activities. You also need to assess Picking Statistics non­financial measures such as customer loyalty, The following is a process for choosing metrics that customer satisfaction, and product quality, and deallow you to understand, track, and manage the termine if they can be directly linked to the financial cause-and-effect relationships that determine your measures that ultimately deliver value. As we’ve dis-company’s performance. I will illustrate the process cussed, the link between value creation and financial in a simplified way using a retail bank that is based on an analysis of 115 banks by Venky Nagar of the University of Michigan and Madhav Rajan of Stan-

Companies that link nonfinancial measures and value creation stand a better chance of improving results.

and nonfinancial measures like these is variable and persistent. The bank can use this information to, for must be evaluated on a case-by-case basis. example, make sure that its process for reviewing

4 In our example, the bank starts with the theory and approving loans is quick and efficient. that customer satisfaction drives the use of bank services and that usage is the main driver of value. This Evaluate your statistics. Finally, you must theory links a nonfinancial and a financial driver. regularly reevaluate the measures you are The bank then measures the correlations statisti- using to link employee activities with the cally to see if the theory is correct and determines governing objective. The drivers of value change that satisfied customers indeed use more services, over time, and so must your statistics. For example, allowing the bank to generate cash earnings growth the demographics of the retail bank’s customer base and attractive returns on assets, both indicators of are changing, so the bank needs to review the driv-value creation. Having determined that customer ers of customer satisfaction. As the customer base satisfaction is persistently and predictively linked becomes younger and more digitally savvy, teller to returns on assets, the bank must now figure out turnover becomes less relevant and the bank’s on-

3 which employee activities drive satisfaction. line interface and customer service become more so.

Identify the specific activities that em- Companies have access to a growing torrent of sta-ployees can do to help achieve the gov- tistics that could improve their performance, but exerning objective. The goal is to make the ecutives still cling to old-fashioned and often flawed link between your objective and the measures that methods for choosing metrics. In the past, compa-employees can control through the application of nies could get away with going on gut and ignoring skill. The relationship between these activities and the right statistics because that’s what everyone the objective must also be persistent and predictive. else was doing. Today, using them is necessary to In the previous step, the bank determined that compete. More to the point, identifying and exploit-customer satisfaction drives value (it is predictive). ing them before rivals do will be the key to seizing The bank now has to find reliable drivers of customer advantage.  HBR Reprint R1210B satisfaction. Statistical analysis shows that the rates Michael J. Mauboussin is the chief investment consumers receive on their loans, the speed of loan strategist at Legg Mason Capital Management and an processing, and low teller turnover all affect cus- adjunct professor of finance at Columbia Business School. He is the author of The Success Equation (Harvard Business tomer satisfaction. Because these are within the Review Press, forthcoming), from which this article was control of employees and management, they are developed.

Cartoon: p.c. vey “I hear they call you the hard-copy guy.”