As one physician observed, “Seldom have there been such imaginative ideas, such enthusiasms and dislikes, and such unscientific observations and study about one clinical picture.” The physician, Dr. Virginia Apgar, an anesthesiologist by training, observed the need for a practical method to rapidly evaluate a newborn’s condition. The goal of the system was to predict survival, to compare methods of resuscitation, and to compare perinatal experiences across hospitals. In business terms, Dr. Apgar wanted to anticipate events and select a strategy to respond, compare the performance of each strategy, and compare performance across organizations. Working with colleagues, Dr. Apgar reviewed the physiological signs that influence neonatal outcomes, ultimately selecting just five indicators — graded on a three-point scale — to assess newborn infants. The system of measurement was selected on the basis that it “could be evaluated without special equipment and could be taught to the delivery room personnel without difficulty.” In short, the system was practical.
The Apgar Score, as the Newborn Scoring System later became known, has saved countless lives over the last 60 years. Its utility is such that it continues to be used to this day despite all of the advances in modern medicine. Dr. Apgar’s innovation has touched the lives of almost everyone living in the United States, yet the story of her innovation and the underlying implications for performance measurement remain largely underexplored by executives and management. Dr. Apgar demonstrated three key points that many executives often overlook or underappreciate: (1) the metrics, or key performance indicators, that you use matter and significantly influence success within and across organizations; (2) the disciplined application of performance measurement outperforms executive intuition (note: this holds without respect to industry or setting); and (3) that in a world increasingly filled with “imaginative ideas,” the key to successful performance measurement involves the very practical approach of asking better questions, not having more data. To see how these lessons matter beyond medical care, let us advance 40 years and recall an American pastime — baseball.
Information Advantages in Baseball
In 2002, frustrated by a post-season loss to the Yankees, the impending departure of his top talent, and a severely limited operating budget, Oakland A’s general manager Billy Beane embraced Peter Brand, a young Yale economist. Peter was possessed by a notion that baseball had developed a fundamentally incomplete view of performance that undervalued certain players — presenting an opportunity to build a winning team at a marginally lower cost. This occurred despite over 100 years of experience and tradition in a sport where managers had full and complete data on every aspect of performance, and teams were ripe with experts and expertise. In economic terms it was a competitive environment with near perfect information, yet to quote from Brand’s character in the 2011 movie “Moneyball,” “Managers have an imperfect understanding of where runs come from … they are asking all the wrong questions … and this leads people who run Major League Baseball teams to misjudge their players and mismanage their teams.”
Like Dr. Apgar, the Oakland A’s story highlights the same three key points for performance measurement. First, despite today’s focus on “big data,” executives need to reframe their focus from more data to better questions. Beane and Brand’s refinement of baseball’s basic questions — including which matters more for management decision-making, batting averages or on-base percentages — allowed them to identify an unexploited market opportunity to reduce the marginal cost of success. Second, systematic application of performance measurement is key. Though executives make decisions in complex and uncertain situations often fraught with biases, the Oakland A’s demonstrated that the disciplined application of sound metrics in the face of expert and cultural opposition yields measureable benefits — more cost-effective personnel selections. And finally — metrics matter. Though much of life is consumed by “vanity metrics” such as calories, steps, or hours worked, a critical role for executive decision-makers is to push their teams to identify high-performing metrics to improve decision-making. Beane and Brand’s use of high-performing key metrics, not simply all metrics available, allowed them to exploit undervalued players and achieve a cost per win in the 2002 season that was three times less than 2002 World Champion New York Yankees.
The Pathway to Metrics that Matter
As each story illustrates, measuring and understanding performance is critical. Metrics matter; they help us identify opportunities and avoid making intuitive mistakes. As we better understand critical measurement questions, we also shed new light on existing data or the need for additional data. Whether by a physician, a baseball manager, or an executive, decision-making is inherently uncertain and complicated by complex dynamics that emerge from the very nature of our daily experience. As a result, performance measurement and related metrics become critical resources to reduce uncertainty, drive decision-making, and impact organizational performance. So how do you move from these two illuminating stories to ensure that your metrics matter? A final story provides a powerful and more commonly known example that sheds light on a practical approach to making metrics matter.
To begin, we must acknowledge that neither effort — either Dr. Apgar’s or Billy Beane’s — were entirely novel. In the case of the Oakland A’s, the use of sabermetrics (the statistical analysis of baseball data) had been underway for some time, even within the Oakland A’s front office. Both efforts possessed limitations, considerations for use, and complementary approaches. However, despite these notable facets, each performance measurement effort possessed uncommon clarity of purpose for measurement. With a clear purpose, each developed a practical approach to identify key facets of performance that were feasible to collect, provided timely feedback, and promoted effective decision-making to guide the achievement of outcomes — not simply actions.
Accordingly, the answer to making metrics matter can be found along a pathway that includes three critical actions: (1) define a clear purpose for measurement; (2) identify the context for measurement; (3) and isolate the key questions — and only those questions — that help you achieve the defined purpose of measurement. From the hard work of defining a clear purpose, understanding the measurement context, and isolating key questions, metrics that matter emerge. In fact, they are often just short of obvious once put in this frame. Let us now examine the example of Enterprise Rent-A-Car, the main face, albeit a subsidiary, of Enterprise Holdings.
Car Rentals and Customer Loyalty
Measuring and managing customer loyalty is critical to every organization and has been a focus of performance measurement since the concept rose to prominence in the 1990s. To-date there have been over 100,000 research articles and almost half a million web pages that discuss measuring customer loyalty — an astounding figure. Historically, the measurement of customer loyalty focused on customers experience with specific transactions — indicators that later research would show were frequently subject to variation not correlated with loyalty. Enterprise Rent-A-Car executives appeared to have intuitively grasped this challenge, and moreover they recognized the practical limitations of detailed customer surveys that were lengthy and took a long time to complete. To develop their approach to measuring customer loyalty, Enterprise executives honed their focus to accomplish a single purpose: identify and measure the customer who will recommend Enterprise to friends — viewed internally as the single most important driver of growth.
From this narrow, almost singular approach to measuring customer loyalty, Enterprise isolated two key facets that they felt both reflected a customer’s likelihood to recommend and could be measured quickly. These two facets — quality of experience and likelihood to rent again — and the resulting two related questions, yielded a performance measurement process that was simple, practical, and fast. It also delivered concise and timely feedback to managers at over 5,000 locations, translating into improved performance and shared learning. A presentation of the approach by then CEO Andy Taylor led Fred Reichheld to develop and advance what today is known as the Net Promoter Score.® The work has since been refined and has informed the development of Enterprise’s Service Quality Index.
Concerning the results of this approach to performance measurement at Enterprise, one cannot directly link this effort to the exclusion of all other strategies employed by the company (e.g., a focus on the off-airport market). However, as of 2013 their combination of strategy, management, and measurement have allowed Enterprise Holdings to achieve record revenues and growth through one of the toughest economic periods in modern financial history. From a customer loyalty perspective, J.D. Power and Associates has ranked Enterprise Rent-A-Car “Highest in Rental Car Customer Satisfaction” among North American airport rentals eight times in the past nine years, and Enterprise Holdings — the parent company of Enterprise, National, and Alamo Rent-A-Car —has held the top three positions in J.D. Power rankings for two of the past three years.
Making Metrics matter
Measurement and metrics that matter, in their simplest form, provide data and information to decision-makers that help reduce uncertainty when making decisions intended to produce or achieve a desired result. Organizations are confronted with uncertain decisions in almost every aspect of both strategy and operations. Two forms of organizational response are generally elicited in these situations. In the first instance, and perhaps most common, we under emphasize the importance of measurement, overweigh consensus and intuition, and move without an abundance of caution to make decisions and develop plans of action. Decision-making focuses on two extremes concurrently: (1) we know it already, and we have no need for measurement; and (2) there is no way to measure it so we can exclude it from further consideration. This leads to obvious challenges that often result in perilous outcomes.
In the second instance, an organization’s decision-makers assemble available data, develop measurements that reflect intuitive or salient assessments of the decision context, and produce lists (sometimes endless) of key indicators and other information that present a seemingly coherent picture of performance that is then quickly adopted and used to guide decision-making. In the rapid procession to analyze the decision context, important measurement components can be overlooked. Components overlooked include: clear problem definition, defining a purpose for measurement, ensuring the alignment of measures to the defined problem, and understanding the context for measurement and the effect of the new measurement system on organizational culture and process. As the three illustrations show, we need move forward with neither intuition nor endless lists of indicators and data. There is a meaningful, powerful middle ground where successful insights are waiting to be found.
