|
|
METHODTOOLSARTICLES Producticity, LLC 251 W. Central Ave. #312 Springboro, OH 45066 Phone: 937.609.8041 FAX: 937.748.3400 Email: mindy@producticity.com |
MEASURING THE IMPACT RESULTING FROM IMPLEMENTING AN ORGANIZATIONAL CAPABILITY PLAN NCR Corporation Sales and Service Operations by Melinda Morrow and William Wurtz ABSTRACT This case looks at the business impact achieved through an organization diagnostic and planning method known as the organization capability assessment and alignment process (OCAA). This approach aids executives in executing a new or reframed business strategy. The assessment helps determine what collective abilities an organization will need to execute the strategy, then sets action items to align those needs to critical capabilities called for in the new strategy. This study, involving the restructuring of a sales support organization of a major international computer manufacturer, demonstrates how to derive a credible return on investment estimate for this type of intervention. INTRODUCTION As companies struggle in an evermore competitive marketplace, organization development and other hard-to-measure functions are coming under increasing scrutiny. OD practitioners are faced with growing demands from clients and executives to demonstrate the value of their work. Indicators of this trend may be seen in both the professional literature and in the topics chosen for professional conferences. For example, Phillips (1977) reviews a growing list of books and articles, written for those in the human and organization performance improvement field, on measuring program outcomes to demonstrate value to senior executives. Another example is a survey, conducted by the world's largest association for performance improvement professionals, to determine member interest in potential topics for its 1998 annual conference. The results showed that evaluation, specifically outcome-based evaluation, to be one of the most demanded subjects by practitioners. A final example is another 1998 conference, this one sponsored by a leading OD graduate program, which promoted the theme of “Empirically-Driven Change” to emphasize the need for more rigor in OD measurement. Evaluation, of course, is not a new topic for OD practitioners. Evaluation has always been listed as the final phase of the action research model, a frequently used strategy in OD interventions (Sherwood, 1972). Yet even the most cursory review of OD projects shows wide variations in the comprehensiveness, rigor and credibility of these evaluations. The authors' informal observations indicate that the majority of OD evaluations feature little more than reaction data from clients and other participants. If these findings are a reasonable reflection of the current state of the field, then it is incumbent on OD practitioners to begin “tooling up” to conduct more rigorous evaluations. This article presents a method that provides a structured, rigorous, results-based approach to generating credible evaluation information. This approach can be of significant benefit to OD practitioners and their clients in assessing both the progress and ultimate impacts of OD efforts. The following study shows how a credible, and impressive, return on investment (ROI) estimate was derived for one application of a relatively new OD technology, the organizational capabilities assessment and alignment (OCAA) process. The analysis was conducted jointly by the principal consultant on the intervention (Morrow) and a colleague with expertise in results-based evaluation (Wurtz), both then members of an internal company OD group. The study begins with background information on the company, a global computer manufacturer, and its sales and service organization, which used the process. The study proceeds with a description of both the OCAA and ROI evaluation methodologies and how they were applied in this instance. It concludes with a discussion of implications of more rigorous evaluation for OD practice. NCR BACKGROUND NCR, formally the National Cash Register Company, has been in business for nearly 115 years. The company enjoyed good fortune for many of those years. It was known as a technical and organizational innovator and a breeding ground for executive talent. For example, Thomas Watson, the founder of IBM, built that company on practices he learned while at NCR. Recent years have been difficult for NCR. The company's fundamental mistake in the 1970s was to ignore the gathering force of the computer revolution until it was almost too late. Finally, as NCR approached financial ruin, the company's executives made the fateful decision to embrace the new technology. Once the decision was made, fully one-fourth of NCR's workforce was dismissed as being redundant on the still fearfully remembered “Black Friday.” Still, success has been elusive. After a few years of recovery, NCR still experienced minimal profitability. These difficulties were highlighted even more in an industry which can be characterized as highly competitive (NCR Information Statement, 1996). Another significant event in NCR's modern history was its hostile takeover by AT&T in September 1991. For five tumultuous years AT&T management attempted to meld NCR's computer manufacturing operations and global infrastructure into its telecommunications business and transform NCR's industrial-era culture to meet the demands of the Information Age. After losing $8 billion on its NCR operations, and another huge downsizing, AT&T spun off NCR as an independent company in 1997 (Nyberg, 1996). A new executive team adopted what it hoped would be a winning strategy for NCR. The goal was to transform NCR from a “box-maker” to a “solutions provider.” This involves reducing or eliminating computer (“boxes”) manufacturing and moving into “business solutions.” Solutions are bundles of consulting services, software, integrated networking, and boxes which NCR can offer business customers to address defined problems. Solutions provide high profit margins, while computers are increasingly becoming low-margin commodities. It was clear from the beginning that new skills and new organizational capabilities would be required for this strategy to be successful. BACKGROUND-The Sales and Service Operations Organization The Sales and Services Operations Organization's (S&SO) was created in January 1997 as part of a reorganization. The intent was to realize savings and efficiencies by eliminating duplication and coordinating related functions more effectively. S&SO's purpose is to perform the “back office” processing, inventory and delivery functions of the NCR sales process. This includes releasing products sold to customers, managing product delivery logistics, and processing invoices and collections. S&SO's 900 employees processed over 81,000 orders in 1997. The reorganization provided S&SO's vice president with the opportunity to significantly change how these functions were being performed at NCR. A successful effort would contribute substantially to the realization of the solutions model. He contracted with NCR's Organization Development Group for help in implementing a new business strategy. THE OCAA METHODOLOGY The organizational capability assessment and alignment (OCAA) process helps organizations examine what collective abilities (“capabilities”) are needed to successfully execute a business strategy and achieve business results (Ulrich, 1990). Capabilities, as shown in Figure 1, are made up, in part, of individual competencies, which in turn are made up of individual skills. OCAA is a form of organization diagnosis, a problem solving activity that identifies gaps between the organization's current and future states, with an action plan as its ultimate output. (Anderson and Morrow, 1997) The process enables organizations to identify the highest priority strategic capabilities that must be addressed for business success. Armed with this information, people in the organization can commit to a plan to create or enhance these capabilities and align everyone's efforts to organization objectives.  Figure 1  Figure 2 The standard ROI evaluation is conducted at five different levels, as Table 1 shows (Phillips, 1995). This schema establishes a chain of impact between the independent variable (in this case, the OCAA intervention) and the impact on the dependent variables (organizational results).
 Figure 3 Cost data for the intervention were kept and compiled by the OCAA practice leader, with the assistance of S&SO's finance department. In accordance with ROI analysis principles, these costs, itemized in Table 2, are “fully loaded.”
When the study was begun, the Level 1 data collected were the group action plan and participant reaction data. As is often the case with many OD interventions, and contrary to the practice with most training programs, the participant feedback was collected only from the intervention's primary clients. These clients were the vice-president and S&SO's internal consultant, the intervention's sustaining and initiating sponsors respectively. They gave scores of 4.13 and 4.68 (on a 1 to 5, low to high, scale), an average of 4.41. The learning (Level 2) that occurs in organization development interventions is also generally different from the learning that occurs in training programs. The goal of training programs is for individuals to acquire specific task-related knowledge and/or skills to some defined level of proficiency. OD interventions may sometimes encompass skill and knowledge objectives, yet at its core, OD efforts are about helping mobilize group commitment to bring about planned change within an organization. (Sherwood, 1972) Ideally, groups in organizations actively participate together in making decisions about the business unit's future direction. Through a process of group members sharing perceptions (learning from each other) about what needs to change and the options for achieving this change, a consensus is ultimately reached on what needs to happen. The caring at the center of this consensus is commitment, and it is this commitment that provides the emotional energy that aids in seeing a change through to full implementation. In traditional training-based ROI evaluation studies, increased commitment is typically classified as an intangible organizational impact (Level 4). We argue it is critically important for OD projects to assess the amount of commitment generated at the time the action plan is developed (in effect, Level 2). This was done informally at the time by the practice leader. Confirmation of this commitment was provided in the follow-up interviews conducted with group members for this study. For example, according to the S&SO vice president, "By using this approach to uncover key issues, and then by addressing the issues in a holistic fashion, we were able to realize the momentum necessary to quickly introduce the required changes in a lasting fashion." Determining Level 3, “application on the job,” was simple for this project. There was total concurrence among the interviewees leaders about how many (20 of the 34) and which of the action items had been implemented. Level 4, organizational impacts, is of the greatest concern to executives and OD practitioners and the source of information for Level 5, ROI impacts. Level 4 benefits are both intangible and tangible. INTANGIBLE BENEFITS Some of the intangible benefits S&SO leaders identified as resulting from participating in the OCAA process bear directly on the commitment issue. These include: focusing on a few key actions for quicker implementation; facilitating communication; and enhancing accountability. According to the S&SO vice president, "By using this approach to uncover key issues, and then by addressing the issues in a holistic fashion, we were able to realize the momentum necessary to quickly introduce the required changes in a lasting fashion." ROI CALCULATION Three primary cost savings, attributed to the OCAA intervention, were identified:
6 positions savings @ $60,000/position = $360,000 Additional savings of $1.7 Million in streamlined processes = $360,000 + $1,700,000 = $2,060,000 = $2,060,000 x (most conservative contribution/confidence estimate) 30% = $618,000 Administrative duties consolidation 68.5 full time equivalent positions savings @ $40,000/position = 68.5 x $40,000 x (most conservative contribution/confidence estimate) 30% = $822,000 Reduced maintenance contract cancellations =$18 Million x (most conservative contribution/confidence estimate) 20% = $3,600,000 TOTAL INTERVENTION BENEFITS = $618,000 + $822,000 + $3,600,000 = $5,040,000 ROI = (OCAA Net Benefits ¸ OCAA costs) x 100 = ($5,040,000 - $103,011) ¸ $103,011 = $4,936,989 ¸ $103,011 = x 100 = 4792% ROI In addition to these three benefits, a number of other potential tangible benefits were identified. Several are too early in the implementation phase to yield quantifiable results. The yields from some of the other items are not significant. The list includes: lowered inventory losses, reduced air freight costs, streamlined processes, and better inventory management. SUMMARY AND CONCLUSIONS The demand for greater organizational accountability is growing, a trend that is likely to continue for some time, as executives try to cope with increasing competition and other forces. OD professionals must respond to this trend with more rigorous measurement efforts. As this study has demonstrated, practical methods exist for developing credible estimates of OD impacts on business results. And while it seems unlikely that many OD efforts will show the huge return evidenced in this study, we are confident that most properly designed and skillfully executed OD projects will show a positive return. ROI can be a convenient shorthand way to communicate with executives about an OD project's impact. It is important, however, to caution against fixating on the production of a ROI number as the major goal of evaluation. We believe this can create a backward-looking habit of mind that is self-defeating, in two ways. First, it can ultimately erode the very credibility the OD practitioner is trying to build, because clients may come to question the practitioner's motives if the only apparent reason for evaluating is to justify his existence. Second, the resource-intensive activity of evaluation makes little sense if its major output is a single number like ROI, printed in a report that ends up gathering dust on a shelf. We believe the goals of evaluation should be the promotion of organizational learning and the advancement of the OD discipline. Let us address these in turn. First, as the action-research model suggests, evaluation can add vital information in iterative cycles to an organization's knowledge capital. We envision a time when evaluation data are systemically entered into a predictive database, enabling the organization to better understand and cope with its environment. Second, we believe that more rigorous measurement will accelerate the development of OD. For example, our participation in this study heightened our awareness of the need to measure more precisely organizational commitment at Level 2 of an engagement like OCAA, as well as Level 4. The challenge for OD practitioners is to develop more accurate and finely-tuned instruments to assess this critical variable. In addition, we have discovered that the evaluation process itself enhances the practice of OD. Working with our clients on the evaluation portion of our intervention plans clarifies for all concerned the expected outcomes and facilitates discussion on how to achieve those outcomes. REFERENCES Anderson, M.C. and Morrow, M. “Organization Capability: Creating Simplicity and Focus in Business Life.” Organization Development Journal, Spring 1997, pp. 72-82. Information Statement: NCR Corporation Common Stock, Par Value $.01 Preferred Share Purchase Rights, 199X, states on page 11: "NCR faces significant competition in all geographic areas where it operates. Its markets are characterized by continuous, rapid technological change, the need to introduce products in a timely manner in order to take advantage of market opportunities, short product life cycles, frequent product performance improvements, and price reductions." Nyberg, Lars, “Chairman's Letter to Shareholders,” in the 1996 NCR Annual Report. The Chairman and CEO of NCR describes on page 12 the company's condition in its final months as a AT&T subsidiary in this way: "...When I joined NCR in June 1995, the company was trying to develop comprehensive solutions for six very different industries. NCR was also trying to compete in the personal computer business. And, as margins shrank in the computer industry, NCR wasn't controlling expenses. The company was losing about $2 million a day...Throughout [the subsequent] turnaround, the people of NCR rallied to fix--perhaps even 'save'--the business." Phillips, Jack J. In Action: Measuring Return on Investment (Vol. 1). American Society for Training and Development: Alexandria, VA 1995. Phillips, Jack J. Return On Investment In Training and Performance Improvement Programs. Gulf Publishing Company: Houston, TX 1997. Sherwood, John J. “An Introduction To Organization Development,” in Pfeiffer and Jones, eds., The 1972 Annual Handbook For Group Facilitators. University Associates: Iowa City, IA 1972. Ulrich, D. Organization Capability. Jossey-Bass: San Francisco, CA. 1990. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||