Using Leading Indicators to Improve Project Performance Measurement

Abstract

Project performance measurement offers a wide range of methods to help project managers monitor projects. However, several critical issues remain in project performance measurement, such as an unbalanced use of indicators due to a lack of prediction-based (leading) indicators with regard to outcome-based (lagging) indicators. For its part, systems engineering measurement, although a more recent discipline, offers a wide variety of indicators, including a set of leading indicators. As our objective is to increase project performance and success rates, this involves improving project performance measurement on which project management decisions are based. This purpose, the proposal put forward in this paper is to extend the number and type of indicators used in project performance measurement by adapting the leading indicators defined in systems engineering measurement. The methodology involves first mapping the systems engineering indicators with the project management activities, resulting in identification of a set of potentially useful indicators for measuring the different activities, then tailoring a selection of these indicators with project-specific data to define a set of the most relevant indicators for a given project. This methodology is illustrated by means of a case study in a manufacturing company.

Keywords

Project performance measurement / systems engineering measurement / leading indicators / lagging indicators

References

[1] Aziz RF, Hafez SM. Applying lean thinking in construction and performance improvement. Alexandria Engineering Journal, 2013, 52(4): 679-695.

[2] Atkinson R. Project management: cost, time and quality, two best guesses and a phenomenon, it’s time to accept other success criteria. International Journal of Project Management, 1999, 17(6): 337-342.

[3] Anbari FT. Earned value project management method and extensions. Project Management Journal, 2003, 34(4): 12-23.

[4] Anderson K, McAdam R. A critique of benchmarking and performance measurement: Lead or lag?. Benchmarking: an International Journal, 2004, 11(5): 465-483.

[5] Alter S. Work system theory: Overview of core concepts, extensions, and challenges for the future. Journal of the Association for Information Systems, 2013, 14(2): 72.

[6] Barber E. Benchmarking the management of projects: A review of current thinking. International Journal of Project Management, 2004, 22(4): 301-307.

[7] Choong KK. Are PMS meeting the measurement needs of BPM? A literature review. Business Process Management Journal, 2013, 19(3): 535-574.

[8] Choong KK. Understanding the features of performance measurement system: A literature review. Measuring Business Excellence, 2013, 17(4): 102-121.

[9] Choong KK. Has this large number of performance measurement publications contributed to its better understanding? A systematic review for research and applications. International Journal of Production Research, 2014, 52(14): 4174-4197.

[10] Choong KK. Use of mathematical measurement in improving the accuracy (reliability) & meaningfulness of performance measurement in businesses & organizations. Measurement, 2018, 129: 184-205.

[11] Cha HS, Kim CK. Quantitative approach for project performance measurement on building construction in South Korea. KSCE Journal of Civil Engineering, 2011, 15(8): 1319-1328.

[12] Choi JW. Forecasting Potential Project Risks through Leading Indicators to Project Outcome, 2007, College Station, USA: Texas A&M University

[13] Cheung SO, Suen HC, Cheung KK. PPMS: A web-based construction project performance monitoring system. Automation in Construction, 2004, 13(3): 361-376.

[14] Cao Q, Hoffman JJ. A case study approach for developing a project performance evaluation system. International Journal of Project Management, 2011, 29(2): 155-164.

[15] Chen HL, Chen WT, Lin YL. Earned value project management: Improving the predictive power of planned value. International Journal of Project Management, 2016, 34(1): 22-29.

[16] Conforto E, Rossi M, Rebentisch E, Oehmen J, Pacenza M (2013). Improving the Integration of Program Management and System Engineering. White paper presented at the 23th INCOSE Annual International Symsposium Philadelphia, USA, 293–298.

[17] Eilat H, Golany B, Shtub A. R&D project evaluation: An integrated DEA and balanced scorecard approach. Omega, 2008, 36(5): 895-912.

[18] Franceschini F, Galetto M, Maisano D, Viticchie L. The condition of uniqueness in manufacturing process representation by performance/quality indicators. Quality and Reliability Engineering International, 2006, 22(5): 567-580.

[19] Franceschini F, Galetto M, Maisano D. Classification of performance and quality indicators in manufacturing. International Journal of Services and Operations Management, 2006, 2(3): 294-311.

[20] Grabowski M, Premnath A, Jason M, John RH, Karlene R. Leading indicators of safety in virtual organizations. Safety Science, 2007, 45(10): 1013-1043.

[21] Guo BH, Yiu TW. Developing leading indicators to monitor the safety conditions of construction projects. Journal of Management in Engineering, 2015, 32(1): 04015016.

[22] Hall NG. Project management: Recent developments and research opportunities. Journal of Systems Science and Systems Engineering, 2012, 21(2): 129-143.

[23] Henttonen K, Ojanen V, Puumalainen K. Searching for appropriate performance measures for innovation and development projects. R&D Management, 2016, 465: 914-927.

[24] Hopkins A. Failure to Learn: The BP Texas City Refinery Disaster, 2009, Darwin, Australia: CCH Australian Limited

[25] ISO/IEC (2007). IEEE Standard Adoption of ISO/IEC 15939:2007 — Systems and Software Engineering — Measurement Process, 2007.

[26] INCOSE Measurement Working Group (1998). Systems Engineering Measurement Primer: A basic introduction to measurement concepts and use for systems engineering, V1.0, INCOSE, 1998.

[27] INCOSE Measurement Working Group (2010). Systems Engineering Measurement Primer: A basic introduction to measurement concepts and use for systems engineering, V2.0, INCOSE, 2010.

[28] Juglaret F, Rallo JM, Textoris R, Guarnieri F, Garbolino E (2011). Occupational health and safety scorecards: New leading indicators improve risk management and regulatory compliance. The 40th ESReDA Seminar-Risk Analysis and Management Across Industries, Bordeaux, France, May 2011.

[29] Kim SY, Huynh TA. Improving project management performance of large contractors using benchmarking approach. International Journal of Project Management, 2008, 26(7): 758-769.

[30] Kakar A, Thompson S (2010). A Case for using the balanced scorecard framework at project stage-gates. Proceedings of the Southern Association for Information Systems Conference, Atlanta, USA, March 26–27, 2010.

[31] Kueng P, Andres M, Wettstein T. Performance measurement systems must be engineered. Communications of the Association for Information Systems, 2001, 7(3): 1-27.

[32] Kerzner HR. Project Management Metrics, KPIs, and Dashboards: A Guide to Measuring and Monitoring Project Performance, 2011, Hoboken, New Jersey, USA: John Wiley & Sons, Inc..

[33] Keegan DP, Eiler RG, Jones CR. Are your performance measures obsolete?. Management Accounting, 1989, 70(12): 45-50.

[34] Kasser JE, Schermerhorn R. Determining metrics for systems engineering. INCOSE International Symposium, 1994, 4(1): 740-745.

[35] Knorr LC (2012). Leading Indicator Analysis for High Speed Sled Test Programs (No. AFIT/GSE/ENV/12-M03DL). Thesis for AFIT Distance Learning MS in Systems Engineering.

[36] Li HX, Zhao YM. International project management strategy applied by a small and medium-sized consultancy in oversea cooperation, 2016, Gothenburg, Sweden: Chalmers University of Technology

[37] Lauras M, Marques G, Gourc D. Towards a multidimensional project performance measurement system. Decision Support Systems, 2010, 48(2): 342-353.

[38] Lipke W, Zwikael O, Henderson K, Anbari F. Prediction of project outcome: The application of statistical methods to earned value management and earned schedule performance indexes. International Journal of Project Management, 2009, 27(4): 400-407.

[39] Luu VT, Kim SY, Huynh TA. Improving project management performance of large contractors using benchmarking approach. International Journal of Project Management, 2008, 26(7): 758-769.

[40] Meredith JR, Mantel SJ Jr. Project Management: A Managerial Approach, 2011, Hoboken, New Jersey, USA: John Wiley & Sons, Inc.

[41] Morris P, Pinto JK. The Wiley Guide to Project Control., 2010, Hoboken, New Jersey, USA: John Wiley & Sons, Inc.

[42] Mearns K. From reactive to proactive — Can LPIs deliver?. Safety Science, 2009, 47(4): 491-492.

[43] Neely A, Gregory M, Platts K. Performance measurement system design: A literature review and research agenda. International Journal of Operations & Production Management, 2005, 25(12): 1228-1263.

[44] Navon R. Research in automated measurement of project performance indicators. Automation in Construction, 2007, 16(2): 176-188.

[45] Orlowski C, Blessner P, Blackburn T, Olson B. A framework for implementing systems engineering leading indicators for technical reviews and audits. Procedia Computer Science, 2015, 61: 293-300.

[46] Pajares J, Lopez-Paredes A. An extension of the EVM analysis for project monitoring: The cost control index and the schedule control index. International Journal of Project Management, 2011, 29(5): 615-621.

[47] Parmenter D. Key Performance Indicators: Developing, Implementing, and Using Winning KPIs, 2015, Hoboken, New Jersey, USA: John Wiley & Sons, Inc..

[48] PSM, INCOSE (2005). Technical Measurement Guide, V1.0, San Diego, CA, USA: International Council on Systems Engineering (INCOSE). INCOSE-TP-2003-020-01.

[49] PMI (2013). Project Management Body of Knowledge (PMBOK® GUIDE).

[50] Rebentisch E. Integrating Program Management and Systems Engineering: Methods, Tools, and Organizational Systems for Improving Performance, 2017, Hoboken, New Jersey, USA: John Wiley & Sons, Inc..

[51] Rhodes DH, Valerdi R, Roedler GJ. Systems engineering leading indicators for assessing program and technical effectiveness. Systems Engineering, 2009, 12(1): 21-35.

[52] Roedler G, Rhodes DH (2007). Systems Engineering Leading Indicators Guide, V1. INCOSE Technical Product Number: INCOSE-TP-2005-001-03.

[53] Roedler G, Rhodes DH, Schimmoller H, Jones C (2010). Systems Engineering Leading Indicators Guide, V2. INCOSE Technical Product Number: INCOSE-TP-2005-001-03.

[54] Shi L, Newnes L, Culley S, Gopsill J, Jones S, Snider C (2015). Identifying and visualising KPIs for collaborative engineering projects: A knowledge based approach. ICED15, The 20th International Conference on Engineering Design, Milan, Italy, July 27–30, 2015.

[55] Sinelnikov S, Kerper S, Inouye J. Transforming EHS performance measurement through leading indicators, 2013, Itasca, IL: The Campbell Institute (24 pages)

[56] Sharon A, De Weck OL, Dori D. Project management vs. systems engineering management: A practitioners’ view on integrating the project and product domains. Systems Engineering, 2011, 14(4): 427-440.

[57] Thomas G, Fernandez W. Success in IT projects: A matter of definition. International Journal of Project Management, 2008, 26(7): 733-742.

[58] Trochim W. The Research Methods Knowledge Base, 2001, Cincinnati, Ohio, USA: Atomic Dog Publishing

[59] Wilbur A (1995). Metrics Guidebook for Integrated Systems and Product Development, International Council on Systems Engineering, INCOSE-TP-1995-002-01.

[60] XUE Rui, BARON Claude, ESTEBAN Philippe, SAHRAOUI Abd-El-Kader. Aligning systems engineering and project management standards to improve the management of processes. Progress in Systems Engineering, 2015, Cham: Springer International Publishing 547-553.

[61] Yang SL, Wang JM, Shi LY, Tan YJ, Qiao F. Engineering management for high-end equipment intelligent manufacturing. Frontiers of Engineering Management, 2018, 54: 420-450.

[62] Yun S, Choi J, De Oliveira DP, Mulva SP. Development of performance metrics for phase-based capital project benchmarking. International Journal of Project Management, 2016, 34(3): 389-402.

[63] Zhang Q, Lu XN, Peng ZL, Ren ML(2019). Perspective: A review of lifecycle management research on complex products in smart-connected environments. International Journal of Production Research, 1–F22.

[64] Zhao SY, Zhang Q, Peng ZL, Fan Y(2019). Integrating customer requirements into customized product configuration design based on Kano’s model. Journal of Intelligent Manufacturing: 1–17.

[65] Zidane YJ, Johansen A, Ekambaram A. Project evaluation holistic framework — application on megaproject case. Procedia Computer Science, 2015, 64: 409-416.

[66] Zheng L, Baron C, Esteban P, Xue R, Zhang Q (2017). Mapping systems engineering leading indicators with leading indicators in construction industry projects. 12ème Congrès International de Génie Industriel (CIGI 2017), Compiègne, France, May 2017.