Since the last version of the Evidence-Based Management (EBM) Guide was published in 2020, significant updates have been made to enhance the clarity and applicability of EBM. The updates in the new 2024 version are based on new insights and learnings since the 2020 version. The fundamental concepts of EBM remain unchanged, focusing on using empirical evidence to improve value delivery under conditions of uncertainty. This blog post delves into the primary differences between the 2020 and 2024 versions of the EBM Guide, highlighting the new insights and clarifications that have been incorporated.
You can download the latest version of the EBM guide here.
Evidence-Based Management?
Wait, what is EBM again?
EBM is a framework that uses empirical data and intentional experimentation to help organizations make better-informed decisions about how to achieve their goals. The core principles of EBM include setting measurable goals, conducting experiments to test hypotheses, and using the results of these experiments to guide future actions. EBM focuses on improving value delivery in uncertain conditions by continuously assessing and adapting evidence-based strategies. To explain how organizations can effectively implement the EBM framework, the first version of the EBM Guide was first published in 2015 and updated in 2018, 2020 and 2024. It includes detailed instructions for establishing measurable goals, conducting experiments, and utilizing feedback to drive continuous improvement.
Comparing the 2020 ⏪ and the 2024 🔄 version
Purpose and focus
- ⏪ The 2020 EBM Guide emphasized “Measuring Value to Enable Improvement and Agility“, as it provided a framework for organizations to continuously improve customer outcomes, organizational capabilities, and business results through empirical methods.
- 🔄 The 2024 update shifts the focus slightly with the subtitle “Improving Value Delivery under Conditions of Uncertainty.” This change reflects a more nuanced understanding of EBM’s mission, emphasizing the dynamic nature of value delivery in uncertain environments. This shift underscores the need for agility and adaptability in response to changing conditions.
Mission, vision and goal-setting
- 🔄 One of the significant additions in the 2024 Guide is the inclusion of Mission and Vision as foundational elements for framing goals. Strategic goals are now explicitly linked to an organization’s mission and vision, providing a concrete context for their development and pursuit.
- 🔄 The 2024 Guide recognizes that large organizations may have multiple strategic goals, each contributing to the overarching mission in different ways. This reflects a more complex organizational structure and the various paths to achieving long-term goals.
- 🔄 The 2024 Guide provides specific examples, such as the response to an infectious disease, to demonstrate the practical application of goal-setting principles. For instance, strategic goals in combating a disease might include reducing infection rates, with intermediate goals like successful vaccine trials, and immediate tactical goals like isolating symptoms and sequencing viral DNA.
Measuring inputs and impacts
- ⏪ The 2020 Guide focused on activities, outputs, and outcomes as measures of organizational performance. It stressed the importance of measuring outcomes to ensure value delivery to customers.
- 🔄 The 2024 update introduces inputs and impacts as additional types of measures. This broader categorization assists organizations in understanding the full scope of their activities and their ultimate impact on strategic goals. Recognizes costs and impacts such as profit or market share as important, though not primary goals. For example, tracking the cost of experiments (inputs) alongside their impact on market share provides a fuller picture of value delivery.
Key Value Areas (KVAs)
- 🔄 KVAs focus on Current Value (CV), Unrealized Value (UV), Time-to-Market (T2M), and Ability to Innovate (A2I). These areas provided lenses to examine the organization’s performance and potential for improvement. The descriptions of the Key Value Areas (KVAs) have been clarified in the 2024 Guide to provide better understanding and application. The relationship between KVAs and types of measures (inputs, activities, outputs, outcomes, and impacts) has been added, providing a more integrated approach to assessing organizational performance.
- Current Value (CV): the updated version provides more detailed questions for organizations to evaluate their current value, such as customer and employee happiness. For instance, asking “How satisfied are our customers with recent product updates?” helps gauge current performance.
- Unrealized Value (UV): the updated version adds more detailed questions and examples, helping organizations better assess their potential market share and the effort required to capture additional value. Questions like “What untapped markets could benefit from our product?” are now included.
- Ability to Innovate (A2I): the updated version includes new measures like employee engagement and provides more examples of impediments to innovation, such as poor product quality and lack of decentralized decision-making. It includes measures like “What percentage of our time is spent on new product development versus maintenance?”
- Time-to-Market (T2M): the updated version details specific measures, such as build and integration frequency, release frequency, and lead time for changes, offering a more granular approach to assessing and improving T2M. Questions like “How quickly can we release new features to users?” are emphasized.
The Experiment Loop
- ⏪ The 2020 Guide described the process of using the Experiment Loop to make progress toward goals through small, iterative steps.
- 🔄 The description of the “Experiment Loop” has been refined in the 2024 Guide to better illustrate the process of forming hypotheses, running experiments, inspecting feedback, and adapting accordingly. This is crucial for emphasizing the iterative nature of EBM and the importance of continuous improvement based on empirical evidence. The updated guide includes clearer steps and more detailed examples of successful experiment loops.
New categorisation of measures
- ⏪ The 2020 version categorizes measures into activities, outputs, and outcomes. It highlighted the challenges organizations face in measuring outcomes compared to activities and outputs.
- 🔄 The 2024 version expands this categorization to include inputs and impacts. This broader framework enables organizations to gain a more comprehensive understanding of their performance, from resources invested (inputs) to the broader effects on stakeholders (impacts). This helps organizations align their investments with strategic outcomes more effectively.
Conclusion
The 2024 update to the Evidence-Based Management Guide reflects a more thorough understanding of EBM principles. By incorporating new concepts, clarifying existing ones, and providing more detailed examples and measures, the updated Guide provides a more robust framework for organizations seeking to improve value delivery in uncertain environments. Whether for strategic planning or day-to-day operations, the enhanced EBM Guide provides organizations with the tools they need to make informed decisions, continuously improve, and achieve long-term objectives.