Measuring Impact Resilience — What We Are Learning from BUILD UPON Cambridge, Madrid, and Brussels

In this 3rd of a series of 4 blogposts, we share what we are learning, as co-investors with BUILD UPON and the European Climate Foundation, about: (1) co-hosting collaboration; (2) realizing the deeper shared purpose; (3) measuring impact resilience; and (4) scaling impact.

MEASURING IMPACT RESILIENCE

The potential we see.  With a shared deeper purpose that brings many diverse stakeholders together, across many countries and industries, the BUILD UPON effort is now ready to ground this abstract purpose, making it a daily reality for all of these stakeholders, showing how they can work together, from their own worlds, in achieving something bigger, locally, nationally, and regionally.  Measurement can support that grounding process, by clarifying:

  1. what we mean by our shared purpose, in very specific terms
  2. how we plan to achieve our purpose, with the unique contribution of each of our efforts
  3. how we can identify and highlight what we are each learning in the development of technical and social innovations

In many organizations, we measure because we are expected to measure, and we manage to the measures.  The reason to measure and manage to the measures is given, by someone else.  While the apparent simplicity of some measures might make them seem straightforward, they rarely are.  For example, profits are revenues minus costs.  For more profits, increase the revenues and decrease the costs.  Simple, until the methods for increasing revenues increase costs faster, or the ways to decrease costs also decrease revenues, such as lower product or service quality.  Or, to decrease the environmental impact of buildings, increase the number of renovated buildings, by increasing demand.  You can do this by making people do it through regulations or by making it cheaper to do through subsidies.  Simple, until the regulations make the technologies much more expensive or the subsidies outpace the supply.

From an impact resilience perspective, we are looking for a more systemic understanding that integrates the multiple stakeholder perspectives on the impacts we are trying to achieve.  With this integrated perspective, we can measure a systemic set of indicators that let us know how resilient we are in achieving our systemic set of impacts.  

To compensate for a lack of clarity of what we are trying to achieve amongst many stakeholder perspectives, we tend to believe that lots of measurement — lots of variable and lots of data — shows that we are very serious.  We measure dozens to hundreds of key performance indicators (KPIs).  To measure lots of variables, we tend to focus on easier to measure variables that are often means to an unspecified, higher-order purpose of the whole effort.  We also leave measures of creativity and collaboration out of the equation, since they seem hard to measure, yet they are critical to the generative processes of creative collaboration.

In the development of a scorecard for impact resilience, we look for a small set of measures that cover the higher-order impacts we want, the strategic areas we want to impact, and the processes that leverage our impact.  While this small set of 10-15 measures can be supported by more detailed reports, specifying how they were achieved in more detail, we want to focus on the most strategic variables and our narrative—our theory of impact resilience—of how they all fit together.  We want to use proxies that directly let us know how we are doing on the strategic variables.  In the impact resilience scorecard, we attempt to do this.

“Committing to measuring process, strategic, social impacts on local, national, international levels would allow the stakeholders joining multilevel collaborative platforms to see the whole and the meaning of their (and others) contribution, but also to improve the way those platforms can work together and, so, achieve greater impact. Sharing simple, meaningful proxies, would allow necessary flexibility, and to get a clear sense of the final purposes all over a complex group of groups. Moreover, and crucially, learnings from best practices could grow geometrically with the network’s dimensions, increasing the ability to be effective, and resilient over time.”

— Sebastiano Cristoforetti, International and Certification Manager, GBC Italia

What we might do.  To assess social, strategic, and process impacts across Europe, at the local, national, and regional levels simultaneously, we can develop a coherent set of a few measures that we can track to identify the common impacts and the specific innovations happening at each level.  As we saw in an earlier blog in this series [link to blogpost “Realizing the Deeper Shared Purpose,”] we developed a draft “Common Vision” with a wide-ranging group of stakeholders across the energy efficiency, renewable energies, and building renovation communities.  The following systems map captures the key elements of that Common Vision, showing how they all fit together (see the blogpost “Realizing the Deeper Shared Purpose” for a description of the systems map).

As these collaborative-process efforts drive the strategic areas and subsequently the social impacts, the growing demand and supply for renovation drives a scaling factor that accelerates social and technical innovation.  Having the clarity of the purpose we share and the dynamics of our system, we can focus the measurement scorecard on the critical variables that express the impacts we want to achieve together.

To measure these social, strategic, and process impacts, we can simplify the work of BUILD UPON into 12 high-level metrics, which can then be supported with detailed metrics, showing systemically how they influence each of the high-level impacts.

The metrics in an Impact Resilience Scorecard, exemplified in the figure above, highlight the social, strategic, and process impacts of a system that leads to greater resilience of the renovation system, at the local, member state, and European levels.  Proxies for each impact measure are provided, which could also include current levels and agreed-upon target levels.

After being exposed to this systemic and high-impact approach to measuring individual and collective success, the participants were asked to reflect on, “What would I need to know and share to fully step up and invest in the future we started to see together?”  The participants shared the need for more knowledge on positively deviating behavior of other members, to be able to increase their own success. Furthermore, they highlighted the importance of continuing the conversation on a strategic level to support their ability to increase their impact and strengthen their resilience. See movie footage here of what they shared.

What could happen.  Much greater collaboration is possible, and it means that the many stakeholders involved in energy efficiency, renewable energies, and building renovation have to see the value in it.  It has to become more than a nice exercise; it has to be interwoven into their ability to succeed individually and together. Collaborative impact is a simple choice, not a complicated option.  A choice we can make every day.  Like the groups we are finding around the world, members of BUILD UPON too can choose their experience and the outcomes they  achieve.
We thank our colleagues at the European Climate Foundation (ECF), the BUILD UPON team, the World Green Building Council, the co-hosts, the Madrid and Brussels participants, the Institute for Strategic Clarity, and Vibrancy—all co-investors in this process together.

Advertisements

Why We Care About the Resilience of Our Agreements — What We Lose When Our Agreements Collapse

Everyone lives in complex, turbulent times.  Will our agreements survive the changes we face?  How resilient are these agreements?  We can look to ecologists for how to think about the resilience of systems and to anthropologists for what has actually happened in human systems.

From earlier work by the ecologist C.S. Holling and colleagues, as described by the Resilience Alliance, “When resilience is enhanced, a system is more likely to tolerate disturbance events without collapsing into a qualitatively different state that is controlled by a different set of processes.”  From an ecosynomic perspective, this means that resilience is the ability to keep a similar level of agreements, meaning the levels of perceived reality they consciously include.  A collapse is then a qualitative shift in the level of agreements.

Anthropologists, like Joseph Tainter, have looked at societal collapse, finding, “The process of collapse..is a matter of rapid, substantial decline in an established level of complexity. A society that has collapsed is suddenly smaller, less differentiated and heterogeneous, and characterized by fewer specialized parts; it displays less social differentiation; and it is able to exercise less control over the behavior of its members . It is able at the same time to command smaller surpluses, to offer fewer benefits and inducements to membership; and it is less capable of providing subsistence and defensive security for a regional population” (Tainter, 1988 pp. 38).  An example of a loss of a level of complexity might be the loss of consciously accepted agreements at the level of the development of capacities and relationships–the verb level–to focus solely on the level of outcomes–the noun level.

Thus, ecologists and anthropologists observe that a more resilient set of agreements is more capable of dealing with changing environments without losing whole levels of complexity in the agreements.  You can find more on the ecosynomics of impact resilience here.

 

Measuring Your Impact Resilience

Impact.  Resilience.  The impact you want to have in the world, as a result of your efforts.  The resilience in the ability to respond to internal and external changes, over and over again, sustainably.  We all seem to want greater impact resilience, yet most efforts seem to lead to low impact, with most efforts failing to achieve the desired impact, and people being less engaged after the effort than before it.  To compensate for the low results and engagement of the efforts, they have high direct costs.  Not the benefit-to-cost ratio most of us foresaw when starting the efforts.

Might a measure of impact resilience help, before, during, and after?  The current mainstream framing of impact resilience focuses on net profits or funds available from the effort.  Essentially, the direct benefits should be greater than the direct costs.  Profits = Revenues – Costs.  Funds Available = Funds In – Funds Out.   This kind of logic leads to the prevailing framing of strategy as the direct interventions that will lead to direct outcomes, often called a “theory of change.”  In explorations my colleagues and I have made into the agreements supporting the very high impact resilience of positive deviants we have found around the globe, we find an alternate framing, which seems to lead to much higher impact resilience.  We call this alternate framing a “theory of impact resilience,” where the focus is on the ability to engage the potential value present in any group, in a very resilient manner.

The measure of impact resilience, as we are using it today, encompasses:

  • impact:  what we want to achieve, the potential value available, the costs of scarcity, and the ecosynomic value realized in service of what we want
  • resilience: the ability to thrive in change, continuously, over time

We use three specific tools to measure the impact and two to measure the resilience.  We assess impact with the tools of (1) deeper shared purpose, (2) reference behavior pattern, and (3) ecosynomic value realized (EVR). We assess resilience with the tools of (1) probability of survival, and (2) the multiples of EVR.

Impact tools.  The deeper shared purpose is the reason why the group comes together in the first place and why it needs a specific mix of voices.  The process for the “deeper shared purpose” tool is described in the O Process.  The reference behavior pattern explores the group’s definition of how the deeper shared purpose is measured, how well the group has done at achieving it historically, the most probable outcomes of the deeper shared purpose going forward, the desired outcomes going forward, and the gap between the most probable and the desired outcomes.  The process for the “reference behavior pattern” tool is described in my chapter applying the tool to poverty alleviation.  Ecosynomic Value Realized (EVR) is the total value realized minus the costs of the utilized resources minus the costs of scarcity.  Said another way, EVR is the total value generated by the recognized resources less the costs of the recognized resources less the costs of the unrecognized resources.  The cost of the unrecognized resources is the total potential available in the available resources, as described through the three levels of perceived reality in an agreements evidence map, less the value of the recognized resources.  This accounts for the costs of not engaging the potential resources available–the costs of scarcity. The process for the EVR tool is described in the Costs of Scarcity framework.  We use the combination of these three tools to determine (1) what we are trying to achieve together–the deeper shared purpose, (2) how we are doing at achieving that impact, and (3) net results in value realized through our efforts.

Resilience tools.  The probability of survival is the probability that the group will continue to have sufficient resources to survive in the future.  Most initial efforts never even get off the ground, and most efforts that do, die within the first years.  This means that the probability of survival for most efforts is very low.  Resilience is the ability to increase the probability of survival.  The probability of survival is the average of the probability of survival for each of the three levels of perceived reality: the risk of stockout at the outcomes level; the risk of not learning at the development level; and the risk of obsolescence at the potential level.  The risk at each level depends on the level of conscious agreements at each level of perceived reality.  Where the agreements are conscious, the probability of survival (one minus the risk of not surviving) is much higher than where the agreements are subconscious, unconscious, or non-existent.  The multiples of ecosynomic value realized convert the probability of survival into a number of probable years of survival, which when discounted over time suggest a multiple of this year’s ecosynomic value realized (EVR).  This multiple times the current EVR suggests a valuation of the current set of agreements of what is valued and engaged, as seen through the agreements evidence map.  We use the combination of these two tools to determine (1) the probability of survival of the agreements in place, and (2) a valuation of the probable lifetime of the agreements.

With the measures of impact and resilience, we have a better sense of (1) the current state of the agreements, (2) the benefits of shifting the agreements, and (3) the costs of not.  We can also assess how the set of agreements compare to other sets of agreements, indicating both what is possible for groups and where to invest for greater impact resilience.  We do this assessment through the five levels of impact resilience.

The five levels of impact resilience range from simply achieving some impact over time to generating great impact resilience by engaging all of the potential value available.  The Institute for Strategic Clarity has set up a certification process for each of the five levels of impact resilience.  Level 1 Impact Resilience is achieved when a group is able to demonstrate that is has achieved its stated impact over five years.  Level 2 is achieved when a group achieves both Level 1 and measures its impact resilience, as described above, independent of whether its EVR is positive or not.  Level 3 is achieved when a group achieves Level 2 and its EVR is net positive.  Level 4 is achieved when a group achieves Level 2 and its Return on Potential Value (RPV = EVR/Total Potential Value) is greater than 0.3, meaning its conscious agreements are well into the development level of perceived reality.  Level 5 is achieved when a group achieves Level 2 and its RPV is greater than 0.6, meaning its conscious agreements are well into the potential level of perceived reality.

Coming back full circle, we find that groups that are able to achieve the higher impact resilience every group imagines, initially–yet few groups actually achieve–score much higher on impact resilience.  By examining what differentiates high impact resilience groups from lower impact resilience groups, we have developed the impact resilience measurement system.  Groups that want to know where they are in their impact resilience, with the desire to achieve much greater impact resilience, can now assess the specifics of what supports their current levels and what agreements are needed to achieve higher levels of impact resilience.  Those groups who are able to demonstrate that they can meet the higher standards of impact resilience can be recognized by impact resilience certification.  This provides that group with a cohort of groups at their level of impact resilience, mentors for the next level, and certification for possible investors and donors of the quality of their agreements in achieving higher impact resilience.

 

3D Tangibilization of Impact Resilience — Recommended Reading

Duckworth, Angela. Grit: The Power of Passion and Perseverance2016, New York: Scribner.  Click here for an excerpt.

Duhigg, Charles.  Smarter Faster Better: The Secrets of Being Productive in Life and Business.  2016, New York: Random House.  Click here for an excerpt.

Dweck, Carol S. Mindset: The New Psychology of Success. 2006, New York: Ballantine Books.  Click here for an excerpt.

To make something real, to make it tangible–to tangibilize–takes a potential, an idea that was seen, and a pathway for developing the potential, the possibility, into an outcome.  To tangibilize is the pathway of seeing the possibility, developing it, and completing it.  This is the creative process.  And while this seems obvious, because we all live this process all day long every day, most of the agreements underlying how we interact within and amongst organizations focus predominantly on the outcome, and very little on the potential and the developmental pathway.

And, there are positive ecosynomic deviants who are very focused on this process and its subtleties.  We know about these positive ecosynomic deviants mostly because of the results they achieve–results that shock us, leading us to assume that these outliers are superhuman, achieving extraordinary results unavailable to the rest of us.  They are able to achieve these results, over and over again, resilient to difficulties on their path, over sustained stretches of time.  I recommend three books I have read recently that uncover different dimensions of these positive deviants: their growth mindset; their ability to persevere; and their ability to be much more productive.  Written by three easy-to-read writers, who bring together great stories with lots of rigorous research–all Ds–Duckworth, Duhigg, and Dweck–they dig into what makes ordinary people capable of achieving high levels of impact and resilience.

Stanford psychology professor Carol Dweck shares her vast research on mindsets, showing how having a growth versus a fixed mindset enables people to learn, to grow.  Her work shows that the fixed mindset–“believing that your qualities are carved in stone” (Dweck 6)–is predominant, socially embedded in many of our unconsciously accepted agreements.   People with the growth mindset–“the belief that your basic qualities are things you can cultivate through your efforts…believe that a person’s true potential is unknown (and unknowable); that it’s impossible to foresee what can be accomplished with years of passion, toil, and training” (Dweck 7).  With many examples, Dweck shows how “the view you adopt for yourself profoundly affects the way you lead your life” (Dweck 6); full of possibility and growth or not.

University of Pennsylvania psychology professor Angela Duckworth focuses on what drives people to find their passion and stick with it to find extraordinary levels of expression of that passion.  Duckworth sets the tone of what is possible, quoting William James, a founding father of modern psychology, “‘Compared with what we ought to be, we are only half awake.  Our fires are damped, our drafts are checked.  We are making use of only a small part of our possible mental and physical resources.’  There is a gap, James declared, between potential and its actualization…James asserted that ‘the human individual lives usually far within his limits; he possesses powers of various sorts which he habitually fails to use.  He energizes below his maximum, and he behaves below his optimum‘” (Duckworth 22-23).  Bringing together years of her own experimental research and field experience with that of many of her colleagues, Duckworth has nuanced what grit is–“perseverance and passion for long-term goals“–what it takes to achieve it, and what it does.  Summarizing all of this research, she finds that grit requires a passionate interest, connection to a higher purpose, lots and lots of practice, and an optimistic perspective that deliberate practice will lead to something.

New York Times reporter Charles Duhigg delves into the contextual factors that lead to some people being able to be much more productive than others. He finds that productivity paragons are much more motivated, focused, and tend to work in groups of psychological safety.  “To teach ourselves to self-motivate more easily, we need to learn to see our choices not just as expressions of control but also as affirmations of our values and goals” (Duhigg 31).  “To become genuinely productive, we must take control of our attention; we must build mental models that put us firmly in charge” (Duhigg, 102).  When people are in groups that “feel a sense of psychological safety…[they] succeed because teammates feel they can trust each other, and that honest discussion can occur without fear of retribution” (Duhigg 69).

The 3 Ds highlight decades of rigorous research and experience that show that the people we most see as exemplars of extraordinary outcomes often are actually ordinary people who have found their passion, spend lots of time applying it, growing in their craft over time, in a very focused, hopeful way, with lots of support from their community.  This might mean that what is out of the ordinary, as the William James quote above suggests, is not that people live into their potential, rather that more people don’t.  What is it that we as humans do in our agreements that shuts down the process of tangibilizing our own unique impact resilience, every day?  I call this negative ecosynomic deviance, and explore what it takes to stop doing it.