Reaching consensus on the measurement of social value; challenges not to be overlooked.
Originally published on the Oxford University, Government Outcomes Lab Blog
Why we need to measure beyond Financial Value
Until we measure beyond financial value, we will continue to make decisions at the cost of our social, environmental and personal wellbeing.
“if we have declared success because we have a relatively high rate of GDP growth, why are the things that we value going backwards like child wellbeing, a warm, dry home for all, mental health services or rivers and lakes that we can swim in?"[1], launching the New Zealand Wellbeing Budget in 2019, Grant Robertson, Treasurer of New Zealand, acknowledged this was because these things “were not being sufficiently valued by the government. And because they were not being valued, they were not being measured, and because they were not being measured, they were not being done”.
To measure progress, we must measure financial value and social value. Without both, we may increase one at the cost of the other. Just as longitude is to latitude, we need social value and financial value to know that we are on track.
The utility of measuring social value is recognised as immense. The social value of an initiative is increasingly driving decision making across investment portfolios [2], consumer spending [3] and policy making. However, in the pursuit to reach consensus and standardize approaches, key technical and ethical challenges must not be overlooked.
Classification
In the pursuit of a standardised approach, measurement frameworks too often only classify impact instead of standardise the process to measure it. The Global Impact Investment Network (GIIN) asserts their IRIS+ as “the generally accepted system for measuring, managing, and optimizing impact” [4]. The system includes ‘a catalog of metrics aimed at increasing data clarity and comparability. However, comparability requires having a standard around the way the data is collected, the actual measurement. Another initiative, the Impact Management Project, has achieved consensus from 2,000 practitioners on a framework across the areas that impact should be measured: who, what, how much, contribution and risk [5]. Certainly, a great step in the direction to standardise understanding of impact, but for rigour and integrity, we must move to standardise the ‘how’ the measurement is done, across all stages of the data lifecycle; the design, tools, data collection, analysis and reporting.
Contribution
Impact measurement is moving from the discipline of scientific evaluation and research and into the hands of management consultants and fund managers, without understanding and respect for the power of measurement. In doing so, impact measurement has started to play a dangerous game with the lives of people. Instead of adhering to scientific discipline through the use of control groups and other counterfactual analysis techniques, often methods, such as Social Return on Investment (SROI), leave causation and attribution to be self-assessed by the service provider themselves or their consultants, using a methodology that incentivises them to inflate contribution.
Complexity
Life does not adhere to well thought-out linear logic models. Measuring outputs and outcomes can be misleading about progress overall. One outcome may be achieved at the expense of another. Instead, life is more accurately understood as a complex system, where all elements interact and impact on one another. “Complex systems tend to remain in a state of equilibrium, that is, they organize around a certain ‘attractor’”[6]. You could analogise that through capitalism we have set up economic growth as our system’s attractor. However, “when conditions change and the equilibrium is disturbed, the system will cross a threshold and move to another basin of attraction” [7]. Recent geopolitical, environmental and social unrest is certainly disrupting the financial system as we know it. Establishing wellbeing as the measure of social value is the opportunity to establish wellbeing as our new system attractor.
Combined Data
Across all development and social services there are underlying power relationships at play between those giving and those receiving; the terminology itself is an indication; ‘development’ and ‘beneficiaries’. But most critically, data is often lacking from the people directly impacted. Observing the living standards of another, especially someone in an environment foreign to our own, is ripe for misinterpretation of what matters for them. Consider yourself, likely you are an educated, relatively well dressed, fed, healthy person in a safe shelter. Does that mean you are in a strong state of wellbeing? The only person to accurately assess your state of wellbeing is yourself. Of course, subjective data does not always stand up alone. The ‘better-than-average effect’ is a type of social comparison, where people believe they are generally better than the average. Proven in countless studies and more recently shown to hold where people assess their own pro-environmental behaviours, considering themselves to be better at these behaviours then they actually are [8]. As in medical assessments, we require subjective data to understand the ‘symptoms’ and objective data to observe the ‘signs’; the two work together to diagnose the issue and identify treatment.
Conversion
One of the biggest drivers in developing social value methodologies is to assess the return on investment. Whether counting up the costs and benefits (CBA/CEA) or using financial proxies to determine cost savings (SROI), methodologies that conflate social value to something that can be measured in terms of financial value miss the point; social value does not accurately translate to financial value. A clear example is offered Australia’s Native Title Settlements. Courts pay out compensation to Native Title holder’s for the extinguishment of their claims, for both economic and non-economic loss due to being disconnected from their land. However, for Aboriginal and Torres Strait Islander people, land is their well-being. Therefore, adequate financial compensation, would surely be close to priceless. Yet, the biggest native title compensation pay out is a mere $2.53 million, of which $1.3 million was for ‘pain and suffering’ [9]. If you find that example too removed from your own world view, try and put a dollar figure on your health, your family, nature and your relationships. The value we hold for these things does not translate to a sum in a bank account. Instead, we need a measure of social value, separate to financial value, that is representative of what it means to us.
In the pursuit to measure social value we must not get carried away and overlook and oversimplify how this is to be achieved. Otherwise we risk establishing a new economy that may create as many issues as it seeks to address.
If you are interested to know more about Huber Social, we are running a virtual Accreditation course in September 2021. You can also contact Georgie directly on georgina.camp@hubersocial.com.au
Footnotes
1. Hon Grant Robertson (Minister Of Finance), New Zealand Parliament, Budget Statement, Second Reading, Sitting Date 30 May 2019, Available Here: Https://Www.Parliament.Nz/En/Pb/Hansard-Debates/Rhr/Combined/Hansdeb_20190530_20190530_082. Larry Fink’s Letter to CEOs, A Fundamental Reshaping of Finance, available here: https://www.blackrock.com/corporate/investor-relations/larry-fink-ceo-letter3. “over 66% of global consumers say they’re willing to pay more for sustainable brands”, Neilsen Global Connect, Sustainability Imperative Report, available here: https://www.nielsen.com/us/en/insights/report/2015/the-sustainability-imperative-2/4. https://iris.thegiin.org/5. https://impactmanagementproject.com/impact-management/what-is-impact/6. Nino Antadze & Frances R. Westley, Impact Metrics for Social Innovation: Barriers or Bridges to Radical Change?, Journal of Social Entrepreneurship Vol. 3, No. 2, 133–150, October 2012, pp47. Ibid. 8. Magnus Bergquist. Most People Think They Are More Pro-Environmental than Others: A Demonstration of the Better-than-Average Effect in Perceived Pro-Environmental Behavioral Engagement. Basic and Applied Social Psychology, 2019; 42 (1): 50 DOI: 10.1080/01973533.2019.16893649. Northern Territory v Mr A. Griffiths and Lorraine Jones on behalf of the Ngaliwurru and Nungali Peoples [2019] HCA 7