In recent years, impact measurement has shifted from being a nice-to-have to a necessity – especially for publicly funded organisations.
In an increasingly competitive funding market, impact measurement allows organisations to demonstrate their social value to key stakeholders, such as funders and the communities they support. It enables organisations to position themselves as leaders amongst a sea of competitors and helps them to remain accountable.
Even more importantly, impact measurement empowers organisations to improve service delivery by allowing them to gain a deeper understanding of what is actually moving the needle. From here, organisations can make data-driven decisions about service improvement, resourcing and strategic priorities, ultimately elevating their services and delivering more impact for their users.
At ReachOut, we are driven by a commitment to maximise our impact and make a difference in the lives of young people in Australia.
ReachOut is a leading youth mental health organisation in Australia, with over 25 years of experience delivering free digital mental health and well-being resources for young people (and their families), using a strengths-based prevention and early intervention approach. Our purpose is to be a trusted and safe digital space, empowering young people in Australia to feel better. To do so, we have to target our services in ways that can achieve the biggest impact on young people and those who support them that we can, within our unique digital model.
In 2021, we embarked on a journey to revise and uplift our approach to impact measurement. And in 2022, we launched a bespoke model and framework for measuring the social value of our work.
The purpose of this social impact measurement project has been twofold. Firstly, we wanted to demonstrate that our services are effective in supporting the wellbeing of young people, to support our accountability, not just to our funders but also to our service users. Secondly, we wanted to develop a suite of data touchpoints to facilitate data-driven decision-making. By drawing on insights and evidence to inform our strategy, we would be better equipped to uplift our services, identify and prioritise new opportunities for innovation, and ultimately strengthen our impact.
However, the path to measuring and reporting our impact hasn’t been without hurdles. The digital mental health sector–and ReachOut specifically–faces a unique set of challenges that can make social impact measurement complex.
ReachOut provides web-based self-help and non-clinical mental health support that is accessed anonymously by our users. Because we don’t know who our ‘users’ are, it can be difficult to measure and test user outcomes. We are unable to engage in pre and post-testing using validated tools, without known ‘clients’. This anonymity is a drawcard for our users and a key reason why young people and their networks seek us out. We needed to work within these constraints when developing ways to measure our value, to ensure that we were not compromising how young people engaged with our digital products and services.
Additionally, as a digital service, we have access to a broad range of output data through digital analytics. We needed to sort the signal from the noise to identify data that explores and captures our social value as a digital service, recognising that digital analytics provide uniquely valuable information about the value of digital supports for their audiences. We realised it was important to re-imagine the connections between digital output metrics and outcomes, seeing things in a different way to that which is usual in the outcomes measurement sector. We also recognised the need to move beyond some of the ‘vanity metrics’ in the digital space, if we were to grow our impact maturity.
Alongside these factors, we wanted to align our new approach to impact measurement with key Principles of Social Value, in particular, making sure we were not overclaiming what our digital platforms might achieve for our audiences, given their purpose and function in the complex mental health eco-system. This was critical to ‘keeping it real’.
It is against this backdrop that we developed a unique and innovative approach to impact measurement for digital services.
ReachOut’s Social Impact Framework
There has been little publicly available work undertaken on social impact measurement that responds to the unique and complex nature of ReachOut’s service delivery as an anonymous, digital, self-help service.
With this in mind, we developed a bespoke approach to measuring impact that is captured in our Social Impact Framework (SIF). This framework was developed collaboratively by the ReachOut team, involving members of the Research and Impact team, our data analysts, service designers, product developers, marketing, and our digital service teams including our peer workers and online moderators. Each of these teams had been using organisational data and had a pre-existing commitment to engagement with our social impact practice. Through cross-team workshops, we cemented our understanding of the importance of not overclaiming changes for our users. We focused on what might logically result and be achievable through brief psycho-educational non-clinical support that meets young people where they are, in online spaces.
We also listened to young people, to ensure that our social impact framework reflected their needs and views about what ReachOut should deliver. We included our Youth Ambassadors in our all-day cross-organisational workshop, where we developed our Theory of Change and outcome domains, and sense-checked the SIF with young people volunteering with ReachOut.
The framework includes a specialised set of indicators that represent key metrics for anonymous digital services such as ours and provide clarity about the positive changes that happen for our users.
Our SIF brings together three components that we can measure to tell our impact story – reach, engagement and outcomes. (Fig. 1). These three elements capture the nuances of our service; entirely digital, anonymous, non-clinical and with a focus on self-help.
The framework, and the Theory of Change underpinning it, identify six outcome domains or areas of positive change: mental health literacy, connection, better understanding of self, improved sense of agency, immediate relief from negative feelings and validation.
Our Inaugural Social Impact Report
We have recently launched our first Social Impact Report based on the new framework. The data has been captured through digital and social analytics services, user surveys (both site pop-ups and annual surveys) and direct live feedback from our one-to-one digital support users via a built-in youth-friendly widget.
Our Social Impact Report demonstrates how we are reaching those who need us, how users are engaging with the support options we offer, and how they are experiencing positive outcomes through using our products and services. These initial results show that young people are experiencing positive changes following the use of our web-based products and services.
What have we learnt from this experience?
Reflecting on our social impact measurement work at ReachOut, there have been several key learnings that have helped us to continue to deepen our practice. These are:
1). Social impact measurement is a journey, not a destination. It takes time to build impact literacy and capacity, to ensure the right resources are committed to data collection, and to develop relationships for cross-organisational work. We had to prioritise the development of infrastructure, new roles and processes of data collection. Seeing it as a longer journey has allowed ReachOut to move towards making data-driven decisions and ‘managing to impact’, not just producing reports.
2). It takes courage to ‘own’ your impact measurement journey, and not wait for funders to frame your approach for you. We decided to develop our own model of social impact based on our deep knowledge of the organisation and our connections to young people.
3). To get started quickly, we used as much as we could of what we were already collecting. Starting like this, our SIF was an ‘MVP’ that we could test without creating too much of an administrative burden for our teams, or unreasonably interrupting the user experience of our digital products. We learned the importance of not trying to measure everything.
4). Having strong and consistent internal messaging and communications strategies are critical to maintaining commitment across busy teams whose work might be impacted by social impact measurement. This helped to support the building of genuine collaborative relationships around both developing our indicators and building capability for making data-driven decisions. We held regular cross-team meetings, and workshops, presented at all staff meetings and used short videos and our organisational confluence pages to keep our social impact work at the forefront of our organisational planning and understanding.
ReachOut has recently launched its 2022-2023 Social Impact Report. Contact the Research and Impact team at research@reachout.com for more information or visit ReachOut to learn more about this incredible organisation.
