Last month, OPEN attended a fantastic webinar hosted by SIMNA (Social Impact Measurement Network Australia), where Victoria Baird from Mission Australia spoke about their implementation of an impact measurement framework across the organisation.
Victoria shared lessons for how to successfully embed measurement practices across a large organisation and a diverse range of programs – and in doing so, create an organisational culture of learning and innovation.
This is a great example of an organisation that is strengthening its evidence culture, systematically collecting data and using outcomes to improve delivery across every project.
Mission Australia exemplifies many features that are known to boost success in transforming an organisation’s way of working – including:
At Mission Australia, impact measurement has been embedded within most of their programs, and is considered business as usual.
This is no small undertaking – Mission Australia has an impact measurement team of just two people, but the organisation tracks the effects of around 80% of its 500 separate programs across Australia.
To achieve this, they work closely with program staff to implement an impact measurement process that works for that team and their clients.
Mission Australia uses the Personal Wellbeing Index (PWI) as the basis for its impact measurement surveys. This validated survey tool has been chosen because it works well for many different services.
As it measures wellbeing, the PWI focuses on a key desired outcome common to all social services: to improve how clients feel. It is also quite a short survey (7-8 questions) which means that additional service-specific questions can be added without making the survey too arduous for clients to complete.
This survey is workshopped with the program team to develop a measurement tool that suits the needs of each program and its clients. The tool is then used with clients, and the data is entered anonymously into a central system where it is analysed and fed back to the program team. The team can then use the results to track the wellbeing of their clients, and consider where they might adapt the program to better meet client needs.
Initially, it took about 6 months to get a new service set up to use the impact measurement tool. Now, the process has been streamlined and takes 6-8 weeks for a new service, and sometimes as little as 2 weeks for a service that is very similar to one already implemented.
This project showcases a number of valuable lessons about how to implement a measurement framework in a large organisation in a way that has meaning to staff and clients, and promotes a culture of continuous learning and improvement.
The Mission Australia model allows program teams to have a direct connection to impact measurement. Having a team member act as an impact measurement champion not only allows the individual to build their professional skills, it also means that they understand and communicate the value of this measurement practice to others in their team.
The effect is that impact measurement is placed in a context of what is relevant for this team and their clients; it is not something that is handed down from a different part of the organisation and becomes a matter of compliance with company policy. This way, it has meaning to the individual teams and they can understand the purpose of it.
Although the impact measurement at Mission Australia is based on a common metric (the Personal Wellbeing Index) the tool is tailored to suit the requirements of each program and population of clients.
This serves two purposes:
It is important not only that staff understand why they are collecting data, and that they are given the chance to contribute to what type of data they collect, but also that they get to see the results.
Mission Australia’s social impact team have invested significantly in technology that allows data to be reported back to teams quickly and comprehensively, so that they can track their clients’ wellbeing across time. This makes the data they collect meaningful and useful, and can lead them to pursue new directions for their programs.
The Mission Australia impact measurement team emphasises the need to make their surveys accessible to clients, and to ensure that the process of completing them is ethical, consensual and does not become burdensome. This means that they are flexible about how surveys are administered (e.g. verbally, online, or via some other method) and how often they are conducted. They are also careful to ensure that the client’s data is entirely confidential.
Part of what makes this work is that when staff buy in to the importance of impact measurement, they tend to communicate this to their clients as well, and everyone involved in the program understands the purpose and value of collecting this data.
The impact measurement team emphasises to all the staff they work with that the data collected during this process should be seen as measuring how their clients are faring, rather than analysing their effectiveness as practitioners – especially as individual wellbeing is affected by so many factors outside any worker’s control.
Contextualising the data in this way makes workers more motivated to collect authentic responses on the wellbeing of their clients, as they know they are not going to be held accountable for any dips in wellbeing responses. In turn, staff can communicate this to the clients that they are surveying, allowing the responses to be an authentic snapshot of how they are faring at a given time.
OPEN is currently developing materials that describe the key features of a strong organisational evidence culture – so stay tuned.
As part of this work we are seeking to showcase examples of service providers that have introduced an organisational approach to outcome measurement.
Please get in touch if you are interested in having your work included. Contact OPEN Manager Mandy Charman at mandy.charman@cfecfw.asn.au
