Event Reflection – Outcomes for Grantseekers 102

Date: 21/2/2023

Location: Online

Presenter: Jen Riley, Smarty Grants, Funding Centre and Our Community.

Overview: Are you ready to measure your outcomes like a pro? OurCommunity has delivered two sessions on getting started with Outcome measurement.

Check out the first instalment “Outcomes for Grantseekers 101” here!

The second session explored different approaches to measuring outcomes including the use of metrics (systems of measurement) to better measure your program outcomes, and the criteria for selecting effective outcome measures.

This session focussed on:

How best to measure your program and its outcomes – should you use existing frameworks or create your own?

• How do we know if a metric is suitable for our specific organisational needs, circumstance and capabilities?
• The self-build: how to construct your own tools for capturing outcomes effectively?
• How best to represent your data to show your progress?

KEY MESSAGES

The overarching key messages of the session emphasised the importance in community services of measuring service and program’s outcomes and/or impacts.

How do I choose the most effective metrics for my outcome? – Metrics and datasets measuring impact should always be assessed for their SMARTness.  Smart measures are  whether it is:

• Specific
• Measurable
• Achievable
• Relevant
• Time-bound.

Where do I find effective metrics (standard of measurements) to measure my impact? – In existing frameworks or you can develop them, in-house!

Existing Frameworks – There are benefits of using a published outcome framework (gives credibility, comparability, evidence-based, benchmarking, saves time, may also suggest a dataset) – but the framework must fit what you are trying to measure, and it must be SMART! Consider existing frameworks from published outcome frameworks. See the Smarty Grants page or Peak Body organisation websites, for examples.

The Self-Build –  In some cases however, it might be better for organisations to produce your own outcome measurements. Outcome frameworks that are constructed in-house must also be SMART – as well as valid, meaningful, intelligible, able to be disaggregated over time, and consistent. The webinar provides in-depth insight into what constructing and collecting your own dataset should involve.

What tools are helpful when building my own measurement frameworks?

• One way to measure outcomes is by self-reported surveys.
  • This involves putting things into statements and getting people to self-report these prompts – Can include 6-part Likert scales for people to rate their response to given statements.
  • When constructing these surveys, you should include a qualifying statement which gets respondents to address the answers in relation to whether they were caused by the program, event, initiative etc. This links what is reported back to to your intervention.
• Multiple and varying data points are encouraged; such as administrative data, observations or focus groups. This is called a mixed methods approach.
• A data dictionary is a great way to show how your program is measuring outcomes. This involves making an excel spreadsheet with all the relevant data framework details.
  • At a minimum this spreadsheet should include the outcome, measures, survey question, data analysis, and an example results statement.
  • Data collection recorded in the sheet may then be easily located, replicated, or referenced within organisations,
  • A data dictionary essentially becomes your measurement bible, showing how you have measured specific outcomes in your program!

Challenges?

Sampling is important to ensure that survey respondents are representative of the whole population. For up to 100 people, you need 97% to fill out the survey for the results to be representative.

Check out if your sample is relevant here.

Here is a link to some ethical data collecting considerations.

Watch this session here.

This is the second in the series of webinars Our Community are hosting. Keep up to date through their website.

Resources suggested for developing your own outcomes framework can be found here:

• The Nest Framework 
• Amplify Online 
• The Community Services Outcomes Tree 
• Centre for Social Impact
• Learn more from the OPEN Embedding Evidence series 
• Take a look at OPEN’s Outcome Journey