Identify, Implement, Evaluate
Anglicare Victoria formed in 1997, is one of the state’s largest out-of-home care providers. Following the amalgamation of three of Victoria’s most experienced child and family welfare agencies: the Mission of St. James and St. John, St. John’s Homes for Boys and Girls, and Mission to the Streets and Lanes, Anglicare continues to focus on strengthening families and keeping them together, through disadvantage and hardship. Anglicare is currently bringing world’s best practice to Victoria with Evidence-based programs: SafeCare and Functional Family Therapy – Child Welfare.
David Giles
Anglicare Victoria’s Principal Researcher
Anglicare Victoria has developed an outcomes framework specific to out-of-home care (OOHC). The framework is aligned to the Department of Health and Human Services’ (DHHS) Looking After Children (LAC) developmental domains (health, emotional and behavioural development, education, social presentation, identity, self-care skills, and family and social relationships).
Results from each client survey inform practice decisions about the individual client’s placement planning and other
Interventions. The survey provides high quality data about the wellbeing of children and young people and subsequent
Evidence-informed practiceadvice which is then used for continuous service improvement at individual, cohort-specific and service-wide levels.
Anglicare Victoria had previously developed and trialled an outcomes framework to measure the wellbeing of children and young people in its OOHC programs.
However, the initial framework tried to measure too much, took a long time for staff to complete, and led to problems with incomplete data.
As a result, they redesigned the framework with a focus on making it easy and efficient to use and embedding it within existing practice processes. David explained that they needed to make sure that relevant,
Evidence-informed practiceprinciples and advice are automatically included within results/reports when the survey indicates that individual children and young people are experiencing particular difficulties.
“It’s important to not to let the pursuit of perfect destroy the attainment of the good”
“For any outcomes framework to be successfully implemented within services, it must be both useful and convenient for workers.”
Throughout the design phase, Anglicare Victoria focused on key outcome areas as opposed to trying to measure every outcome that may possibly be of interest. The research team reduced the total number of questions from several hundred to more manageable 30-75 questions (depending on the age of the child or young person).
The revised survey includes a mix of
Validatedpsychometric measures and specific questions relating to placement quality issues. Check out the measures Anglicare Victoria used at the bottom of the page.
Most importantly, the survey now only takes between 5 – 15 minutes (depending on the age of the child) and can be completed over the course of several input sessions – providing staff with the opportunity to gather additional information if required.
A major consideration for the design team was to save staff time, which would increase the likelihood of survey completion and quality data.
By structuring the survey in accordance with the LAC developmental domains, workers can simply copy and paste straight from the results-report into the required fields for DHHS reporting, specifically the qualitative-input fields of the DHHS Review of Care and Placement Plans and 15+ Care and Transition Plans.
Survey scheduling is also built into each child or young person’s individual placement review cycle, enabling survey-related work to be staggered across the year, rather than all coming at once for all children and young people within a worker’s caseload.
When designing the practice framework, Anglicare Victoria identified instruments and questions that could serve as a measure of client psychosocial wellbeing and developmental progress, as well as an assessment of placement and service quality. The survey results-report enables practitioners to reflect on client strengths and challenges, including in relation to the quality of placements.
This highlights useful areas of focus for the next six months, leading up to the next review. Additionally,
Evidence-informed practiceprinciples and advice is automatically included within the results-reports. This provides useful practice guidance around how to intervene regarding areas of difficulty highlighted in the data for an individual child.
Anglicare Victoria recognised the need for the survey interface to be easy for workers to access and use. Their Learning System Grant enabled the agency to use the services of Orima Research, which provide the interface and secure data-capturing systems for the survey. However, organisations with a smaller budget could potentially harness existing data analytics services offered by the government.
Anglicare Victoria’s research team dedicated significant thought to the development of the outcomes tool. The project drew on in-house expertise of staff in key positions, including executive management and principal practitioner roles, research and analytics, and direct service staff. This kind of multi-disciplinary collaboration helps ensure the finished product meets the needs of all stakeholders – including service staff, researchers, management, funders and – most importantly – clients.
“Your assessment/review measures and outcomes measures should be the same tools in a process sense…requiring workers to complete separate assessment/review and outcomes measures is just too much work. They need to be spending as much time as possible engaging with clients, not completing paperwork.”
The revised OOHC survey was officially launched in November 2018 and survey data analyses are occurring progressively over the course of the next 12 months.
Anglicare Victoria now has an outcomes measurement tool that they are actively using with clients in care. As a practice tool, the framework will assist with improving the services provided to individual clients by highlighting areas of strength and challenge at a client-cohort and service level, and so usefully inform continuous improvement processes. Additionally, outcomes data will be used for a variety of advocacy-oriented research and service-evaluation initiatives, including rigorous testing of service models to determine if these can be placed on the DHHS evidence–based service menu.
Over the next twelve months, Anglicare Victoria will develop a complete set of repeated measures data for each child/young person in their care. This information will allow for improvements in practice delivery so that services continue to provide optimal care, keeping children and young people at the centre of all efforts and planning.
Identify, Design, Evaluate
Strength and Difficulties Questionnaire (SDQ)
The SDQ is a well validated and population-normed instrument which assesses in respect of emotional problems, peer problems, conduct problems, hyperactivity, and prosocial behaviour. Learn more about the SDQ...
Quality of School Life – Adventure (Motivation) subscale (QSL)
The QSL measures primary-school-aged students’ perceptions towards school against three dimensions, 1) general satisfaction with school 2) commitment to school work 3) attitudes towards teachers. Learn more about the QSL
School Attitudes Assessment Survey-Revised
This article describes the School Attitudes Assessment Survey - Revised (SAAS-R). This survey is a validated instrument used to measure the attitudes of adolescents toward school and teachers as well as their goal-valuation, motivation and academic self-perceptions. It is also used to explore below average academic achievement in high school students.
