Evaluation of National Strategy activities
As outlined in the Introduction, a key output from this Framework includes evaluations of National Strategy action plans and their associated activities. This evaluation work will assess the implementation of the National Strategy and specific activities to determine whether they are having the intended impact.
A co-designed list of monitoring and evaluation principles and key evaluation questions were developed to support these evaluations. The National Office will coordinate the alignment of these evaluations with the Framework.
Principles for monitoring and evaluation
The following set of principles to guide all monitoring and evaluation activities for the National Strategy was co-designed by the Co-design Working Group. These principles serve as an ethical framework and should be used by both activity owners and independent evaluators.
Principle 1: ensure monitoring and evaluation approaches are victim and survivor-centred and focus on the needs of children and young people
- Ensure that the voices and perspectives of victims and survivors are reflected throughout.
- Ensure that all processes and practices are trauma-informed (including ensuring victims and survivors are not re-traumatised by participation) and culturally safe.
- Protect the privacy, confidentiality, and physical and emotional safety of all involved. This includes victims and survivors, secondary victims and those affected by vicarious trauma.
- Acknowledge and reflect on the diverse lived experiences of victims and survivors, the intersectional impacts of diverse backgrounds and practices, and the value of stories and data that captures the experiences of people experiencing intersectional disadvantage (especially the National Strategy priority groups).
Principle 2: involve local partners and stakeholders
- Co-design monitoring and evaluation activities in partnership with those who are delivering or impacted by the programs, including families, kin and carers, children, young people, victims and survivors. This is especially important when designing monitoring and evaluation strategies with First Nations people and people from other diverse cultural backgrounds.
- Use participatory and culturally safe approaches at all stages of evaluation, from question identification and design, data sourcing and collection, results interpretation and reporting.
- Recognise children and young people as stakeholders and beneficiaries, ensuring their voices are heard and they are believed.
Principle 3: ensure a focus on learning, accountability, and transparency
- Ensure a focus on learning, accountability, and transparency, so that data collected and reported are useful for program design, adaptation, and decision making.
- Ensure there are feedback mechanisms so that participants can learn from, act on and advocate using the results.
- Encourage transparency by ensuring open access of monitoring data, seeking feedback on monitoring and evaluation frameworks and publishing evaluation findings in a timely manner.
- Commit to using plain English, so that documents are accessible to all stakeholders.
- Evaluations should provide shared data for all priority groups and be contextualised beyond statistical trends to build a full change narrative and produce products that are meaningful to people.
- Consider in an ongoing manner whether the National Strategy and measures are meeting the Royal Commission’s recommendations and intended outcomes.
Principle 4: plan for evaluation from the beginning, allocate adequate budget and reasonable timeframes
- High-quality evaluation is dependent on considering the theory of change, and relevant outcomes, indicators, and measures in advance.
- Conduct major evaluations independently of, though with input from, policy makers, implementation agencies and program managers.
- Measure any evaluations with consideration for all National Strategy priority groups.
- Consider the progress of the Framework against the recommendations of the Royal Commission.
Key evaluation questions
Key evaluation questions are the high-level questions an evaluation is designed to answer, rather than the specific questions that may be asked of participants in an interview or through data analysis.
The first set of key evaluation questions focus on the process of design and implementation for the activities under the National Strategy. These questions will both identify whether activities have been implemented as planned, and whether modifications are required to activities prior to implementation of subsequent Action Plans.
Key evaluation questions
- To what extent was the design of the activities under the Action Plan appropriate to support its intended outcome?
- To what extent have the activities under the Action Plan been implemented, and to what extent have activities and outputs reflected a timely, cost-effective and quality use of inputs?
- To what extent did the implementation of activities reflect the National Strategy values? To what extent do families, kin and carers, victims and survivors and members of priority groups feel engaged in the Action Plan implementation, evaluation and reporting?
- To what extent have key outputs achieved short-term outcomes, and how has this contributed to medium- and long-term outcomes? Have there been any unintended consequences?
- To what extent did activity owners use evaluation artefacts (for example: theory of change, program logic, data matrix) to design and plan for evaluation?
- Are there opportunities for improvement in relation to program fidelity?
The second set of key evaluation questions focuses on the outcomes achieved across the 5 National Strategy themes and overall objective.
Outcomes and impact key evaluation questions
- Is Australia safer for children and young people than in previous years? Have changes been distributed equitably across the National Strategy’s priority groups?
- Are Australians better able to understand and recognise child sexual abuse?
- Are Australians more likely than in previous years to speak out against child sexual abuse, seek or offer help?
- Are victims and survivors of abuse feeling more empowered and getting the support they need from families, communities, governments, support services and organisations?
- Are systems and services better integrated and equipped to support victims and survivors?
- Is there a consistent and effective national approach to supporting children and young people who have displayed harmful sexual behaviours?
- Are offenders, perpetrators, and those at risk of committing child sexual abuse being better identified, held accountable, and engaged in effective behaviour change?
- Are systems and services better integrated and equipped to respond to offenders, perpetrators and those at risk of committing child sexual abuse?
- Are there high-quality and accessible data available on child sexual abuse?
- How effective has the National Strategy been in reducing the risk, extent and impact of child sexual abuse and related harms in Australia?
- Do priority group members feel change has occurred in areas relevant to them? Is that change positive or negative?