The Australian Government’s Education Analytics Service has discovered valuable lessons on research methodologies for future education research through a multi-year study series on teacher development investments in Timor-Leste, Vanuatu, and Lao PDR.
As part of the Education Analytics Service, the Australian Council of Educational Research (ACER) was commissioned to investigate how the Australian investment has contributed to improved teaching quality and student learning.
The study series, available to read on ACER’s website, provided insights into the challenges of data availability for measuring and reporting on student learning outcomes, as well as the use of classroom observations and case studies to investigate and understand teaching quality and student learning.
An examination of student learning outcomes should point to evidence regarding improved teaching quality, but a primary challenge for the study series was the availability of valid and reliable data.
“A systematic approach to monitoring learning achievement over time is important for any education and learning system because it enables governments to make the changes required to lift student achievement levels and equity,” the ACER team wrote in article about its investigation. “Sustained support is needed to implement national cycles of collection of student outcomes data.”
Faced with the absence of consistent data approaches across the three countries, ACER adopted a mixed-methods data collection approach and designed innovative solutions on a country-level basis.
The team used existing national and regional data in Vanuatu to provide a reference point for student learning in literacy and numeracy. In Laos, where no Grade 1 national data was available, a project-specific student assessment was designed. In Timor-Leste, a range of student data sets collected by different stakeholders was examined.
In-depth case studies involving teacher and education stakeholder interviews were also conducted by the team. Additionally, classroom observations provided important evidence of changes in teaching practices to supplement stakeholder reports, while offering an example of the adaptive approach of the study series.
ACER said it improved each cycle of data collection based on previous lesson learned across the three countries, with local research teams tapped for support.
With the help of in-country research partners, each study benefited from a more comprehensive understanding of local contexts, which strengthened the quality of sampling processes, instrument development, and accuracy of data analysis and reporting.
In all three countries, local research teams were able to continue the data collection process to a high standard with remote support from ACER during the pandemic.
“Where possible, this capacity-sharing approach will be used as a model moving forward,” ACER said further.