
For Australia’s schoolteachers, the marking pile never seems to get smaller, and they are running out of hours to get through it.
In Australia’s lower secondary schools, staff already work an average of 46.4 hours a week, the third-longest working week in the developed world, according to OECD data. Nearly six of those hours go to marking, and half say there’s simply too much of it to get through on a daily basis.
In recent years, chatbots like ChatGPT, Claude and Gemini have helped teachers lighten the administrative workload. In some cases, this has involved having AI chatbots do the marking for them. Last year, the Teaching and Learning Survey (TALIS) revealed that nearly two-thirds of teachers used AI at work, putting Australia as the fourth highest country within the OECD, and far above the OECD average of 36%.
But while the technology may be saving time, some of the most popular chatbots weren’t built for assessment, and more than four in five teachers worry AI can make incorrect recommendations.
That leaves Principals in a pickle.
On one hand, if they restrict teachers from using these tools, teachers stay buried in marking. On the other hand, allowing teachers to use AI could leave leaders answerable for marks that may not be consistent, and for student work uploaded to platforms the school doesn’t control.
In a recent webinar hosted by Seven Steps to Writing Success, assessment researcher Ben Lawless, founder of Lawless Learning and a lecturer at the University of Melbourne, made an evidence-based case for purpose-built, expert-written assessment tools and explained how to identify the best ones.
Drawing on his work, Lawless presented five questions school leaders can ask when weighing up different AI writing assessment tools.
- Who wrote the rubric, and has it been validated?
- Was the AI trained on expert marking, or is it a general tool like ChatGPT?
- How secure is student data?
- Does the tool help teachers act on the results, or does it stop at a mark?
- Does the tool do the thinking for the student?
A blind spot in teacher training
For all the attention paid to how student work is marked, far less scrutiny falls on the tools used to judge it in the first place. That blind spot, it seems, begins with who writes them.
Lawless said most assessment rubrics are written by classroom teachers who don't have any specialist training in rubric writing or learning progressions or assessment literacy in general.
“As someone who both works in teacher training and delivers training to in-service teachers in assessment, I'd say the existing training is very poor or non-existent,” Lawless told The Educator.
“The Seven Steps rubrics were written by an expert with 20 years' experience in assessment who worked with the person who invented developmental rubrics, Professor Patrick Griffin, at the Assessment Research Centre.”
Lawless said that if the data shows one of the levels isn't working, then the rubric will be rewritten.
“There's no such thing as a perfect rubric. There's no perfect way to describe improvement in a skill, especially one as complicated as writing,” he said. “The best we can do is improve the learning progression, which describes how writing improves, as we learn more from analysing data and performing psychometric analysis on it.”
The gap between policy and practice
With the Federal Government’s “digital detox” underway, many Principals now face a stark choice between banning large language models such as ChatGPT outright or giving staff free rein to use them. For Lawless, the answer lies somewhere in between, though not necessarily where school leaders might expect.
“The sensible middle ground that most schools are using is a bespoke or locked-down version of a commercial retail AI,” he said. “This is even what you see major universities do.”
Lawless pointed to the University of Melbourne, which has built its own AI tool so that any information entered into it stays within the university’s secure ecosystem. That lets staff and students use the technology without sending their data to an outside commercial platform.
“I don't actually think the AI policy is the biggest lever here, though,” he said. “If you've got teachers that gather assessment evidence and don't do anything with it or create assessment measurement tools that are not high-quality in the first place, then using a teacher to mark or an AI to mark isn't going to make that much difference.”
As for what a good school policy on AI might look like, Lawless said schools needn’t reinvent the wheel, with plenty of sensible frameworks already out there.
“I would look at people like Leon Furze and Matt Esterman and see what they have to say. But again, schools are very complicated systems and, as the saying goes, ‘there’s many a slip ’twixt cup and lip’,” he said. “A school can have whatever policies it wants, but I think the extent to which it enforces those policies is far more important.”
Lawless said that, without wanting to criticise anyone, many leaders assume their policies paint an accurate picture of their school.
“Far more important than the exact nature of the policy is whether schools are bothering to enforce their policies or whether they have a culture in which policies are followed,” he said. “The implementation gap for most school policies is enormous.”
The trouble with letter grades
It’s often assumed that schools invest heavily in moderation so a B in one classroom carries the same weight as a B in the next. But when asked how AI-assisted marking might disrupt that consistency, Lawless questioned the premise.
“I'm interested in how you've arrived at the conclusion that schools put a lot of time into moderation. I don't believe that is the case,” he said. “Certainly, there's a lot of moderation in, say, English, for example, and probably, just from my hunch, more moderation in primary than in secondary. I guess it depends on what ‘a lot of time’ means.”
Lawless said the idea that a B in one class means the same as a B in the next is “simply untrue” and is “a huge problem” for the way that schools report grades to parents.
“Even if it were the case that a B in English in Year 7 is the same in one class as in the other, which is probably the most likely of all of these scenarios, the idea that a B in science is the same as a B in English, or that a B in Year 7 is the same as a B in Year 8, is completely unfounded and almost certainly not the case,” he said.
“Almost no school that I have seen has ever done a big data-gathering exercise where they analyse the number of marks, average marks, spread, etc., in different subjects in different year levels.”
Lawless said reporting letter grades to parents is “one of those myths, like the idea that private schools have better teachers.”
“It's just something that's believed by society but just isn't true at all,” he said. “I'm sure there are schools out there that are doing way better than how I've just characterised it, but I think that would be the exception.”
The human check on the machine
Seven Steps CEO Kathleen Killick said the company’s AI-powered assessment tool is designed to support teachers’ professional judgement, not replace it.
“Teachers can review and change any mark they disagree with. Where their judgement differs from the tool’s – that difference can spark a moderation discussion with colleagues,” Killick told The Educator. “These individual adjustments don’t feed back into the tool in real time. Instead, patterns in teacher feedback and assessment data inform its ongoing refinement.”
With more than 38,000 assessments uploaded, educators can see which criteria are performing well and where the model’s accuracy can improve and make evidence-based adjustments, Killick said.
“Ultimately, our aim is a tool teachers trust, so they can teach writing with confidence and lift results.”