The technology flywheel is building momentum everywhere
The huge technology flywheel is accelerating across every market and industry. We are now at the inflection point where there are very few jobs being done in the world that don't involve some kind of technology be it up or downstream from the job being done. As teachers of English and humanities subjects, our job is to coach students in how to translate thinking into writing, of course within the rules and conventions of written language. The mission hasn't changed, the capabilities we now have to get the job done, have.
Walk with me into a generations old family run bakery in Spello, a quaint hilltop town in Umbria Italy. I had a romanticised, technology-free vision of hand made Artesian bread baked daily in the traditional way. It looked good, smelt great, it was the real deal. Mid morning I met the elderly baker in the square, varying the oven temperature on his iPhone, while watching the bread baking via bake-cam, while he had a cappuccino in the spring sun. As Stephen Covey said, "innovation happens at the speed of trust" and this is exactly what I experienced. The first generation of this baking family is operating with trust in much younger technology.
Technology impacts us all in ways that work for us, or, as Clayton Christensen put it, "technology is only useful to help do the jobs to be done". The iPhone automation of the ovens, with cameras, was mind-blowing. What happened there? This is simply another example of 'par for the course'. Technology is aligning with everything and many times it is key to getting the job done, quicker and better. I do note that good bread starts with the dough and the iPhone is useless at formulating and kneading dough. That's a key parable for educators right there.
Teacher hours spent in grading and feedback
This week my research went deep into teacher utilisation hours to understand the time and energy that grading and feedback consumes. I went looking for the detailed breakdown of the 'jobs to be done', recording hours spent on what it actually takes to give feedback and grading across an academic year for teachers with multiple classes.
When students are asked to write, what are the subsequent workflow steps of jobs to be done by teachers? What parameters are involved ? What's the point of asking students to practice writing if there is no coach to fine tune the form of each writer? What sort of hours are we talking about?
I entered into a whole new world of time related variables and insights. Turns out there are multiple parameters involved in calculating the number of hours needed to do the grading and feedback job manually. I collected most of the the parameters and whilst there were more, these were enough!
The base parameters involved
- The number of students in your class. Label this S
- The number of classes you have. Label this variable C
- The number of times per week students write, formative or summative across the school year. Label this variable N
- The number of words students write each time they write - Label this W
- The 24 seconds it takes to read 100 words. Label this R
- Total Read time in hours for all student essays = TR
- The minutes spent giving feedback to each student. Label this F
Teachers cognitively deal with these parameters in a complex equation of time management, often on auto pilot, every day. Like the baker who for most of his life never left the oven unattended, teachers also get caught in business as usual workloads that create a huge number of additional hours, often not thinking how to change gears, and sit in the sun! The baker's life and health changed with technology, and in his view, so did the quality of his bread.
The hours add up across multiple inputs
The formula to calculate what takes time is quite extensive. The base calculation all depends on the number of words being written across the number of students and classes and the number of writing events that happens. Keep in mind that research says students need to write more, more often. If these rather conservative numbers align with you, this is your reality. The data inputs follow:
No Students(S) | 30 | 30 | 35 | 35 |
Words per Essay (W) | 300 | 400 | 500 | 600 |
Number of Classes (C) | 4 | 4 | 4 | 4 |
Number of weeks of writing per year (N) - 1 essay per week | 20 | 20 | 20 | 20 |
Teacher feedback per Essay- (F) (Mins) | 10 | 12 | 15 | 15 |
Hours Reading Texts / month (TR) across ((S*W)*C)*N | 80 | 100 | 156 | 187 |
Hours invested per Week for N weeks in reading, feedback and grading | 15 | 17 | 27 | 28 |
Gradebook, Remediation, Report hours per week | 6 | 6 | 7 | 7 |
Total Teacher Hours per week for feedback and grading for all Classes | 21 | 23 | 34 | 35 |
The more words written, the more hours needed, no surprise there. Even if you amortise the hours per week across week days, there are 4 -6 hours a day - extra needed and that is at a very low volume of writing practice.
So, it's true, teachers with a text based subject in English or humanities invest in a significant number of feedback and grading hours. Many of these hours overflow into weekend and personal time. A Single class takes some five additional hours per week.
What about using a third party LLM to help with feedback and grading?
Now hold that thought! This needs a whole new set of parameters and workflows that need to be worked through. While the LLM promise is to do the 'reading and checking', workflow hours grow around being organised for using the LLM and then manually dealing with LLM inputs and outputs.
The simple view of the process is:
- Copy and paste text into a Chat interface,
- Using one of the Chat LLMs online, construct and run a prompt or series of prompts to create feedback, suggestion, corrections etc
- Review and check the feedback,
- Cut and paste the feedback into to somewhere for students to read and then,
- Do and check the grading, against your rubric
The detail around how long all this actually takes and how technically complicated the whole process can get is usually overlooked.
The first thing to mention is that using an LLM anchors you to a desk and laptop for hours. You have to move information between multiple interfaces, copy and paste and keep your wits about you. One slip on the copy and paste and everything is out of sync.
The parameters involved in teachers using an LLM for student essay feedback and grading are also numerous.
Step in the Process | Minutes Taken per Essay | Description of the process step |
Cut and paste student essay | 2 | Taking text from Google or LMSs or Word, into an LLM is a cut and paste hackathon. From one system to another is cut and paste. |
Prompt, Execute | .5 per prompt | One prompt usually cannot cover all feedback elements. .5 minutes per prompt also amplifies the management of the text and advice returned. |
Check / Change Feedback | 2 | If you dont like the feedback or want to change it, you have to change your prompt or put the feedback into somewhere where it can be edited. |
Create Insights | 1.2 | Manually looking and recording extra insights per student is intensive and additional to the LLM feedback. |
Copy Paste back for students | 2.5 | Once you have feedback, you have to out it somewhere that is accessible for students. |
Grade , check, modify | 2 | If you are grading, you will probably want to do that yourself. So it's rubric time for you. Then you have to record that somewhere. |
Total per Essay | 9.7 minutes | Total Hours for 4 classes = 4 * (((S*C)*N) * 9.7) / 60) in minutes = 19 hours |
The net result is that an LLM saves 2 hours - assuming you know what you are doing
The end result is that using an LLM with the same scenario of student writing saves teachers 2 hours a week. To remind you, the scenario is 30 students in 4 classes writing 20 texts per school year at 300 words per essay.
Who said LLMs saved time and stress? All hours spent are very much 'cut and paste' hours chained to a laptop . That is an intensive process and does not at all optimise the cognitive power of teachers. Copying and pasting text from one medium to another is not really progress.
If your students look for feedback on drafts and you give feedback on drafts, the numbers here amplify by the number of drafts you give feedback on.
Of course, the more words you have to check, more hours are saved in the LLM reading and feedback process steps however, the mechanical process of using LLMs in a full loop of feedback, guidance and grading are not reduced or made easier.
Whilst the mission to improve student writing has not changed, capabilities have.
Research from many sources agree that to lift student writing skills there are nine mechanisms that work in a very inter-related way. Three of the most impactful mechanisms deliver the foundational steps. Laying these foundations needs a rethink.
Manually applied hours at the level needed are not sustainable for teachers. Not only are they excessive but also not what teachers should be doing. Research is very clear that students need to be encouraged to focus on self mastery with teacher oversight and pedagogy designed to steer student success. These are a different set of hours that carry more importance to learning outcomes.
The three mechanisms that make the biggest difference to improving student writing outcomes are:
- Increase student motivation to write. These strategies mostly involve changes that make writing more meaningful, make students more comfortable writing in the classroom, and encourage them to focus on mastery goals with supportive feedback. Research suggests the strongest effect sizes come from adult-teacher feedback. Graham, Harris, and Hebert (2011) found strong overall effects for adult feedback (ES= 1.01)
- Self-regulated strategy development (SRSD) consistently shows a strong impact on student learning. SRSD targets not only self- regulation, but also genre knowledge (through direct instruction), strategy development (through direct instruction and scaffolded learning activities), and motivation (through a gradual release model designed to increase students’ sense of self-efficacy). Graham and Perin (2007), reported a SRSD effect size of 0.82.
- The impact of word processing on engagement. Murphy and Graham (2012) found a very large impact of word processing on student motivation to write (ES= 1.42). At similar proficiency levels, digital writing is significantly faster and requires less effort than writing by hand. It enables faster revision processes, and simplifies sharing and collaboration.
In the next and final post of this series, we look at how integrated platforms like Scribo from Literatu help teachers "watch the oven" and manage the output without having to stand near heat every day. As has been said, the speed of innovation travels at the speed of trust. I'm sure the experienced baker took his time moving across to his integrated baking and oven app. I'm also sure he has never looked back.
Innovative AI platforms can quickly build trust, greatly reduce teacher workloads, improve the feedback cadence of students and engage them in self directed strategies. All in all, the feedback and grading job can be done better and quicker giving more time for teachers to mentor students.
Imagine a world where 21 hours was replaced by 3 hours. 18 more teaching hours / resting hours / thinking hours / mentoring hours / enjoying a coffee in the sun. Take your pick!
