The Setup
This school year was an interesting one. While one half of my colleagues scrambled to “A.I.-proof” their lessons, and the other half felt no concern, I felt drawn to face the challenge head on. As with most issues involving school, I felt the best way to address the issue was to involve the students and face the abyss—as I tell them to do with any topic. So, as I prepared for the school year, I made the required changes: adjusting questions that students would take home to resist rote answers (the same process as de-Googling) and ensuring that the work we did focused on the process instead of a product. Otherwise, “A.I” was fair game; I would be looking, but students could use it.
In the beginning…
I start off the year getting to know my students, so we write a lot of narratives. This meant that machine generated writing could not compete with the time saved by not using it. Why tell a computer what you did during summer break and ask it to write a narrative, when you could just write what you did during summer break? See, no sense in it. During this machine generated writing became a bit of a mockery. Outside of class I would throw the assignment into a generator and see what it spit out. In brief bellringer discussions, students remarked at how “functional” the computer narratives were. One memorable discussion was around a passage generated about spending a summer afternoon sunbathing.
The students noted that every aspect of sunbathing was accounted for: heat, sweat, sunscreen smell, etc. But each was given only the most basic function in the story. Nothing had a purpose.
The blazing sun showed no mercy as Sara lay motionless on her towel, determined to attain the perfect golden-brown glow. Beads of sweat formed on her brow as she periodically flipped over, ensuring even exposure. The faint smell of coconut sunscreen wafted through the heavy air. Despite the scorching heat, Sara embraced every sizzling ray, jealously eyeing the darker complexions around her. She couldn't wait to show off her bronzed skin, the tangible proof of her summer adventures once back at school in the fall. Wincing through squinted eyes, Sara figured a few more hours of meticulous baking would produce just the right sun-kissed hue.
For what it’s worth, this was generated with Antrhopic’s Claude. But more importantly, the students had touched upon something important. I introduced them to flash and sudden fiction—stories that fit about the same size as had been generated. I pulled from Ernest Hemingway’s “A Very Short Story” and the difference was not hard to spot. Luz, the main character of Hemingway’s fiction, appeared constantly as the topic of the sentences. And more often than not, the main idea stays consistent. Luz is on the bed, Luz is relaxed, Luz enjoys the night, Luz is happy, Luz is still on the bed—constancy. Sentence after sentence has a purpose.
The generated story, however, is fragmented. It is possible to force a purpose onto the text, but the reader is the one clearly doing that. Hemingway’s story has a purpose, and the reader can discover this. (This is separate from authorial intent.)
At this stage of the experiment, even my students were unconvinced by the silvery-smooth talk around “A.I.” and the like. Promising, but we were still early in the year.
Midway Upon the Journey…
The writing in my class cannot stay narrative and personal forever. Eventually it does need to progress to analysis, interpretation, and research. I had no concern, for the same purpose that underlies narrative is present in other types of writing.
But I was incorrect.
The author of exposition and critique is far more intimate than one might first realize. Narrative, under this perspective is a considerable layer through which the story unfolds. (We dabble a bit with intent here.) The author attempts to convey something, and an artifice does so. This allows the author to invent (relating to the origin of the word fiction) rules and images that best describe what the author wants. And it was this that the generated content struggled with. Critique requires the removal of this artificial layer. While the critic still conveys their point logically, and by figuratively expressing images the reader will understand, the writing is ultimately more stable.
Unfortunately, that meant that the previous discussions around purpose were no longer as effective as they had once been. Connecting that idea of purpose with concepts like summary and paraphrase made sense to the students, but generating: “a summary of this passage that highlights the idea of X” is a very easy thing for machine generators to do. (And, to my chagrin, reads fine enough in an essay.) So, I set about reflecting on how I could change my approach to help the students attach to their own writing. To see the human part of the process as more valuable than the generated content.
A Tangent on Value
Ultimately my reflections fell upon the terrifying concept that even if I were successful in convincing my students to value the human-ness of their writing, they would be subjected to an endless push for efficiency1 as the only thing of value. This unfortunately is the “point” of writing for most people. A writing task is what it is because of the tradition rather than any inherent value a different task might have. (Think about the fact that most job opportunities require a resume, but also for that information to be entered into an electronic system—one writing task is considered valuable for tradition, the other because of efficiency, neither particularly valuable to the process at hand.) Regardless of the role that writing would play in their lives after school, it would always desire to be efficient first. As with other concepts like this, I find it best for the students to approach it head-on in the classroom. So, I decided to incorporate ideas about efficiency into the discussions we had. And then an unexpected breakthrough happened.
“Error. Error. Error. Examine.”
If students sat down for a conference with myself or a peer review there wasn’t really a way to identify if a passage had been generated (despite the claims of various “detectors” and the like). That is not to say it was impossible, many clues gave away generated content: a change in vocabulary, stunted tone, abrupt topic shift. Context says that this is generated content, but this is not exclusive. Abrupt shifts in vocabulary occurred before generative machines, and usually had its roots in plagiarism (if disingenuous) or just an interest to “write like” another author. The guidance for such an experience is to offer the student advice as to how to create a consistent voice that considers their audience and their own vocabulary needs. Basically, it doesn’t matter how it came about, it’s just poor writing.
Then, of course, there’s the “define it yourself” path, which I have taken many times to explain to students that the work they do does not occur in a vacuum. If it did, their many arguments that laziness is the best policy would be valid. By making student work part of the larger discourse around a subject, it is much more difficult for them to rely on the path of least mechanical resistance, because everything gets leveled out in terms of effort exerted. And this became a keystone to my whole system surrounding generated writing.
I had a student sit down to conference with me. This student was not one that was experimenting with generated writing; they just weren’t that type of student. Now, conferencing constitutes the bulk of my writing-centered units. The goal is to give students the freedom and space to create something, but have multiple checks to see how the work is going. In the case here, I noted many passages that I enjoyed in the piece of writing, and a few here and there where the wording could be better structured. I sent the student off to do their work. Upon the next conference, I could no longer find some of the passages that I had enjoyed. And while the weaker spots had been changed, I would not consider them “fixed” if we are to give that type of description for this discussion around generated writing.
I asked the student directly what had happened to the passages I had liked, asking him to pull back out the first draft to make comparisons. I don’t remember his answer as to why he changed them, but I told him to consider putting some of them back and to put more focus on the specific changes we had identified.
But, upon the next conference, it was as if I was seeing a third first draft. Sure some of the original positive spots had returned, but now bits I had liked from draft two had gone missing. Really, it looked like the student was just “stirring” the essay instead of actually revising—as if all the pieces were always the same, but moved about.
I realized it was generated.
Now that we were just being honest with each other, I changed my tactics for helping this student write. Throughout the year, I had even developed techniques for helping students use generated writing thoughtfully and creatively. So, as the year came to a close, I also felt that I had arrived at a conclusion for this experiment.
“It functions irrationally.”
So, the conclusion to the experiment was simply to say that the experiment approached a problem that didn’t exist at all. As with much of the “A.I.” hype, the problems purported to be solved, are only created by “A.I.” to be solved. The point of writing, any writing really, is that it is created. When I’m selecting good writing my students create for us to celebrate, I’m not just looking at writing that demonstrates that they can follow grammar and mechanics. I look for writing that does that AND contains within it an essence of themselves.
These past two headings have been from a Star Trek episode called “The Changeling” where an artificial intelligence called Nomad takes over the show’s starship, the Enterprise. Nomad defines itself as perfect, and anything that isn’t should be sterilized. The machine observes that one of the crew “functions irrationally” as the “logic” the crewman used was not strict and more gestural. But here’s the rub, from a human perspective the crewman’s “logic” is perfectly fine, it makes sense to the human. And that is what was missing from this experiment.
So, generated writing has become banned and fully usable in my classroom. The apparent paradox causing no issue at all. During the year, I also came to consider this current wave of hype to be mostly hot gas. I think that in time, it will all settle and become just another piece of unique technology. But the lesson to take away, at least for me, is to make sure that my students are seen as human writers, not as machines that need programming in writing.
The Magic School
During this year, I also had a chance to try out the newest startup pushing its way into public education—an “artificial intelligence” do-it-all called Magic School. I was thoroughly unimpressed. The many advertised use-cases had either already been automated decades ago, or had Emperor’s New Clothes’d their non-issueness. Only one, the text-leveler, really angered me, as it was billed off the same strange anti-semantic fluff that other “reading-level alignments” always used. That somehow changing all the lexical words in a text didn’t Ship of Theseus itself into meaninglessness. No, the program2 itself didn’t inspire anything in me. However, it’s diction did. According to all the little bits of flavor-text, everything it—and by extension, teachers did was magical!
Teachers and Magicians share a similar misunderstanding in their popular conceptions. Specifically, I see a connection between teachers and the often conflated magician / alchemist profession.3 The Alchemist is viewed in the public eye as a knower of the unknowable, and if only they had the right tools they could turn lead into gold. A teacher is much the same in the public eye. Our rhetoric around students places them as the base metals and it is through the teacher’s craft that the transmutation into adulthood occurs.
Teachers, then, are often enticed with tools that offer to unlock the hidden potential in every student—a phrase that is not uncommon when discussing alchemy, or other magical manuscripts.
But, I am of the opinion that this commonplace of classroom as magician’s lair largely metastasizes as a symptom seen today: that teachers, classrooms, and more largely public education is a scary place because of the dark arcana occurring therein that seduces innocent children from their parents. If the way we talk about school bears magical connotations, it should not be surprising that some consider it gospel, even if reality is otherwise.
Having learned from my experiment that “A.I.” answers nothing, then this “magic” school offers very little, but still plays into problematic conceptualizations about how teaching works. Instead reality seems to indicate that the best way to help students is not to devise the newest most magical tech that turns every move they make into a chunk of mineable “data” but to consider their humanity.4
In Defense of Inefficiency, a lovely video which describes the phenomenon and strength of efficiency.
I hesitatingly use the term program, as with all things Magic School is web-based, and, therefore, could take all your work with it in the nanosecond it goes down.
For a more historical understanding of Alchemy, I recommend Dr. Justin Sledge and this demonstrative video.