Grade the thinking, not the essay

Everyone is cheating their way through college, writes New York Magazine and the cheating is of course done using GenAI. On Substack, Gary Marcus concludes:

The likely outcome here? College students from here on out will get little from their educations, professors will be overwhelmed, unable to adapt to the scale of what would be required to do individualized education with budgets that don’t really support that, and the Sam Altmans of the world will laugh their way to the bank.

Employers will struggle to find well-educated employees. Democracy, which thrives on having an educated citizenry, will crumble.

While I understand that anyone in the United States is entitled to a sense of impending civilisational collapse, to me these concerns reek of moral panic.

Lord of the comfort zone

Let's start with the opening of the article in New York Magazine. It describes Chungin Lee, who seems to fit a particular archetype that everyone who works in higher education sees pop up time and again. Self-confident to the point of closed-mindedness, this student type goes beyond mere academic entitlement, which is about externalising responsibility for learning, straight into the assumption that they know better than anyone else what is valuable and worthwhile to do or learn. In practice, of course, it's always the uncomfortable activities with limited immediate payoff that are worthless in the eyes of this archetype. These are students who do not just remain in their own comfort zone, but rule that zone with an iron fist, making sure as little as possible goes in and out.

Some excerpts from the article illustrate the type. First of all, Lee thinks it's perfectly okay for him to be the arbiter of whether college assignments are even relevant:

When he started at Columbia as a sophomore this past September, he didn’t worry much about academics or his GPA. “Most assignments in college are not relevant,” he told me. “They’re hackable by AI, and I just had no interest in doing them.”

He is also fully aware of what jobs for coders look like and therefore he has a much better understanding of what their job application process should be than the people who are actually doing the hiring:

As a coder, he had spent some 600 miserable hours on LeetCode, a training platform that prepares coders to answer the algorithmic riddles tech companies ask job and internship candidates during interviews. Lee, like many young developers, found the riddles tedious and mostly irrelevant to the work coders might actually do on the job. What was the point? What if they built a program that hid AI from browsers during remote job interviews so that interviewees could cheat their way through instead?

And when he was reprimanded by Columbia University for actually building and promoting such a tool, his response was one of disbelief.

Lee thought it absurd that Columbia, which had a partnership with ChatGPT’s parent company, OpenAI, would punish him for innovating with AI.

Do you see the pattern? Here is someone who basically sees any pushback by the world as an inappropriate obstacle to his self-perceived genius, and who justifies cheating by telling himself that he is just ahead of the curve. He says he "doesn’t know a single student at the school who isn’t using AI to cheat," but what weight should these words carry? They come from someone who has built a belief system around this claim and has a vested interest in fatalism about student use: he recently raised capital to develop Cluely, a tool that is marketed as cheating help.

So is he wrong? Isn't everyone cheating? Surveys and usage statistics of ChatGPT suggest that a large majority of students are using the tool for education, but let's not forget there are non-fraudulent use cases for the technology. While I don't want to dismiss the concerns of the teachers who were interviewed for the New York Magazine article, I personally still see most students wanting to learn and some actually express reservations about LLM use because they think (quite reasonably, I should add) that it might interfere with their learning process. While I am all for making course structure and assessment more resilient against fraud and plagiarism, I do not want a majority of eager students to lose out on learning opportunities, just to stop a few others from coasting through university while they feed a TikTok addiction, as one student in the article motivates her LLM use.

Validating learning activities

One thing that became more important through the introduction of LLMs is the validation of learning activities. Students need to know what they are learning and why they are learning it. Validation will invite the Lee archetype to argue against anything he dislikes, which is why teachers are reluctant to do it, but it's important.

By itself, however, it is not enough. Take the case of Wendy, offered by New York Magazine:

Wendy, a freshman finance major at one of the city’s top universities, told me that she is against using AI. Or, she clarified, “I’m against copy-and-pasting. I’m against cheating and plagiarism. All of that. It’s against the student handbook.” Then she described, step-by-step, how on a recent Friday at 8 a.m., she called up an AI platform to help her write a four-to-five-page essay due two hours later.

Wendy actually indicates she looks back fondly on non-LLM-assisted writing, but still has her whole outline set up by GenAI, effectively turning the writing process into a filling exercise. Could it be that she does not realise the actually valuable part of writing a paper is drafting the outline and refining your argumentation? Upon reflection, she does know:

"I think there is beauty in trying to plan your essay. You learn a lot. You have to think, Oh, what can I write in this paragraph? Or what should my thesis be? ” But she’d rather get good grades. “An essay with ChatGPT, it’s like it just gives you straight up what you have to follow. You just don’t really have to think that much.”

As this shows, validation alone is insufficient. The incentives for students should also change. It is remarkable that someone can find beauty in the process of doing their academic work but feel that engaging with it would lower their grade. That should not be possible – this wasted learning opportunity should have consequences somewhere in her degree programme.

The formative turn

The trick, I believe, is to reconsider these tried-and-tested assessment forms of term papers and research proposals and look at them as formative moments. They are opportunities for students to hone their thinking skills and for teachers to engage with their students' reasoning. The summative assessment should then be something else, such as an oral examination, a CLA+-type assignment or anything else that asks students to think on their feet and articulate their reasoning. If self-written essays are serving the purpose that we think they are, students like Wendy should get better grades for these assessment forms if they put in the legwork during formative assessment.

Now such solutions may sound like they would stretch the budgets that Gary Marcus was already lamenting, but oral assessment can be done in the time it takes to grade an essay and performance assessments are perfectly scalable. These changes are possible.

Of course, Lee would probably still shirk his learning in such a system and he would possibly look for ways to bring smart glasses or other cheating devices into the summative assessment. Yet there would at least be recognition that putting in the actual work pays off, so that the incentives for students are aligned with the eagerness for learning and growth that I still believe are present in most students.