Navigating the impact of LLMs on education
Last week, I had the pleasure of speaking to Harvey Mudd Assistant Professor Josh Brake. Professor Brake has been actively writing about his opinions and experiences with AI and its influence in education in his Substack, and I was excited to chat with him to discuss his thoughts on how educators are adopting and adapting the technology, inside and outside the classroom.
A key takeaway from my conversation with Prof. Brake was that AI tools like ChatGPT, and in particular, large language models (LLMs), are just that: tools. They are not sentient and should not be treated as a replacement for one’s brain. In his own words:
“We need to decide whether we are driving or handing the wheel over, and I think we should drive. LLMs can fabricate information or make wrong predictions because they are based on a statistical model. That being said, I do think we can use them in helpful ways if we are thoughtful about it. Brainstorming is one example, where it’s like a sparring partner that pushes you in areas you might not have thought about otherwise, but you always have the control to make the final decision.”
As LLMs are designed to mimic human responses, it’s easy to fall into this trap and think that there’s an actual brain behind the screen, leading many to trust an LLM to think or write for them. Prof. Brake told me, “It is no mistake that ChatGPT has been implemented as a chatbot that has conversations with you, types out words one by one to delude into thinking there’s a person on the other side”. However, a large language model’s only job, and the only thing it is doing behind the scenes, is trying to predict what the next word will be in a block of text, and then writing out that word. They’ve simply gotten very good at predicting and writing humanlike blocks of text due to copious training. But remembering that the model is not somehow self-aware, or intelligent, and is simply a next-word-predictor, helps to put into perspective how the model should be used, and why you shouldn’t let it do the thinking for you.
This leads into another key issue to address in educational settings; both students and educators need to know the basics of how LLMs work; their capabilities, their limitations, and more. As Prof. Brake put it, “My biggest thing right now for educators and students alike, is that as a first step we need some base level literacy about these tools. We need to know what an LLM is, how it works, what it does and does not do and understand whether it is a hammer or a screwdriver. So, AI literacy is the first thing and the second is being open and doing experimental teaching and learning.”
That last bit about experimental teaching and learning is important. In our discussion, he likened the rise of ChatGPT and other LLMs such as Bard, Bing AI and Claude to the rise of calculators, Wikipedia and search engines in the past. Each of these tools were highly disruptive to education once they became available. The calculator in particular has many parallels with LLMs today; they are both technologies that appeared to be capable of destroying many aspects of learning through their incredible power. As a result, the solution to the integration of ChatGPT is likely similar to the integration of calculators in math classrooms; rather than outright ban the calculator in all aspects of math class, teachers adapted their curriculums to leverage their power, creating new classes of assessments and assignments to adapt to the new technology. Today, most high school math courses flat-out require you to bring a calculator and develop proficiency with one. Therefore, an openness to shake up the methods of teaching and the types of assignments given to students is necessary to find what works and what doesn’t in the context of utilizing LLMs as tools to supercharge, not undermine, learning.
Of course, there is an elephant in the room: plagiarism. The reason that many school districts across the country banned LLMs like ChatGPT was to have an immediate counter to a new tool that showed up on the scene very abruptly. A favorite quote of mine from Prof. Brake to describe the situation that these districts faced is that “The mouse just found a rocket launcher and the cat is caught flat footed with no missile defense tools.” Naturally, I asked Prof. Brake about what missile defenses would be effective, and what he told me was insightful:
“There are people using it to cheat and do academically dishonest things, which is dangerous, but I am not in favor of banning this technology. It is not conducive to set up an arms race between teachers and students, and I think it is much better to try and cultivate a relationship of trust and engage with it together and walk through it.”
In the vein of trust, Prof. Brake and I discussed the honor code at his institution, Harvey Mudd, which he has cited before as a positive and constructive way for educational institutions to approach the academic integrity and cheating problem. While he did mention that an honor code is no silver bullet, and is dependent on the culture of an institution and its respect by the community, “an honor code does make dealing with a new technology like ChatGPT much easier and opens up space for fruitful conversations than needing to have knee-jerk reactions.”
LLMs like ChatGPT caused a massive stir in the world of education, with wildly different opinions on what place they had in the classroom flying around, and new capabilities and developments hitting us thick and fast. Relentless hype and bluster from much of the media hasn’t helped the case either, and has led to some rather unfortunate situations, with students being accused of plagiarism on the basis of tools with questionable accuracy, and outright bans on a technology that, for all its flaws, really has something to offer. My conversation with Prof. Brake highlighted this: with the proper expectations and a level-headed approach to using these tools, they have the potential to be an incredibly positive tool in learning, for both students and educators.