How LLMs Impact Student Learning?

As artificial intelligence has become a mainstream and ubiquitous tool for hundreds of millions or even billions of people around the world — specifically, through chatbots and large language models (LLMs) like Claude, Gemini, ChatGPT, Llama, and many others — it's both enabled new pathways for problem solving and also led to new pitfalls for students, early career professionals, and non-experts seeking to mimic the illusion of expertise. While many who create, sell, or promote LLMs laud their use cases, a significant worry has arisen among students, teachers, professors, and education researchers: that students are not using these artificial tools to enhance their learning, but rather to replace it, outsourcing the hard and rewarding work of critical thought to these LLMs simply by prompt-hacking them.

To be certain, this is a real problem, and a real new way that students (as well as many others) are using the tools at their disposal to complete their assigned work, with the consequence of short-cutting their own intellectual development. But how big of a problem is this really, and who does it impact the most? That’s what physics graduate student Cameron Bishop wants to know, asking:

“As a student in physics now entering my 3rd year of [graduate studies], have you written about the impact of these LLMs/chatbots on students?”

As always, there's a lot to unpack here, but it's not all "doom and gloom" as some might fear. Let's go through the impacts of these new technologies, but with a view to how students historically have cheated themselves as well, to put things in their proper context.

When it comes to learning, whether in a scholastic environment or otherwise, the goal — as so many misunderstand — is not for the student to get the right answer, or to come up with an acceptable answer, to whatever question they're prompted with. The goal is for the student to gain an understanding of the material: understanding that is a further step along their journey toward mastery of a subject. Whether that's reading comprehension, a foreign language, wrapping your head around a historical event, mathematical fluency, or the ability to solve scientific problems isn't particularly relevant. The goal is to become "good" at whatever you're working toward learning.