Artificial Intelligence continues to shape education anew, and the latest study from Oregon Health & Science University is firing up fresh conversations about the future of learning and assessment. It pitted six LLMs—including ChatGPT—against real-world students in a biomedical informatics course. The AI models outcompeted most of the students who participated, falling within the top 50-75 percentiles on exams designed for human learners.
The breakthrough study was led by Dr. William Hersh, a professor of medical informatics at OHSU. His goal was simple: to see just how well generative AI models—like ChatGPT—would do on tests designed for one particular academic discipline. What did emerge is that AI did better, with important questions now arising about the future of education and student assessment.
This finding has implications that go far beyond simple academic curiosity. “This does raise concern about cheating, but there is a larger issue here: how do we know that our students are actually learning and mastering the knowledge and skills they need for their future professional work?” he added, expressing his concern not just on the capability of AI but also on the modes of assessment altogether in academia.
AI vs. Real-World Students: What Happened?
It pitted the AI models against 139 students in an online biomedical and health informatics course. Hersh and his co-author, Kate Fultz Hollis, first configured the AI models with the same assessment materials, including multiple-choice quizzes and short-answer questions, as taken by the students. Astoundingly, the AI models continued to rank among the very top in performance and outscored as many as three-quarters of the students. The results were published in the journal NPJ Digital Medicine.
While the AI did great in these examinations, the models were concerned with knowledge-based tests only. This is what actually influenced the researchers to conclude that those courses, which may be most susceptible to AI domination upon considering the underlying factor of knowledge retention, would be those. In contrast, more participatory or experiential courses may provide an academic framework best suited to human learning, where the skills, critical thinking, and decision-making processes go further than simple recalling of facts.
But Hersh’s musings about medical education reveal a larger historical shift. “When I was a medical student, one of my attending physicians told me I needed to have all the knowledge in my head,” he said. “Even in the 1980s, that was a stretch. The knowledge base of medicine has long surpassed the capacity of the human brain to memorize it all.”
For a field like medicine, which is so heavily dependent upon understanding vast amounts of data, students have always been required to retain copious amounts of information. However, this acquired experience underlines a greater trend that is gradually sweeping through almost all fields of study these days: the human brain does have its limits; in comparison, AI does store and retrieve information at incredible speeds, and that therefore makes memorization a lesser priority for any professional in the future.
Implications for Education: Can There Be a New Type of Learning?
The exponential growth in AI raises some fundamental issues for educators. If AI can outperform human capabilities on knowledge-based testing, what does this portend for learning in the future? Hersh stresses that while memorization may be less important, critical thinking and decision-making do remain important, particularly in health.
“There’s a lot of things that health professionals do that’s quite straightforward,” Hersh said. “But there are those situations where it becomes more complex, and you need to make judgment calls. That’s when it helps to have that broader perspective, without necessarily having to have every last fact in your brain.”
These findings point to a growing imperative to reimagine the assessment of students. Rather than having assessments center on recall, for example, future assessments may put a greater emphasis on more sophisticated competencies, including problem-solving, creative thinking, and judgment. One of the biggest challenges will be to decide precisely how such competencies should be assessed in a world when AI can already reproduce or surpass certain aspects of human cognition.
The report also raises other questions:. How will educational institutions deal with the potential for cheating as AI becomes even more integrated into daily life? Hersh is not especially concerned about cheating in his courses since he regularly changes what he teaches to reflect the latest scientific developments. “In any scientific area, there are new advances all of the time, and large-language models aren’t necessarily up to date on all of it,” he said. This is going to mean that instructors create more dynamic, nuanced assessments beyond the capability of AI to solve independently.
A Future of Human-AI Collaboration in Education
But even as skeptical as he might be, Hersh and others in this field do not believe AI will replace humans in education. It rather opens up another avenue of collaboration. The enormity of AI capacity to store knowledge will allow students and professionals to engage themselves in abstract thinking, analysis, and decision-making. According to Hersh, medicine “will always require the human touch.
The more functional AI is in various disciplines, the more it will assume the role of an assistant. Yes, in that sense, one can say that it will be able to aid and abet students in rapidly sourcing material, automating routine work, and providing instantaneous feedback. But critical and ethical judgments, especially in what have been termed as life-career professions—medical, law, and education, for instance—are humankind’s alone to take. It goes without saying that a human touch will always be required, no matter how advanced the technology gets.
The Way Ahead: A Combination Approach to AI in Academia
The challenge ahead will be the balance that will have to be achieved between AI and human learning. Students will have to develop skills that are beyond the capability of AI, like skills in critical thinking, ethical reasoning, and emotional intelligence.
Presumably, there is already a hint that this shift has begun to take place. Some educators have moved toward project-based learning where students attack real-world problems in collaborative teams or experiential learning that places a high value on skills such as leadership and creativity. Others are using the AI tools themselves to help students learn more efficiently, using personalized feedback and adaptable lesson plans to meet individual needs in learning.
For now, Hersh argues that the key is in how we use AI, not whether we use it. “Clearly, everyone should have some kind of foundation of knowledge in their field,” he said. “What is the foundation of knowledge you expect people to have to be able to think critically?”
Conclusion: The Future of Education in the Age of AI
With the inevitable rise of AI in education, it is not supposed to spell the end of human learning. Instead, this could usher in an era when knowledge would no longer be the sole basis of education. As learners and educators fumble their way in this new territory, the interest would likely shift away from what we know but toward how we think and apply our cognition to life situations.
Undeniably, AI will play a role in shaping the future of education, but this also forces us to seriously reconsider exactly what learning is. Precisely whether that is a future for better or worse depends largely on how we address some of these challenges and opportunities.
