AI improves exam success by 45%, depending on student skill level

Reading time: two minutes

Professors Jonathan Choi and Daniel Schwarcz conducted a study using GPT-4 to examine the effect of using AI on student’s work. The study found that the results of low performing students improved by 45% when using AI, while higher performing students were affected negatively from the use of AI.

According to the study, AI impacted students’ results differently, depending on their pre-existing knowledge.

The study focused on 48 law students studying at the University of Minnesota. They were given a final paper, which they completed without using AI. They then received prompt training before completing a second paper using GPT-4, OpenAI’s latest and most advanced AI model.

The results highlighted that on average when students used GPT-4, their results were 29% better than without. The study also found that the students who received lower results in the first exam improved their scores, on average, by 45% when using AI. However, higher performing students’ results dropped by 20% when AI was used in the second exam. Choi and Schwarcz said: “GPT-4’s impact depended heavily on the student’s starting skill level.” They wrote that their study “suggests that AI may have an equalising effect on the legal profession, mitigating inequalities between elite and nonelite lawyers”.

Choi and Schwarcz added: “We found that assistance from GPT-4 significantly enhanced performance on simple multiple-choice questions but not on complex essay questions.”

The quality of the prompts given to AI were also found to have affected results as “with basic prompts, GPT-4 was a mediocre student, but with optimal prompting it outperformed both the average student and the average student with access to AI”. This suggests that with optimal prompting AI could be more efficient for completing simple processes. This translates to how AI is currently being used by firms, with many beginning to implement the technology to conduct administrative processes like document reviews, allowing more time for employees to work on complex issues.