A chatbot powered by reams of data from the internet has passed its US law school exams after writing papers on topics ranging from constitutional law to taxation and torts.
ChatGPT from OpenAI, a US company that got a big cash infusion from Microsoft this week, uses artificial intelligence (AI) to generate text streams from simple prompts.
The results are so good that educators warn it could lead to widespread cheating and even signal the end of traditional classroom teaching methods.
Jonathan Choi, a professor at the University of Minnesota School of Law, gave ChatGPT the same test students face, including 95 multiple-choice questions and 12 essay questions.
In a white paper titled “ChatGPT Goes to Law School,” published Monday, he and his co-authors report that the bot scored an overall C+.
While that was enough to pass the exam, the robot was near the bottom of the class in most subjects and “bombarded” in multiple-choice questions involving mathematics.
not a good student
“While writing the paper, ChatGPT demonstrated a strong grasp of the underlying legal rules and consistently solid organization and composition,” the authors wrote.
But the bot “often struggled to spot questions when it gave open-ended prompts, a core skill on law school exams.”
Officials in New York and other jurisdictions have banned the use of ChatGPT in schools, but Choi thinks it could be a valuable teaching aide.
“Overall, ChatGPT is not a good law student acting alone,” he wrote on Twitter.
“But we want to work with humans, and language models like ChatGPT can be very useful for law students taking exams and practicing lawyers.”
Downplaying the possibility of cheating, he wrote in reply to another Twitter user that two-thirds of markers had found papers written by bots.
“(They) had a hunch, and their hunch was correct, because ChatGPT has perfect grammar and is somewhat repetitive,” Choi wrote.