(CNN) — ChatGPT is smart enough to pass the prestigious graduate exam, though not with particularly high marks.
Powerful new artificial intelligence (AI) chatbots recently passed a law exam in four courses at the University of Minnesota and another at the University of Pennsylvania’s Wharton School of Business, according to professors at these centers.
To test ChatGPT’s ability to generate responses on exams in all four courses, University of Minnesota Law School professors blindly administered the exams. After answering 95 multiple choice questions and 12 essay questions, the chatbot averaged at the level of a C+ student, earning a low but passing grade for all four courses.
ChatGPT fared better on the Wharton Business Management Course Exams, earning a B to B- grade, equivalent to an 80 on a 100-scale. Wharton business professor Christian Terwish said in an article that ChatGPT did “a wonderful job” of answering basic operations management and process analysis questions. However, he added that the tool struggles with more advanced questions and makes “shocking mistakes” with basic math.
“These errors can be of large magnitude,” he wrote.
The results of this test come as a growing number of schools and teachers express concern about the immediate impact of ChatGPT on students and their potential to cheat on assignments. Some teachers have moved remarkably quickly to resume their tasks in the presence of ChatGPT, even though it’s unclear how widespread the tool’s use has been among students, or whether it actually leads to learning. How harmful can it be?
Since it was made available to users in late November, ChatGPT has been used to generate original essays, stories and song lyrics in response to user input. He has written summaries of research papers that mislead some scientists. Some CEOs have even used it to write emails or do accounting work.
ChatGPT uses large amounts of online data to generate answers to user questions. While it has gained traction, it has also raised some concerns, such as inaccuracies and its potential to perpetuate bias and spread misinformation.
John Choi, one of the law professors at the University of Minnesota, explained to CNN that the goal of the tests was to explore ChatGPT’s ability to help lawyers in their practice and for students on exams, whether their professors allow it or not. , because the questions often mimic words that lawyers use in real life.
Choi said, “The ChatGPT struggled with the more classic components of the law school exam, such as spotting potential legal issues and in-depth analysis to apply legal standards to the facts of a case.” “But ChatGPT can be very useful in creating a first draft that a student can refine later.”
According to Choi, collaboration between human and artificial intelligence is the most promising case for ChatGPT and other similar technologies.
“My firm view is that AI assistants will become the standard tool for lawyers in the near future, and law schools should be preparing their students for that scenario,” he said. “Certainly, if law professors want to continue testing simple memory of legal rules and principles, they will have to impose restrictions such as prohibiting Internet access during exams.”
Wharton’s Tervish also found that the chatbot was “remarkably good” at modifying its responses after receiving human cues, such as flagged responses after an error, which people associate with artificial intelligence. Suggests the ability to work.
However, in the short term, the quandary remains on whether students should be using ChatGPT. For example, public schools in New York and Seattle have already banned students and teachers from using ChatGPT on district networks and devices.
Noting that ChatGPT performed above average in its testing, Terwiesch told CNN that he doesn’t mind the restrictions being placed on students when they’re taking the test.
He said, ‘Restrictions are necessary. “At the end of the day, when you give a degree to a doctor, you want them to know medicine, not how to use robots. The same goes for certifying other skills like law and business,” he said. .
But Terwiesch believes that, ultimately, this technology still has a place in the classroom. “If we all have the same education system as before, we’ve wasted an incredible opportunity that came along with ChatGPT,” he said.
