Artificial intelligence is threatening to directly undermine the value of the flagship graduate law degree: the LL.M.
Law professors at the University of Minnesota used the wildly popular ChatGPT system to generate answers to exams in four courses last semester. The AI eked out a passing grade, but fell below most law students.
ChatGPT’s average C+ performance was shy of the B+ average for humans, the law school professors found. But the AI has important implications for LL.M. programs. It has sparked concern from many academics that students will use it to cheat in their essays and exams. These generative AI systems are capable of generating high-quality responses based on just a few words or sentences, including from legal texts.
Michele DeStefano, a Professor of Law at Miami Law and Guest Faculty at Harvard Law School and IE Law School, says: “I know that some law students cheat and ChatGPT will make it even easier to do so.”
“Each professor will have to decide whether ChatGPT should be able to be used during an exam -- just like they now decide whether an exam is open notes and whether students can access the internet during the exam,” she says.
An oath of integrity
However, her main concern is not cheating, given that being honest is integral to being a lawyer and it is “part of our oath” as a profession. She believes there could be upsides, too -- perhaps ChatGPT will encourage more professors to stress the importance of ethics and morals in all LL.M. classes.
Additionally, DeStefano says the “cool thing” about ChatGPT is that it might be an opportunity to enhance students’ ability to write more effectively, strategically, and persuasively. “You can push it in the direction you want, you ask it to fix paragraphs, re-order content, change the tone, and even argue against itself,” says DeStefano.
One of the most common complaints about lawyers is that they do not communicate (in memorandums, emails, or conversations) in a straightforward way, she adds. “To that end, law schools should teach students how to use ChatGPT ethically -- and also how to use it as a learning tool to make their writing better,” DeStefano adds.
An overnight sensation
ChatGPT became an overnight sensation when it launched in November last year, attracting more than 100 million active users two months after launching. It has been hailed as a breakthrough moment for AI, poised to disrupt some parts of the legal profession.
This new wave of AI systems has revived concerns about the technology’s threat to millions of jobs. But law firms insist that AI won’t replace their workforce, or reduce billable hours, at least in the short-term.
Some firms are adopting the much-hyped technology to find efficiencies for their attorneys and clients. Allen & Overy, the elite “magic circle” law firm in London, is using an AI chatbot to help lawyers draft documents or memos to clients, under the supervision of licensed legal professionals.
Stefano Barazza, Associate Law Processor at Swansea University in the UK, is “more interested than concerned” by AI systems, which he says “are here to stay”. It is “imperative” that law schools understand how best they can be used to assist students and lawyers.
He believes AI can be used to prepare students for legal careers that are increasingly being shaped by technology. “Against this background, it is essential that our students understand the technology, learn about its limitations and use cases, and develop solid competences in this field -- including the ability to reflect on ethical and regulatory aspects,” he says
Reimagining legal pedagogy
More widely, “we are in need of an urgent data-centric re-imagination of legal pedagogy”, he says. “This could be a fantastic opportunity to reconsider how students learn, in light of new technologies, and to proactively shape the future of legal practice, putting universities in the driving seat of legal innovation.”
Matthew D’Amore, Director of the Law, Technology and Entrepreneurship Program at Cornell Law School and Cornell Tech, adds that “our goal is to encourage students to use tools like this as research aids, not replacements for their own analysis”.
He adds: “The tools are not a replacement for legal judgment, which is an important part of how lawyers provide value. Simply reprinting what a tool offers is no better than copying from another source – it shortcuts the learning process and leaves students worse off as lawyers and professionals.”
In other words, they need to learn how to be better than the bot.
