So meta. AI-written paper says AI is the future of legal scholarship

By Karen Sloan

Jan 2 (Reuters) - A law professor tapped ChatGPT to write his latest article, which argues that generative artificial intelligence programs will "expand the horizons" of legal scholarship.

Suffolk University Law Dean Andrew Perlman, a longtime scholar in the field of law and technology, asked ChatGPT — an AI program trained to understand and produce human-like text — to “develop a novel conception of the future of legal scholarship” that hinges on how legal scholars might use generative AI.

Perlman concluded that the draft produced by ChatGPT “demonstrates the creativity and linguistic sophistication of a competent legal scholar.” His Dec. 26 article, titled “Generative AI and the Future of Legal Scholarship,” is the latest attempt to gauge the capabilities of generative AI in the law school arena. It appears in the Social Science Research Network, an online repository of scholarly articles.

A 2023 study found that GPT-4, an earlier version of ChatGPT, would pass the bar exam with flying colors — though some later questioned the methodology and the conclusion that the AI would score in the 90th percentile in most states. A separate study found that using GPT-4 on final exams improved test scores among lower-performing students but hurt the scores of their high-performing classmates. Yet another study concluded that generative AI helped law students complete writing tasks faster, but the technology did not improve their overall work product.

Perlman's ChatGPT-produced article said AI could

help scholars “uncover trends in judicial reasoning, legislative behavior, or administrative action that might otherwise go unnoticed.”

AI tools could also “produce new lines of argument, brainstorm counterarguments, or test hypothetical scenarios” much faster than human researchers alone. Scholars would then serve as curators, determining which concepts merit further study and which should be discarded.

In addition, the technology could help scholars integrate different methods of analysis, such as empirical legal studies and economic modeling to produce a “richer and more holistic” analysis, the article said.

The technology would not replace law professors, whose judgment and ethical oversight will remain crucial, ChatGPT concluded.

Perlman wrote in the article's epilogue that tomorrow's legal thought leaders will be those "who reimagine what scholarship can achieve when human wisdom guides artificial intelligence.”

Read more:

Bar exam score shows AI can keep up with 'human lawyers,' researchers say

These law students got to use AI on final exams. How'd they do?

(Reporting by Karen Sloan)