Can AI Make a Case? AI vs. Lawyer in the Dutch Legal Context?

Title Can AI Make a Case? AI vs. Lawyer in the Dutch Legal Context
Author Lena Wrzesniowska
Affiliation University of Amsterdam
Region/Country Netherlands
Pages 1-49
Download
Permalink https://www.ijlet.org/4.3.1
DOI
Keywords GPT-4; AI; AI vs Lawyer; Legal Writing; Language Technology; LLM; Dutch Legal Context
Abstract The integration of AI, specifically GPT-4, into the legal field is a subject of both potential promise and intricate challenges. This thesis delves into the transformational possibilities of AI within the Dutch legal context, examining not only the quality and persuasiveness of AI-generated legal argumentation but also its competence in information retrieval, as measured by models’ ability to spot relevant legal issues. An experiment was conducted with 25 legal professionals, using a realworld Dutch case with the purpose to assess GPT-4’s capabilities against that of a human lawyer. To enable GPT-4 to handle case documents, the author first performed so-called co-reference resolution to remove ambiguities. Given the token limitations of GPT-4, a so-called prompt reducer technique was used to compress the text, retaining essential information. The above methods produced a coherent and full case summary within GPT-4’s token constraints. This case summary, together with original lawyer’s letter (denoted as Text A) was fed to ChatGPT-4 to obtain its AI-written alternative (Text B). The study subjects were presented with a case summary as well as both texts and asked for their preferences. The outcome of the experiment is as follow: 80% of participants chose the AI’s composed legal document, demonstrating a strong preference for both its linguistic competencies as well its ability to spot relevant legal issues. This preference for GPT-4 writing is very consistent among genders, age groups and professions surveyed. Contextualising these findings within the broader implications for legal practice, the thesis explores potential benefits including increased access to justice and transformation of certain legal procedures. Insights and recommendations are offered for legal professionals, considering the technological evolution and ethical considerations inherent in AI integration. Acknowledging the need for further exploration, the study recognises its own limitations and encourages replication to solidify the understanding of AI’s transformative role in the legal realm.