Rashri Baboolal-Frank
LLB, LLM, LLD, MBA, BSC
Associate Professor, Procedural Law, Faculty of Law, University of Pretoria
Sylvia Papadopoulos
BLC, LLB, LLM, LLD
Associate Professor, Private Law, Faculty of Law, University of Pretoria
Elsabe Schoeman
BLC, LLB, LLD
Emeritus Professor, Faculty of Law, University of Pretoria
Volume 58 2025 pp 282 - 301
Download Article in PDF
SUMMARY
The legal profession is being challenged to harness the predictive capabilities of AI as a fundamental tool reshaping the future of law. New technologies require a reimagining of the essence of legal services. Against this background, the University of Pretoria, Faculty of Law was invited to use ChatGPT to generate an arbitration award for the 2024 Johannesburg Arbitration Week (JAW) Young AFSA Moot. The AI-generated award was then juxtaposed with an award produced by a panel of expert human arbitrators. The primary objective of this experiment was to gauge the effectiveness and reliability of AI in producing legal judgments, a domain traditionally dominated by human expertise. This article discusses the findings and implications of this experiment, illuminating the evolving legal landscape and AI’s profound impact on the future of legal practice. It discusses the concept of generative AI and its application in law. It then explains and reflects on the arbitration award experiment. It goes on to analyse key issues as produced by ChatGPT and the human arbitrators. The article suggests several aspects that must be considered in the context of generative AI and legal practice and education. It concludes by emphasising that the integration of AI in legal practice promises to revolutionise the field, offering unprecedented efficiency and accuracy. However, for this transformation to be effective and ethically sound, both legal education and practice must evolve substantially.
1 Embracing the digital frontier
The legal profession is being challenged to harness the predictive capabilities of AI, not as a mere accessory but as a fundamental instrument to shape the future of law. It is not simply about adopting new technologies but reimagining the essence of legal services where digital and physical realities converge.
Susskind, in his 2023 Lionel Cohen lecture, stated that we are in the middle of two transformative decades in law; a transformation that surpasses the changes in law of the last two centuries.1 Reflecting on a Black and Decker anecdote, he asks the legal profession not to consider what lawyers traditionally provide -- a service requiring a significant level of skill and knowledge in the craft of lawyering, but rather to focus on the outcomes that we achieve, the solutions that we provide.2
The message conveyed is that most legal professionals tend to think about how we do things and how we can improve with the available technology, instead of considering how to do things differently or how things will be done differently. This type of thinking calls for a fundamental re-evaluation of the lawyer’s role and the tools we use.3
With this in mind, in April 2024, the University of Pretoria, Faculty of Law was invited to use ChatGPT to generate an arbitration award for the 2024 Johannesburg Arbitration Week (JAW) Young AFSA Moot. This AI-generated award was then juxtaposed with an award produced by a panel of expert human arbitrators: Patrick Lane SC, Malcolm Wallis, Fuyong Chen, James Banda, Susan Mutangadura, and Leyou Tameru. The primary objective of this experiment was to gauge the effectiveness and reliability of AI in producing legal judgments, a domain traditionally dominated by human expertise.
The comparison of an AI-generated award against one drafted by seasoned expert arbitrators, was important for several reasons. First, it provided insights into the potential of AI to enhance or disrupt the legal profession. As AI tools become more sophisticated, understanding their capabilities and limitations is crucial for integrating them effectively into legal practice. Second, this comparison helps in identifying areas where AI can improve efficiency, reduce costs, and augment human decision-making, thereby reshaping the delivery of legal services. Finally, it addresses the ethical and practical implications of relying on AI for legal judgments, ensuring that the integration of such technologies upholds the standards of justice and human dignity.
This article will discuss the findings and implications of the AI experiment of a prompt-based AI award compared against an award drafted by qualified arbitrators. Through this exploration, we aim to illuminate the evolving landscape of legal technology and its profound impact on the future of legal practice. The award is also analysed from a law substance paradigm of jurisdiction, formalities of contract and the duties and obligations as raised from the contract, which pertains to arbitration agreements.
To fully appreciate the impact of AI in legal practice, it is important to understand the underlying technology and therefore, the following section provides an overview of generative AI, detailing its capabilities and the advancements that have enabled its integration into the legal field.
2 Generative AI
According to the OECD, an ‘AI system’ is a machine-based system that operates to achieve explicit or implicit objectives. These systems can infer from the inputs they receive to generate outputs such as predictions, content, recommendations, or decisions, which can influence physical or virtual environments. AI systems can vary significantly in their levels of autonomy and adaptiveness after deployment.4
In a much similar definition, the EU’s landmark AI Act describes AI systems as machine-based systems designed to operate with varying levels of autonomy. These systems may exhibit adaptiveness after deployment and are intended to achieve explicit or implicit objectives. By inferring from the inputs they receive, AI systems generate outputs like predictions, content, recommendations, or decisions that can affect both physical and virtual environments. 5
ChatGPT is an example of Generative AI, a branch of artificial intelligence focused on creating new, original content through learned patterns and data. It has been developed by learning from vast, generalised datasets, enabling it to understand queries and generate text-based content, images, and video outputs. Generative AI, including ChatGPT, is the result of extensive training on enormous datasets. This training allows the AI to not only recognise but also generate complex text, video, and images from simple text prompts. ChatGPT can respond to nuanced queries, summarise large documents succinctly, and create content that is credible, correct, and creative. 6
ChatGPT’s capabilities extend to processing and analysing substantial volumes of legal text, as demonstrated by its proficiency in passing the American Bar Exam and producing high-quality legal drafting. The technology underlying ChatGPT is revolutionising business by streamlining the analysis of complex documents, enhancing document preparation, and facilitating thorough due diligence. It can generate almost anything the human would, in a matter of minutes.7
Generative AI trained on quality law specific data sets, can streamline workflows by assisting in the preparation of detailed legal documents and contracts, automating legal processes, analysing arguments and evidence, predicting outcomes and weaknesses of cases, conducting due diligence, and providing insights that were once only attainable through prolonged human effort. It offers the agility to respond to a myriad of legal inquiries, enhancing the quality and efficiency of legal services offered. 8
Examples of Generative AI that is trained specifically on legal materials, such as legislation, cases and judgements, articles, and law textbooks, show us that it reaches a level of capability that allows it to generate almost anything a human lawyer would, in a matter of minutes. This next-level AI further streamlines legal specific workflows and processes, assists in preparing detailed legal documents, contracts, and arguments, automates legal processes, and as we show below, can apply logic, reasoning and law to a set of facts in making an arbitral award. 9
Generative AI is not the only technology in a dynamic landscape of legal technology. There are several technologies that are already in widespread use, significantly reshaping the business of law. Examples include:
- Legal Case Management Software: This includes cloud-based applications that help in processing, organising, and managing legal cases more efficiently, enhancing productivity and profitability for law firms. 10
- Litigation Management: Tools in this category assist with tracking, organising, and scheduling activities related to litigation cases and court hearings, making it easier for lawyers to manage caseloads and collaborate within legal teams. 11
- Virtual Legal Assistants and Chatbots: AI-powered virtual assistants are being used to automate legal processes, provide real-time support for legal matters, and handle tasks like legal document drafting, research, and filing. 12
- Identity Management and Client Data Protection: These technologies focus on securing access control of digital identities, authentication for law firms, and protecting client confidentiality against cybersecurity threats. 13
- Automated Document Review and Smart Legal Contract Management: These involve the use of AI to streamline the review of documents and manage contracts more effectively. 14
- Immersive Technology and Advanced Computing: These are emerging areas that include the use of virtual and augmented reality for legal education and training, as well as advanced computing techniques for legal research and analytics and presenting evidence in/to courts.15
With a foundational understanding of Generative AI, we can now turn our attention to a practical application of this technology in a legal context. The moot case analysis below provides an examination of how AI and human arbitrators approached the same set of facts, highlighting key issues and their resolution.
3 The University of Pretoria AI generated ChatGPT award 16
3 1 The Process
The process of generating an arbitration award by ChatGPT required that all the necessary documentation generated by AFSA was uploaded, namely the fictitious facts set in the annexure below and the instructions and rules pertaining to the drafted pleadings of the three teams that participated in the moot court competition.
Our observation was that the ChatGPT results from the 4.0 version (used in experiment) were vastly different to the free initial 3.5 version. With the 3.5 version, there were inaccuracies, for example hallucinations and incorrect information, however, it displayed more accuracy and depth to the discussions of the case law. The information given for the free 3.5 version was more concise than the paid version, which gave more details of cases, and explanation of the substantiated reasons in applying the given facts uploaded to produce reasons and application to the facts.
The ability of ChatGPT to process a plethora of documents is impressive to say the least, as the written submission of the three teams was approximately 88 pages which was copied and pasted into ChatGPT from the word format of the documents, and the analytical response generated by ChatGPT was complete in approximately 30 seconds. Thus, the ability of ChatGPT to process information at a fast and efficient rate is truly remarkable.17
If prompted18 for case law, ChatGPT will generate case law with a brief synopsis of one to three lines of salient points in the decision. When we verified the cases cited, we were surprised to find that all the citations were accurate and correctly used in the context. We expected hallucinations or fabrication of text or citations to be generated.19 In this award there were none. The AI prompts and human research was used to verify the veracity and authenticity of the information produced.
A case in point of hallucinations and fabrication is the matter of Parker v Forsyth N.O. and Others which was an earlier case involving the use of ChatGPT 3.5 where AI-generated legal research was used by the plaintiff's attorneys. The plaintiff's counsel relied on ChatGPT to produce a list of cases that purportedly supported their arguments. However, these cases turned out to be fictitious, with the names, facts, and decisions entirely fabricated by the AI. The court criticised the plaintiff's attorneys for not verifying the AI-generated content, emphasising the importance of independent, accurate legal research. The court found that while there was no intent to mislead, the incident underscored the risks of over-reliance on unverified AI tools in legal proceedings. As a consequence, the court dismissed the plaintiff's action with costs, reinforcing the need for legal professionals to diligently verify their sources and maintain the integrity of the legal process.20
An important aspect when interacting with ChatGPT is asking for a more in-depth explanation of answers received. So, for example, there was reference to principles of law for contract, and when asked for specific details of that breadth, ChatGPT provided accurate cases through honing in on and isolating the specific contractual principles. 21
In preparing the AI award, the only adjustments made was to technically edit the text into a format acceptable to practitioners and academics (i.e. fonts, spacing, tabs and paragraphs etc). No text whatsoever was amended and the ChatGPT generated text was used in the AI award verbatim.
AI has advanced significantly in a short time, making it noteworthy that the submissions uploaded to ChatGPT were the written arguments only, but it is possible for the oral submissions made to the tribunal panel to be recorded, transcribed by AI tools, and submitted to ChatGPT for analysis, though this was beyond the scope of the current testing exercise. The written submissions were analysed to produce a reasoned award.
3 2 A Reflection
It is important to emphasise that ChatGPT is a powerful AI tool that aids and enhances accuracy. Judges in the United States have admitted to using ChatGPT to analyse and draft their judgments by inputting witness testimony.22 This practice of embedding AI as a tool can help with evidentiary burden, documentary evidence, oral evidence and analysis of bundles of evidence. It highlights the changing landscape of the legal profession, where artificial intelligence is increasingly being used to augment the skill sets of lawyers, practitioners, and academics. Importantly, this integration should be seen as a collaborative effort, combining the strengths of both AI and human expertise, rather than choosing one over the other.
For instance, some judges in the U.S. have issued guidance on the use of Generative AI in court filings, following incidents where attorneys cited non-existent cases generated by ChatGPT.23 This cautious but progressive approach indicates a growing acceptance and reliance on AI tools to enhance legal processes, if there are safeguards in place to ensure accuracy and reliability.
There are still ethical and legal concerns that arise from the perspectives of transparency, accountability and integrity. It is trite that machines cannot be held accountable for mistakes and that humans still need to take the responsibility for errors that may arise in the write up and presentation of the information to the clients. This means that the ethical accountability still vests upon the lawyers that are using the AI to ensure the veracity of the information by reading the cases directly, to ensure that there are no hallucinations and made-up facts and that the law is accurate and serves the clients’ interests and the interests of justice.
To better understand the differences in performance between human and AI-generated awards, we now provide a detailed comparison based on several key criteria.
With the comparison table providing an overview, we move into the specific key legal issues addressed by both the human and AI awards to see how each handled the complexities of the case.
3 2 2 Analysis of Key Legal Issues
The following were the key issues that we identified:
a. Jurisdiction: Jurisdiction determines the authority of the tribunal to hear a case. A valid arbitration agreement and the adherence to procedural rules are central for establishing jurisdiction.
- Human Award: The human arbitrators provided a detailed examination of the arbitration agreements in the email correspondences, citing the UNCITRAL Model Law and relevant South African legal principles. They meticulously argued that the language used by the parties indicated a mutual agreement to arbitrate under the AFSA rules, despite minor ambiguities.
- AI Award: The AI also recognised the arbitration agreement but was more direct and concise in its reasoning. It identified the essential elements of mutual consent to arbitrate, reflecting a clear understanding of the legal principles involved.
- Comparison: The AI’s efficiency in summarising the jurisdictional argument was notable, potentially saving time without compromising on the accuracy of the legal analysis. However, the human award provided a richer contextual understanding and more detailed citations, which could be critical in complex cases.
b. Contract Formation: Contract formation requires an offer, acceptance, and mutual intent to be bound. In this case, the timing and conditions of the acceptance were critical elements to be considered.
- Human Award: The human arbitrators delved deeply into the principles of offer and acceptance, including the nuances of electronic communications under the Electronic Communications and Transactions Act 25 of 2002. They questioned the precise moment of acceptance and its implications for contract formation.
- AI Award: The AI identified that a contract was formed when Paul Smith's acceptance email was sent, referencing the same Act. It highlighted the importance of electronic communication in contract formation but was less thorough in discussing potential ambiguities in the timing of acceptance.
- Comparison: The AI demonstrated a clear and correct application of the law, potentially outperforming humans in efficiency. However, the human arbitrators’ more nuanced discussion provided a better understanding of the complexities involved, which is essential for thorough legal reasoning.
c. Single Contract v Independent Transactions: Determining whether transactions are part of a single contract or independent transactions requires examining the intent of the parties and the conditional nature of the agreements at hand.
- Human Award: The human panel carefully analysed the intent behind the transactions, considering Paul Smith’s desire to own both diamonds as a single commercial objective. They examined the communications for indications that the agreements were conditional upon each other.
- AI Award: The AI identified the conditional nature of the transactions but was more definitive in concluding that the agreement with Fernanda de Sousa was standalone. It emphasised the clear acceptance of the offer without explicitly linking it to the purchase of the second diamond.
- Comparison: The AI’s clarity and decisiveness were strengths, potentially offering quicker resolution. However, the human award’s detailed analysis of the parties’ intent provided a more comprehensive understanding, crucial for cases with significant commercial implications.
The formation of a contract hinges on the clear communication of acceptance and the offeror’s awareness of it, with the timing of acceptance in electronic communications being particularly relevant. The human arbitrators delved extensively into the principles of electronic communication, addressing whether an email constitutes acceptance when sent or when received and read. They carefully evaluated the ambiguity in Badela’s email concerning the timing of his commitment to another buyer.
In contrast, the AI concluded that the contract was formed when the acceptance email was sent, aligning with modern electronic communication principles. While the AI recognised the potential for ambiguity, it did not delve as deeply into the timing issues and the offeror's awareness. This comparison highlights that the AI excelled in efficiently applying straightforward legal principles, whereas the human award’s more detailed examination of timing and awareness underscored potential pitfalls that the AI might overlook, demonstrating the importance of thorough legal scrutiny. AI sometimes missed the deeper context and potential ambiguities in legal communications, which could lead to oversimplified conclusions.
Both the AI and human awards demonstrated strengths and weaknesses, highlighting the complementary potential of AI and human expertise in legal practice. AI’s efficiency and clarity can enhance legal processes, while human judgment ensures nuanced and thorough legal reasoning. The future of legal practice will likely benefit from integrating both approaches, leveraging AI’s capabilities to support and enhance human decision-making.
Having examined the nuanced differences and similarities between the human and AI-generated awards, it is essential to understand how these findings resonate with the broader legal community present at the conference. The following section delves into the poll results from the Johannesburg Arbitration Week (JAW) conference, offering valuable insights into the acceptance of AI in legal contexts.
3 3 The poll
Participants were invited to participate in a poll at the Johannesburg Arbitration conference through scanning a QR code that exposed four questions, and one had to simply select the answer, which took less than five minutes to complete on the last day of the conference. The attendees had been given the printed hard copies of both the human and AI awards without indicating which was which and simply labelled Award A and Award B.
The poll was based on four questions that was formulated by the AFSA Chairman to adduce the impact of the award.
- Which award did the humans write? Award A or Award B
- Which award was correct award? Award A or Award B
- Could the result have gone either way? Yes or No
- Do considerations of speed and costs override the human touch? Yes or No
The results of the poll were as follows:
- 69,4% Award A and 30,6% Award B. Award A was indeed the one that the humans wrote.
- 56,5% Award A and 43,5% Award B. So, there was almost a fair split of opinion regarding the correctness due to the vagueness of the facts.
- 52,2% Yes and 47.8% No. An almost even split of people believed that the award could have gone either way, due to the ambiguity in the facts.
- 80,4% No and 19,6% Yes. An overwhelming result that the majority believed that human element outweighs the benefits of costs and speed, but this may be because the legal profession is not willing to cede their jobs and income to AI and the market may not see it the same way.
The poll results suggest a nuanced view of AI’s role in legal contexts among legal professionals. A significant majority of participants were able to correctly identify the human-generated award, reflecting an awareness of the distinctive qualities of human legal reasoning such as the detail in unpacking case law and the legal issues elucidated from a case depth discussion. However, the nearly even split on which award was ‘correct’ demonstrates that the AI-generated award was sufficiently credible to challenge human expertise.
The 52-48 split on whether the result could have gone either way underscores the ambiguity inherent in the case or suggests that AI, when properly trained, can produce legally sound decisions comparable to those made by humans. This finding is pivotal as it indicates a growing acceptance of AI’s potential to handle complex legal issues.
The most telling result is the overwhelming preference for the human touch over considerations of speed and cost. This suggests that, despite recognising the efficiency and cost-effectiveness of AI, the legal profession still highly values the human element in legal decision-making. Additionally, this preference may partly stem from concerns about job security and the potential replacement of legal professionals by AI technologies.
These findings indicate a cautious but growing acceptance of AI in legal contexts as an enhancement tool to read and interpret oral and evidentiary documents. Legal professionals recognise the potential of AI to enhance efficiency and reduce costs, which could drive its broader adoption in tasks such as legal research, document review, and even decision-making in straightforward cases. However, the results also underscore the importance of maintaining human oversight to ensure that AI-generated decisions uphold the nuanced and ethical standards of the legal profession. Furthermore, it highlights the need for balanced integration strategies that consider both technological advancements and the preservation of human jobs within the legal sector.
The acceptance of AI in legal practice will likely continue to grow as technology advances and legal professionals become more familiar with its capabilities and limitations. The integration of AI offers the potential to transform legal practice by enhancing the efficiency and accuracy of legal services while allowing human lawyers to focus on more complex and strategic aspects of their work due to the manner in which AI can interpret and summarise information presented from files uploaded.
The poll results raised important questions about the future integration of AI in legal practice from an accountability, transparency, integrity and reduction of legal costs. The next section explores these implications, offering some ideas on how AI can be effectively and ethically incorporated into legal education and practice.
4 Innovation in law: The shift from enhancement to transformation
The legal profession's technological metamorphosis is not just about enhancing practices with technology but redefining them. Legal professionals are called upon to look beyond the hammer and to envision the house it can build.24 This means embracing AI as a core aspect of legal work, where legal AI powered bots might predict outcomes and suggest legal strategies with nuanced precision that exceeds human capabilities.25 As legal professionals, we cannot afford to be technologically myopic because it is clear that the technology is increasingly capable, and that capability is increasing at an exponential rate.
If we look at neural networks that are at the heart of machine learning and large language models that form the basis of ChatGPT and other generative AI tools,26 it is estimated that the capabilities are doubling every 3 ½ months. Sam Altman himself is quoted as saying that GPT-3.5 has limitations but that the future of AI has the possibility of limitless aspects of enhancements for the legal profession.27 We were at ChatGPT 4.0 at the time that the moot award was produced and are currently using ChatGPT 5.
AI is capable of e-discovery, contract management, due diligence, data analytics and chatbots, dissecting legal texts with astonishing precision, predicting the possible outcomes of a court case, making predictions on offenders’ chances of recidivism to mention a few aspects.
As we progress, legal professionals must engage with these technologies to remain competitive. It is this article’s submission that the following constitute some of the aspects of AI technologies that we should be considering:
- Training and Digital Literacy: Every legal professional must prepare for the era of intelligent machines. This entails a comprehensive understanding of how the digital influences legal proceedings, client engagement, and the management of information, with a particular focus on cybersecurity. Discussing AI from an academic and legal practitioner perspective but implementing and using it effectively is challenging.28
Digital literacy transcends mere proficiency with technology, it encompasses a thorough understanding of how digital systems function and their strategic application to meet specific goals, i.e. it’s not just the ability to use software for document management or research.29 It involves adeptly navigating virtual courtrooms, mastering electronic filing systems, and rigorously adhering to security protocols to safeguard client data against pervasive cyber threats.30
AI literacy is a nuanced segment of digital literacy, specifically in its application within the legal domain. It involves discerning the appropriate use of AI tools to enhance efficiency and outcomes in legal work. This not only includes knowing which questions to pose to these tools but also critically evaluating their outputs and grasping the underlying technology to assess their reliability and any inherent biases (the dangers within). Furthermore, it requires an acute awareness of the ethical implications of AI, ensuring that its use upholds the legal profession’s dedication to confidentiality, impartiality, equality, and justice.31 - Legal-Tech Tools: Legal-Tech is already widely implemented in practice. It is our submission that we will also see the significant rise in regulatory technology tools (Reg-Tech) i.e. the same mechanisms that prevents you from making illegal moves on a chess-bot board will be upscaled and eventually compliance will be embedded in systems with real time risk analysis.32 Compliance with all laws affects all systems and organisations, no sector is immune to compliance measures as it concerns alignment with the current laws and international standards.
- Disruption on Fee Structures: In an era where artificial intelligence is pivotal to legal operations, traditional fee structures within law firms are on the brink of disruption as AI tools can replace junior staff output.33 The faster, accurate and more cost-efficient practice of law, will force the uptake of AI. Therefore, AI’s potential to democratise access to justice is inevitable, reshaping public expectations around the delivery and cost of legal services.34
Consider this scenario: if Law Firm A bills three hours for drafting court documents and Law Firm B bills for only thirty minutes to review an AI-generated document, it's clear where business is likely to gravitate. Moreover, there’s an ethics and integrity consideration; lawyers cannot justifiably bill for three hours if AI assistance reduces the task to thirty minutes.
With legal-focused AI, chatbots and sophisticated algorithms, basic legal guidance will become more accessible and affordable. This shift will gradually alter the public's expectation of legal service delivery and pricing. For law firms, this evolution challenges the status quo of billable hours and fee arrangements. Clients, empowered with alternatives, may no longer accept traditional pricing models as the only option.35
Incorporating AI into service offerings is not just about cutting costs; it's about creating value. It's about enhancing the client experience by providing swift, accurate, and cost-effective legal solutions, and it's about law firms redefining their roles--as not merely providers of legal services but as holistic problem-solvers in a complex digital world.36
Moreover, transparency becomes paramount. Clients today are more tech-savvy and expect to know if and how AI is being utilised in handling their matters. They are interested in understanding how the use of AI might translate into cost efficiency and better, faster resolution of their legal issues.37 - Prediction and Data Analysis: Predictive analytics in the legal industry utilises data, statistical algorithms, and machine learning to predict future events or outcomes. It transforms legal decision-making through:38
- Case Assessment: Predictive analytics analyses past case data to predict the success or failure of a case, helping lawyers make informed decisions on strategies, negotiations, and resource allocation.
- Litigation Risk Management: By assessing historical data, predictive analytics estimates risks associated with litigation, aiding in decisions to pursue legal action or explore alternatives.
- Early Case Assessment: Predictive analytics evaluates case viability and potential outcomes early in the process by analysing relevant data, assisting in strategy development and decision-making.
- Legal Research and Precedent Analysis: Predictive analytics accelerates legal research by identifying relevant precedents and extracting insights from large volumes of legal data, enhancing argument quality. 39
- Settlement Negotiations: Insights from predictive analytics on settlement values based on historical data facilitate informed decisions during negotiations and in managing expectations.
- Resource Allocation and Case Prioritisation: Predictive analytics assists in allocating resources effectively by predicting the effort required for different cases, optimising case prioritisation, and improving operational efficiency.40
- Ethics and Transparency: The ethical considerations in AI model training are crucial, especially concerning data quality and sourcing.41 We should be taking note of international best practices such as the OECD AI Principles to align the AI tools accordingly:
- Inclusive Growth, Sustainable Development, and Well-being: AI should benefit people and the planet by driving inclusive growth, sustainable development, and well-being.
- Human-Centred Values and Fairness: AI systems should respect the rule of law, human rights, democratic values, and diversity. This includes ensuring fairness and avoiding bias.
- Transparency and Explainability: AI actors should commit to transparency and responsible disclosure regarding AI systems to ensure that people understand AI-based outcomes and can challenge them if necessary.
- Robustness, Security, and Safety: AI systems must function robustly and securely throughout their life cycles and potential risks should be continually assessed and managed.
- Accountability: AI actors should be accountable for the proper functioning of AI systems and for compliance with the accepted regulatory principles.
- Fairness In Decision Making - what rights and freedoms do we expect data driven technology to respect?42 What steps are necessary for ensuring that AI technologies are trustworthy, respect human rights, and align with democratic values?
- Liability-Related Risks: Clear regulatory frameworks are necessary to manage AI’s risks, ensuring outputs are checked, and that the quality of training data is maintained commensurate with the standards expected in administering justice. Mitigation requires identification of risks in design and development, cybersecurity, privacy, preventing bias, violation of IP protections, unfair trade practices, cybercrime, and fraud.43
- Avoiding Technological Determinism: AI is a tool to assist, not replace, the human element in law.44
Susskind challenges lawyers to decide whether to compete with AI or to build alongside it. Competing suggests a business-as-usual mindset, gradually adopting automation but with the complacency that AI will not supplant the role of a lawyer. Building, on the other hand, is proactive; it’s about embracing the retooling and enhancement of our skill sets.45 Whichever way, it necessitates a rethinking of legal education, to prepare for future roles such as a legal data analyst, legal process engineer, e-discovery specialist, cybersecurity specialist, compliance technologist, legal technologists, legal data scientists, and legal data visualisers, all of whom will have to be multidisciplinary specialists in law and technology.46 The aspects that is critical is using AI tools responsibly, with accountability, accuracy and relevance.
This will require a pivot in legal education towards curriculum innovation, incorporating digital and AI literacy, ethical training, and hands-on technology management. Transdisciplinary learning is essential for developing ‘T-shaped’ professionals with legal knowledge and broad technology acumen. Legal education must evolve to ensure that human-centred lawyering and soft skills remain at the core, even as technology becomes more prevalent.47 The AI award generated, shows that junior personnel need to upskill and stay relevant otherwise AI can replace them as human capital.
Curriculum innovation means that legal education must evolve to include digital and AI literacy, teaching not just the use of AI tools but also the ethical and societal implications of this technology in legal practice. This includes specialised courses in legal-tech, regulatory-tech, cybersecurity, and integrated programmes that weave computer science, and data analysis, into legal studies.
Transdisciplinary learning means that the lawyers will need to combine depth in legal knowledge with a breadth of interdisciplinary expertise. This involves mastering a specific legal domain while also engaging with technology, business, and data analysis, enabling collaboration with a spectrum of experts in diverse fields. In practical training, young lawyers should be provided with practical experience with Legal-tech, Reg-tech, Fintech
Analytical and critical thinking skills will be more important than ever and the ability to understand and critically assess AI predictions will be required. A good understanding of data ethics and privacy must be integral to legal training.
Future legal professionals need to be educated on the effective use of legal technology. This encompasses selecting the right tools for the firm's needs, overseeing the implementation of these systems, ensuring they integrate seamlessly with existing processes, and staying vigilant on updates and advancements. Law students must be well-versed in the potential liabilities that come with using AI, including regulatory compliance and ethical considerations.
The analysis and findings from the AFSA Moot Court Experiment reveal both the potential and limitations of AI in legal decision-making. The use of AI tools generating an award from provided reading, still needs the human intervention, of checking, analysing and applying the facts to the conclusion, and adding more depth to the analysis of case studies. As we reflect on these insights, it is imperative to consider the broader implications for the legal profession keeping in mind that the AI capabilities continuously grow exponentially. The concluding section offers a perspective on the integration of AI in legal practice, emphasising the need for comprehensive AI literacy and ethical standards.
By embracing AI as a complementary tool, the legal profession can enhance its capabilities while upholding its commitment to justice and ethical practice. This approach will enable lawyers to navigate the complexities of the digital age effectively, ensuring that technology serves the greater good in the pursuit of legal excellence.
5 Conclusion
As we embrace the journey ahead, the legal profession must leverage the power of technology and AI while maintaining an unwavering commitment to justice and human dignity. The future will be shaped by those who can navigate and innovate within the new technological landscape, ensuring that technology serves the greater good.
We conclude with a short table detailing some of the benefits and risks of AI for lawyers, as from the aforementioned discussions.48
Looking into a proverbial crystal ball, the integration of AI in legal practice promises to revolutionise the field, offering unprecedented efficiency and accuracy. However, for this transformation to be effective and ethically sound, both legal education and practice must evolve. Legal education should incorporate comprehensive AI literacy, ensuring that future lawyers understand the capabilities, limitations, and ethical considerations of AI technologies. This includes courses on AI and law, hands-on training with legal-tech tools, and interdisciplinary programs that combine legal studies with code, data science and data analytics.
In practice, law firms must adopt a balanced approach to AI integration, leveraging AI for tasks such as document review, legal research, and predictive analytics, while ensuring human oversight in decision-making processes to maintain ethical standards. This involves developing robust frameworks for verifying AI outputs, mitigating biases, and safeguarding client confidentiality. Moreover, continuous professional development programs should be established to keep legal practitioners abreast of the latest AI advancements and their implications for the legal landscape.
By embracing AI as a complementary tool rather than a replacement, the legal profession can enhance its capabilities while upholding its commitment to justice and ethical practice. This approach will enable lawyers to navigate the complexities of the digital age effectively, ensuring that technology serves humanity.
1. Susskind 2023 Leonard Cohen lecture. See full lecture at: https://youtu.be/uAcNvdgodvA?si=xOrJTqOmfnQiKNLU (accessed 25-03-2024).
4. OECD Legal Instruments “Recommendation of the Council on Artificial Intelligence” adopted on 22/05/2019 and amended on 03/05/2024, article I definition for ‘AI system’ available at: https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449 (accessed 01-04-2024).
5. European Parliament “Corrigendum to the position of the European Parliament adopted at first reading on 13 March 2024 with a view to the adoption of Regulation (EU) 2024 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) P9_TA(2024)0138 (COM(2021)0206 - C9-0146/2021 - 2021/0106(COD))” 2024 (hereinafter the EU AI Act) available at: https://www.europarl. europa.eu/doceo/document/TA-9-2024-0138-FNL-COR01_EN.pdf (accessed 28/03/2024). The EU’s Artificial Intelligence Act, which was approved by the European Parliament on 13 March 2024 focuses on creating a framework for the ethical development, deployment, and governance of AI technologies across EU member states.
6. IBM Stryker, Scapicchio “What is Generative AI?” (2024). Available at: https://www.ibm.com/topics/generative-ai (accessed 30-03-2024).
7. Arredondo “GPT-4 Passes the Bar Exam: What That Means for Artificial Intelligence Tools in the Legal Profession” 2023 CodeX-The Stanford Center for Legal Informatics. Available at: https://law.stanford.edu/2023/04/19/gpt-4-passes-the-bar-exam-what-that-means-for-artificial-intelligence-tools-in-the-legal-industry/ (accessed 01-04-2024).
8. See for example products by LexisNexis at https://www.lexisnexis.com/en-us/products/lexis-plus-ai.page or Thomson Reuters Practical Law Available at: https://africa.thomsonreuters.com/en/products-services/legal/practical-law.html (accessed 02-04-2024).
9. Thomson Reuters has, for example, expanded its legal technology offerings with new generative AI functionalities on its Practical Law platform. Adding in a “Contracts Clause Finder”, a comprehensive clause library integrated with Microsoft Word, to help in contract drafting. Additionally, their CoCounsel tool uses generative AI to assist in statement preparation, document summarization, and data extraction. See also Lexis plus AI which uses the fastest legal generative AI for conversational interactive searches, for drafting legal documents, legal argument, contractual clauses and concise client communications, providing legal summaries in seconds, uploading document extract and summary analysis, and hallucination-free linked legal citations.
10. Law firms such as Bowmans and Cliffe Dekker Hofmeyer Attorneys have their customised AI tool and systems. Harvey is also a UK tool that is being used.
11. Caselines is utilised by the Gauteng Division and has now rolled out to the other divisions in the nine provinces under the Constitutional Court Judge President, Justice Maya.
12. Firms and companies have invested in automated chatbots, that have specified prompts to direct business to the relevant people.
14. Copilot in Microsoft and Adobe assist with summarising documents that have been uploaded for viewing.
15. Different computer software is utilised for finding documents in discovery when thousands of pages have been exchanged to streamline the navigation of the bundle of documents before the court.
17. Khlystova-Gowda “Friend or Foe-AI’s Invasion of the Legal Battlefield” 2023 Cardozo Arts & Entertainment Law Journal 355.
18. The prompts were “analyse the facts uploaded and write a reasoned award”, “write a reason award with case law incorporating the facts uploaded”.
19. Bates Norum 2023 “Changing All the Time: Ai’s Impact on Humanity’s Role in Common Law Development and Interpretation” Boston University Law Review 2215-2249 at 2233. See also Wright “Professionals Beware: The Opportunities and Risks of Generative AI in Legal Practice” 2023, Available at: https://opus.lib.uts.edu.au/bitstream/10453/176325/3/Professionals%20 Beware%20The%20Opportunities%20and%20Risks%20of%20Generative%20AI%20in%20Legal%20Practice.pdf (accessed 02-04-2024).
21. The prompts inserted: “Explain the cases used in the arbitral tribunal’ or “explain the facts and legal issues of the case law used in the arbitral award.” The prompt used was to explain the cases used both from South African and English jurisdiction.
22. Standing Order Re: Artificial Intelligence (“AI”) Cases Assigned to Judge Baylson (E.D. Pa. 2023). Available at: https://www.paed.uscourts.gov/documents/standord/Standing%20Order%20Re%20Artificial%20Intelli gence%206.6.pdf; and Standing Order for Civil Cases Before Magistrate Judge Fuentes (N.D. Ill. 2023), https://www.ilnd.uscourts.gov/_assets/_documents/_forms/_judges/Fuentes/Standing%20Order%20For%20Civil %20Cases%20Before%20Judge%20Fuentes%20rev%27d%205-31-23% 20(002).pdf (accessed 25-06-2024).
23. Standing Order Re: Artificial Intelligence (“AI”) Cases Assigned to Judge Baylson (E.D. Pa. As above. See the US cases: a) Mata v Avianca, Inc. 2023 WL 4114965 at 3, 9 (S.D.N.Y. June 22, 2023) (imposing sanctions on attorney who used ChatGPT for legal research even though attorney was not aware that ChatGPT could make up cases, and failed to check whether the citations were real or accurate); b) Park v Kim 2024 WL 332478 (2d Cir. Jan. 30, 2024) (referring attorney to grievance panel for relying on ChatGPT without checking its results and for citing non-existent decision in reply brief) and c) People v Crabill 2023 WL 8111898 at 1 (Colo. O.P.D.J. Nov. 22, 2023) (suspending attorney for violating various ethical rules by failing to check cases provided by ChatGPT).
24. American Bar Association, Stepka “Law Bots: How AI Is Reshaping the Legal Profession” (21-02-2022). Available at https://www.americanbar.org/groups/business_law/resources/business-law-today/2022-march/law-bots-how-ai-is-reshaping-the-legal-profession/ (accessed 23-03-2024).
26. Lawton “What Is Generative AI? Everything You Need To Know” (undated) TechTarget. Available at https://www.techtarget.com/searchenterpriseai/definition/generative-AI (accessed 23-03-2024).
27. Altman: OpenAI CEO on GPT-4, ChatGPT and the Future of AI as interviewed by Lex Fridman Podcast #367 (2023). Available at: https://youtu.be/L_Guz73e6fw?si=q6F-Np06Bbb5luE7 (accessed 23-03-2024).
28. The Ethical AI Law Institute “Tech Literacy in the Legal Industry: Is the Legal Profession Finally Waking Up to the Need for Legal Tech Literacy?” (2023). Available at https://ethicalailawinstitute.org/blog/tech-literacy-in-the-legal-industry/ (accessed 01-04-2024).
30. As above. See also American Bar Association (ABA) “Cybersecurity for Law Firms: What Legal Professionals Should Know” (2022). Available at: https://www.americanbar.org/groups/law_practice/resources/tech-report/2022/cybersecurity-law-firms/ (accessed 01/04/2024).
31. As above. See also Garingan and Pickard “Artificial Intelligence in Legal Practice: Exploring Theoretical Frameworks for Algorithmic Literacy in the Legal Information Profession” 2021 Legal Information Management 97-117.
32. American Bar Association (ABA) “Legal Innovation and AI: Risks and Opportunities” (2024). Available at: https://www.americanbar.org/groups/law_practice/resources/law-technology-today/2024/legal-innovation-and-ai-risks-and-opportunities/; Murad “Seven Ways Technology Is Revolutionizing Legal Practice” (2024) available at https://kpalawyers.ca/2024/01/19/seven-ways-technology-is-revolutionizing-legal-practice/; Caret Legal “2024 Law Firm Technology Trends to Be Aware Of” (2024). Available at: https://caretlegal.com/blog/law-firm-technology-trends-to-be-aware-of/ (all accessed 29-06-2024).
33. LexisNexis “How Generative AI is Disrupting Law Firm Billing Practices” (2023). Available at: https://www.lexisnexis.com/community/insights/legal/b/thought-leadership/posts/how-generative-ai-is-disrupting-law-firm-billing-practices (accessed 01/04/2024).
34. Justice Innovation Project Stanford Legal Design Lab “Opportunities & Risks for AI, Legal Help, and Access to Justice” (2023). Available at: https://justiceinnovation.law.stanford.edu/opportunities-risks-for-ai-legal-help-and-access-to-justice/ (accessed 01-04-2024).
36. Legal Tech Solutions Thomson Reuters “Creating new advisory service and pricing models with AI” (2024). Available at: https://legal.thomson reuters.com/blog/creating-new-advisory-service-and-pricing-models-with-ai/ (accessed 29-06-2024).
37. American Bar Association (ABA) “The Time Is Now To Incorporate Ai Into Your Legal Services Functions” (2024). Available at: https://www.american bar.org/groups/law_practice/resources/law-technology-today/2024/the-time-is-now-to-incorporate-ai-into-your-legal-services-functions/ (29-06-2024).
38. See https://www.technology-innovators.com/predictive-analytics-in-legal-decision-making-improving-case-outcomes/ (accessed 23-11-2023).
41. Nikolic “Ethical Generative AI At The Crossroads Of Innovation And Responsibility” (2024). Available at: https://www.capgemini.com/insights/expert-perspectives/ethical-generative-ai-at-the-crossroads-of-innovation-and-responsibility/ (accessed 25-03-2024).
45. Susskind “Lionel Cohen Lecture (2023). Available at: https://youtu.be/uAcNvdgodvA?si=xOrJTqOmfnQiKNLU (accessed 25-03-2024).
48. Lathrop, Curl & Fan “Law Firms Leveraging AI: Maximizing Benefits and Addressing Challenges” 2023 JOLT. Available at: https://jolt.law.harvard.edu/digest/law-firms-leveraging-ai-maximizing-benefits-and-addressing-challen ges; Bloomberg Law “What Are the Risks of AI in Law Firms?” (2024). Available at: https://pro.bloomberglaw.com/insights/technology/what-are-the-risks-of-ai-in-law-firms/#how-ai-is-changing-the-law-firm-business-model (accessed 29-06-2024).
