AI Hallucination in Legal Practice: When Technology Gets the Law Wrong

AI hallucinations in legal practice can produce fabricated case law, and inaccurate legal information. AI hallucinations present problems for lawyers and their clients. AI hallucinations can negatively impact a client’s case. Relying on AI without factual verification violates an attorney’s duty of competence and candor to the court.

Without a human-in-the-loop, AI hallucinations can result in court sanctions, harm an attorney’s professional reputation, and expose them to malpractice claims. For more information about AI hallucinations, contact an experienced science and technology lawyer.

What Is AI Hallucination?

AI hallucination refers to the problem of artificial intelligence making up non-existent case law and false conclusions to AI prompts. AI hallucinations in legal practice can involve false statements of law, incorrect legal holdings, and case law for cases that do not exist.

The issue of AI hallucinations was in the headlines when a lawyer submitted court filings with legal citations to fake cases that never existed. The lawyers involved used generative AI to write a legal brief rather than rely on traditional legal research.

In legal pleadings and other court documents, attorneys often use case citations to binding precedent to support their legal arguments. In this case, the district court judge was unable to find the cited cases.

Find top Science and Technology Law lawyers easily

Connect with a qualified attorney today.

Find a lawyer today

Reducing AI Hallucinations and False Information

Some legal service tools attempt to avoid legal hallucinations with more focused artificial intelligence programs. For example, LexisNexis has Lexis+ AI and Thomson Reuters offers Westlaw AI-Assisted Research. Instead of basic large language AI models, these use retrieval-augmented generation (RAG).

RAG combines AI with focused datasets, to reduce potential hallucinations. RAG queries focus on specific source information before generating AI results. Focused AI within specified datasets can reduce AI hallucinations but it does not eliminate other forms of AI misinformation.

A Stanford Human-Centered Artificial Intelligence study found RAG-focused AI still produced incorrect information (from 17% to 34% of the time). Errors include confirming false premises, incorrect legal holdings, citing proposed legislation as law, and relying on overturned or outdated precedents.

The Dangers of Fabricated Case Law

Chief Justice John Roberts addressed AI tools and fabricated court cases in the 2023 annual report on the federal judiciary. According to Justice Roberts, generative AI tools have great potential but any use of AI and large language models needs human oversight. As always, the law is slow to respond to technological advancements.

According to Justice Roberts, “machines cannot fully replace key actors in court.” Human input can evaluate sincerity, physical cues, and balanced unquantifiable factors. Many court rulings rely on fact-specific gray areas that computer algorithms cannot balance.

Lawyers already rely on case law citation tools to make sure cited cases are still good law in the applicable jurisdiction. These tools include Shepardizing in Lexis and KeyCites in Westlaw. Any citations of case law and statutes with AI tools require double-checking to make sure the cases are real, still good law, and applicable for the jurisdiction.

A Lawyer’s Duty of Competence and Factual Verification

Legal professionals can benefit from chatgpt and AI use. However, any AI-assisted legal work requires full review and factual verification by an experienced attorney. If AI-generated content cites case law, court rules, or statutes, an attorney must check and verify those sources.

Lawyers have legal and ethical duties to their clients and to the administration of justice. Lawyers have a duty to their clients to act in their best interests and to provide competent, honest, and diligent legal services. Relying on AI systems without verification fails to uphold the model rules of professional responsibility.

Potential Consequences: Court Sanctions and Malpractice Claims

Attorneys can face disciplinary action for relying on AI programs in bad faith. Potential consequences can include fines, legal license suspension, or disbarment. Attorneys can also face legal consequences from their clients. Litigants could file a professional malpractice claim against their attorney for improper reliance on an AI program resulting in harm.

Duty To Supervise AI and Candor to the Court

The American Bar Association (ABA) has issued ethics guidance on the use of AI models in the practice of law. The Model Rules of Professional Conduct 1.1 requires lawyers to provide competent representation. Attorneys should understand the risks and benefits associated with legal AI tools in delivering legal services to clients.

Attorneys should also consider their duty of client confidentiality in using AI programs. The fine print of many AI tools warn collected data is used for model training and marketing. AI programs generally do not differentiate between sensitive and non-confidential information.

Even using AI systems to help draft a client email may violate the duty of client confidentiality. Using AI can compromise attorney-client legal privileges of confidentiality.

Legal professionals also have a duty to the courtroom and the justice system. Attorneys must obey the law and promote the proper administration of justice. This includes taking responsibility for any legal documents submitted to state or federal judges.

How Lawyers Avoid AI Hallucinations in Legal Practice

AI tools can save time for lawyers with many common legal tasks. Saving time can also save the client money, providing just, speedy, and inexpensive resolution to legal disputes. However, attorneys have an ethical and legal duty to provide competent legal service. Any use of AI requires a “human in the loop” to check and verify the facts, case law, precedent, and conclusions.

If you have concerns about an attorney using AI and how it can impact your case, talk to your lawyer about your concerns. A professional attorney has a duty of candor and honesty to be upfront about if and how they use artificial intelligence in their legal services. Contact your science and technology attorney to understand how to avoid AI hallucinations in legal practice.