Avoiding Algorithmic Bias: Top 5 AI Liability Issues in Courts

Algorithmic bias refers to discriminatory outcomes in artificial intelligence when using flawed data. Using judicial records, court case law, and law enforcement statistics for AI risk assessment can pose due process and equal protection issues because of reliance on biased data.

AI developers are protective of their algorithms, which causes problems with transparency and accountability for identifying and reducing AI bias. For more information about AI bias in the courts, talk to a science and technology attorney for legal advice.

What Is Algorithmic Bias?

Algorithmic bias refers to errors and inequalities resulting from using biased data to produce AI decision outcomes that reflect those human biases. There is a phrase in computer science for using bad information in data analysis — garbage in, garbage out. Using biased training data to generate AI-based results can yield biased outcomes.

Artificial intelligence (AI) is continuing to expand its reach into legal practice and the judiciary. Large language models use a variety of data sets to analyze and generate text and query outcomes. These algorithms review the data, identify patterns, and use these patterns to make predictive decisions.

In legal practice, AI technologies use data, including case law, legal scholarship, legislation, public records, judicial statistics, and other sources of information. The majority of the data analyzed by AI systems is produced by humans. That information contains decisions made by humans, which can include their direct and implicit biases. The AI outcomes reflect the same biases that went into the original data.

Find top Science and Technology Law lawyers easily

Connect with a qualified attorney today.

Find a lawyer today

How Flawed Data Leads to Discriminatory Outcomes

Using historic legal information to provide current outcomes can carry over potential bias from the past. The legal system has a history of bias and disparities among different groups, including racial bias, gender bias, and economic bias. Using biased data with a history of inequality can continue to produce biased outcomes.

For example, people of color are more likely to face arrest, have higher bail, receive more serious criminal charges, and receive harsher criminal sentencing than white defendants. The criminal justice system continues to work on identifying human bias in law enforcement and the courts. However, this history of inequality goes back to the founding of the country.

Liability Issue 1: Due Process and Equal Protection

The U.S. Constitution guarantees the right to due process of law under the Fifth Amendment. No person shall be deprived of life, liberty, or property without due process. This includes substantive due process and procedural due process. Due process gives individuals a right to fair laws and a fair legal procedure.

Relying on AI applications in applying the law can threaten due process for individuals. Due process requires fair laws and a legal process. Machine learning based on unfair legal precedents can violate an individual’s due process protections.

The Fourteenth Amendment to the Constitution also guarantees equal protection. The Equal Protection Clause has been a powerful force in establishing anti-discrimination protections and civil rights laws. Equal protection makes sure there is equal treatment for all people under the law, without regard to race, gender, or socioeconomic status. The use of AI presents challenges to due process and equal protection.

For example, there are various private sector risk assessment tools marketed to law enforcement agencies. These software systems use algorithms to evaluate recidivism risk factors to make decisions in sentencing, bail, and parole. A 2016 study by ProPublica showed these systems disproportionately classified black defendants as high risk compared to white defendants.

Liability Issue 2: Accountability of AI Developers

AI algorithmic bias can have another cause beyond using biased data. Improper design of algorithmic systems can also introduce bias. Many AI systems develop their algorithms based on inaccurate data sets to identify patterns. It is up to developers to prevent compounding bias by solely focusing on patterns identified in flawed data.

AI developers can try to avoid accountability by claiming their systems treat all data equally. However, even if AI systems do not have any discriminatory intent built into the algorithmic systems, they can still have a disparate impact on certain groups.

Disparate impact and disparate treatment are common issues in discrimination law. Under anti-discrimination laws, an employer is liable for unequal treatment based on disparate impact, even if they never intended discrimination. Disparate impact shifts the burden to the other party to demonstrate necessity. The same issues could apply to AI developers in bias liability cases.

Liability Issue 3: Lack of Transparency

You must understand what goes into AI actions to ensure it is fair and reliable. AI systems must be clear about data collection, data use, and any measures to prevent or reduce inherent bias. However, many AI software tools are not transparent about their data sets, data privacy rules, and algorithmic decision-making.

This is referred to as the black box problem. With AI systems, you can see what goes in and what comes out. However, you have no idea what happens in between. Lack of transparency prevents others from understanding what goes into the algorithm, identifying errors, and ensuring equal treatment.

Companies offering automated decision-making systems claim their systems are proprietary. Without understanding what goes into the decision-making process for these software systems, it is difficult to evaluate their AI bias.

AI models are complex and challenging to understand, even when they are open and transparent. Focused testing can help identify real-world problems with predictive processing based on biased data. Regular assessments and auditing can identify and mitigate bias.

Liability Issue 4: Proving Causation

Even after demonstrating AI bias, proving it was the cause of a given outcome is a separate issue. Law enforcement, healthcare providers, and employers can all use AI algorithms in decision-making systems. However, plaintiffs wronged in the decision-making process must show the use of algorithms was a but-for cause for their damages.

For example, an employer uses AI to review employment applications and recommend potential interviewees. The AI tool uses an algorithm with a potential for discrimination bias based on gender. However, the applicant did not meet the minimum requirements for the job. In this case, it would be difficult to prove that the biased AI tool was the cause of not getting an interview.

Liability Issue 5: Inadequate Risk Assessment

A common issue with using new technology is that many of the risks are unknown. It takes time to evaluate and test these new adaptive technologies to identify their strengths and weaknesses. Internal risk assessments by developers often fail to take into account real-world problems. Even relying on expert evaluations fails to assess how the common user will use the information technology.

Human oversight is a vital part of using AI in legal functions. Laws are complex, vary by jurisdiction, and often use several factors and balancing tests that can defy computer-based predictive functions. Human lawyers also have an ethical and legal duty to their clients to act in their clients’ best interests.

How To Mitigate Algorithmic Bias and Legal Risk

AI regulations are slow to respond to developing technologies like AI algorithms in the law. Policymakers often struggle to understand the technology, processes, uses, scope, risks, and how to limit science and technology law. Most technological regulations in recent years have been adapted from prior technologies, like using wire and telecommunications laws to apply to internet regulations.

The most important tool to mitigate algorithmic bias and legal risk is human oversight. Legal issues often involve issues of serious concern, from potential jail time to child custody disputes. Relying on software algorithms to make predictive decisions can be helpful but they are not infallible. Courts and judges make decisions based on the individual situations. Algorithms based on historic patterns cannot take into account every individual factor.

AI systems can be very helpful in many legal contexts. However, there are also risks of relying on potentially biased algorithms. To understand how to mitigate your legal risks when using AI algorithms, contact a local science and technology attorney for legal advice.