Fans of science fiction are familiar with the role that Artificial Intelligence (AI) might eventually play in our daily lives. Even in the present, machines are being programmed to simulate intelligence by learning, adapting, and making reasoned decisions that respond to new situations.
Self-driving cars create the illusion of intelligence because they use algorithms to make decisions based on constantly changing data. Doctors depend on AI to help them diagnose health conditions, a task that machines may eventually be able to perform more accurately than physicians and without their assistance. Electronic tutors help children learn by assessing whether a student is bored or struggling and by changing lessons to meet the student’s needs.
Some have suggested that lawyers and judges will eventually be replaced by AI counterparts. Legal reasoning might even by improved by removing bias and political leanings from the decision-making process.
Jurors cannot be replaced by machines without amending federal and state constitutional guarantees of jury trials. Whether it would be wise to remove human emotion from the process of rendering a verdict probably depends on how lawyers feel about humans.
Expert Witnesses and AI
Matthew Robert Bennett and Marcin Budka have been “investigating the potential for AI to study evidence in forensic science.” Their results have been mixed. Focusing on the ability of AI to analyze footprint evidence, they found that the AI “was better at assessing footprints than general forensic scientists, but not better than specific footprint experts.”
Footwear imprints are commonly found at crime scenes. The FBI contends that “footwear impression evidence often provides an important link between the suspect and the crime scene.”
Forensic investigators gather shoeprint or footprint evidence at crime scenes. Forensic experts then offer opinions about their source. Scientists warn that prints found at crime scenes are often of low quality and that methods used to take impressions of the prints may change their characteristics. “For this reason, there will always be some uncertainty concerning whether a suspect’s shoe truly matches the crime scene print, or if the match is simply a false positive.”
Training an AI to Be an Expert
Footprint experts may be able to determine a suspect’s height, weight, and gender from the size of his or her footprints. However, Bennet and Budke determined that podiatrists have a 50{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4} success rate when they determine gender based on footprints — the same success rate that guessing would produce.
Bennet and Budke trained an AI to perform the same task. The AI arrived at a correct conclusion about gender 90{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4} of the time.
Shoeprint experts rely on experience and databases to identify the make of a suspect’s shoe. Bennet and Budke concluded that true experts are rarely mistaken in their identification. Unfortunately, there are few true footwear experts in the UK, where Bennet and Budke conducted their study.
Forensic investigators and police detectives make use of the UK’s extensive database of footwear and shoeprints. They are much less successful than experts at identifying footwear from shoeprints. Bennet and Budke tried to determine whether an AI could do a better job.
They trained a second AI to identify the make and model of footwear from black and white photographs of shoeprint impressions. They ran several trials with casual database users and discovered that their success rate varied from 22{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4} to 83{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4}. When they ran the same trials with their AI, the success rate varied from 60{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4} to 91{d61575bddc780c1d4ab39ab904bf25755f3b8d1434703a303cf443ba00f43fa4}.
The AI was better than database users who were not among the UK’s elite footwear experts, but still had a significant error rate. Actual footwear experts performing the same trial were nearly always right. The lesson learned is that AI isn’t yet ready to supplant human experts, at least when it comes to footwear identification — and that detectives who fancy themselves to be experts are not remotely qualified to render accurate opinions.
AI in Court
Perhaps R2D2 will one day be allowed to testify as an expert witness. At present, courts don’t need to grapple with the ability of an AI to swear an oath. As filtered through human witnesses, however, evidence created by an AI can influence the outcome of trials.
ExpertPages recently discussed the perils of relying on ShotSpotter, a form of AI that uses algorithms to deduce the location of loud noises that it identifies (not always correctly) as gunshots. Even if the accuracy of ShotSpotter were a given, the ability of humans to tweak the results that algorithms produce raises questions about the reliability of ShotSpotter. Investigations by journalists have “identified a number of serious flaws in using ShotSpotter as evidentiary support for prosecutors.”
Apart from the potential unreliability of human witnesses who present the conclusions drawn by AI, the accuracy of an AI’s conclusions depends on the reliability of the algorithm that teaches the AI to “think.” The creators of AI generally claim a proprietary interest in the algorithms, refusing to open them for inspection by an opposing party. That makes “trust us” the common fallback position of human witnesses who explain how an AI reached its conclusions.
In cases that depend on evidence generated by an AI, opposing parties may need to hire an AI expert. The expert will likely have a background in computer science or information technology but might need to work with a second expert who has specialized knowledge of biomechanics, e-commerce, or a variety of other disciplines, such as footwear recognition or acoustics. While the goal of AI is to improve the human condition by allowing computers to arrive at faster and more accurate conclusions than humans, the present reality is that human experts are often needed to testify about the flaws in AI decision-making.
