Can a Computer Program Be an Expert Witness?

Written on Thursday, January 12th, 2017 by T.C. Kelly
Filed under: Research & Trends

When DNA testing is too complex for police crime labs, the labs sometimes outsource their testing to a company like Cybergenetics. Most crime labs can handle DNA testing when the DNA to be analyzed comes from one person, but the process is much more complex when DNA from multiple sources has been mixed together.

Cybergenetics has developed a “probabilistic genotyping” software program called TrueAllele. The software uses algorithms that make judgments about how to separate DNA samples in order to create an individual DNA profile for each contributing source.

Computer Programs as Expert Witnesses

According to Cybergenetics, the TrueAllele program is superior to human analysts. Human experts, however, have the capacity to admit that they might be mistaken, particularly when an apparent flaw in their judgment is pointed out in court. The inability to convince a program that it reached an incorrect result is one reason why human experts might be preferable to computer software.

The use of a computer program to provide an expert opinion has sparked a controversy in the criminal justice system. While lab analysts can be cross-examined, it isn’t possible to cross-examine a computer program. In lieu of cross-examining TrueAllele, defense attorneys would like to have their own experts examine the program’s source code to determine whether the program might be flawed. Asserting that the source code is proprietary information, Cybergenetics refuses to release it, making it impossible to determine whether the program actually functions as intended.

The developers of TrueAllele claim that the program removes human bias from the testing process. But perhaps that claim reflects the bias of Cybergenetics, which makes money when outside agencies rely upon the program. Is TrueAllele really more accurate than human experts or is that a marketing claim?

Software Marketing

Cybergenetics offers to conduct a free preliminary analysis for police agencies. If the results show a probable DNA match, the agency can then decide to pay for a full analysis, complete with a report that can be offered as evidence at trial. That model provides an incentive to report a match — without one, Cybergenetics doesn’t get paid.

Cybergenetics’ primary competitor is STRmix, a program developed by a New Zealand-Australia collaboration and marketed by Nichevision in the United States. The FBI uses STRmix. David Balding, a professor of statistical genetics at the University of Melbourne, cites a study in which TrueAllele and STRmix were both used to examine a lab-created sample and returned “significantly different results.” That study suggests that one or both of the programs might be seriously flawed, but which one?

While both Cybergenetics and Nichevision claim that their programs are reliable, the studies that purport to demonstrate their reliability were conducted by the programmers, not by independent testing authorities. The recent White House PCAST report assessing the reliability of expert evidence noted that independent research is needed to verify those claims.

Cybergenetics argues that its internal test results should be sufficient, but Cybergenetics has an economic incentive to produce validation results that will help it sell its services. In the absence of independent testing, why should a court assume that probabilistic genotyping software is reliable?

Admissibility in Court

Courts that have considered whether to admit DNA testing results generated by computer programs have arrived at mixed results. A New York judge disallowed STRmix results in a high-profile case involving the strangulation death of 12-year-old Garrett J. Phillips. An earlier TrueAllele test in that case returned an inconclusive result. The judge decided that the prosecution failed to demonstrate the reliability of the STRmix program.

Other courts have decided to admit the evidence. For example, police officers in Onondaga County, New York stopped a car that was being operated without headlights. The car’s occupants fled into a park and shots were fired. Officers found a gun in the park, but they did not apprehend, and could not identify, any of the car’s occupants.

Police investigators determined that Frank Thomas owned the car. They had no other evidence that tied him to the gun. The local crime lab found the DNA of four to six people on the gun. Because the DNA was mixed, separating it and matching it to a particular suspect was beyond the crime lab’s ability.

The lab sent the samples to Cybergenetics. An analysis by the TrueAllele program computed a very high probability that DNA on the gun matched that of Thomas (or someone related to him). Believing he should not be required to take Cybergenetics’ word for the reliability of its program, Thomas’ lawyer asked for the program’s source code so that an expert could determine whether the program actually works. Cybergenetics refused to provide it and Thomas was convicted of reckless endangerment for firing the handgun. His case is on appeal.

Challenging Computer Software as an Expert Witness

A coding error in the STRmix program affected test results in at least 60 criminal prosecutions in Australia. Whether other errors exist in the source code can’t be determined without analyzing it, but the companies that developed TrueAllele and STRmix refuse to provide their source codes to defense attorneys.

Defense attorneys maintain that convicting defendants on the basis of “secret evidence” is inconsistent with constitutional values that emphasize the importance of cross-examination and disclosure of the basis for expert opinions. Both are fundamental to a fair trial, but it isn’t possible to cross-examine a computer program, and exposing flaws in the program isn’t possible without access to the source code.

Defense attorneys faced with DNA evidence that was produced by software rather than human experts can use their own expert witnesses to point out the inherent uncertainty of test results that cannot be validated by independent, unbiased scientists. Defense attorneys can also challenge the admissibility of evidence that depends upon an unproven methodology and that frustrates the constitutional right to confront a witness. Finally, defense attorneys can continue to demand to have the program’s source code analyzed by their own forensic computer experts.

Until appellate courts begin to recognize and address the problems caused by using software as an expert witness, the admission of probabilistic genotyping test results in a criminal trial will continue to raise serious concerns. As the Electronic Privacy Information Center contends, “Secrecy of the algorithms used to determine guilt or innocence undermines faith in the criminal justice system.”

About T.C. Kelly

Prior to his retirement, T.C. Kelly handled litigation and appeals in state and federal courts across the Midwest. He focused his practice on criminal defense, personal injury, and employment law. He now writes about legal issues for a variety of publications.

About T.C. Kelly

Prior to his retirement, T.C. Kelly handled litigation and appeals in state and federal courts across the Midwest. He focused his practice on criminal defense, personal injury, and employment law. He now writes about legal issues for a variety of publications.