United States v. Williams

William H. Orrick, United States District Judge

INTRODUCTION

Starting May 6, 2019, defendant Reginald Elmore will stand trial for the August 14, 2008 double murder of Isiah Turner and Andre Helton, among other crimes. This motion concerns expert testimony about the DNA sample collected from the right rear door handle of the rental car in which the two men were shot. Elmore moves to exclude testimony by Phillip Hopper from the Serological Research Institute ("SERI"), who used a probabilistic genotyping program called Bullet to inform his conclusion that there is "very strong support" for the proposition that Elmore contributed DNA to the sample. Elmore's challenge does not require me to decide whether probabilistic genotyping in general or Bullet in particular are reliable and accepted methods of DNA analysis. Instead, the question before me is whether Bullet was validated to analyze the mixture at issue here. Because Bullet was only validated to analyze complex mixtures of up to four contributors, and because Hopper did not reliably conclude that only four people contributed DNA to this mixture, this evidence is not reliable. Accordingly, the motion is GRANTED.

BACKGROUND

I. RELEVANT PROCEDURAL BACKGROUND

The grand jury returned an indictment against Elmore on January 9, 2014. Second Superseding Indictment [Dkt. No. 31]. Elmore was originally set for trial with the first group of defendants beginning in November 2017, but I granted his motion to sever on October 16, 2017. Minute Entry [Dkt. No. 1409]. The trial for the second group of defendants was scheduled to begin on July 16, 2018, but I had to continue it because of an interlocutory appeal that remained pending in the Ninth Circuit.¹ Order Re: Trial on July 16, 2018 [Dkt. No. 1807].

In 2011, the government asked SERI to test swabs from thirteen areas of the rental car where Turner and Helton were murdered. SERI executive director and chief forensic serologist Gary Harmor produced a report in February 2012. Mot. Ex. 1 ("2012 Report") [Dkt. No. 2008-1]. In April *9302018, the government asked SERI to retest the evidence. SERI forensic serologist and DNA technical leader Phillip Hopper conducted a second analysis and produced a report in December 2018. See Mot. Ex. 2 ("2018 Report") [Dkt. No. 2008-1]. On March 21, 2019, Elmore filed a motion to exclude Hopper's testimony. Motion to Exclude ("Mot.") [Dkt. No. 2008]. An evidentiary hearing took place on April 24, 2019, during which I heard testimony from Hopper, Harmor, and Dr. Dan Krane, an expert for Elmore.² I heard argument the following day.

II. DNA ANALYSIS

A. Analysis of Complex Mixtures

DNA analysis for single-source and simple mixtures-those with DNA from just one or two individuals-is objective and reproducible in part because it requires the exercise of little if any human judgment. See Mot. 6 (citing Katherine Kwong, The Algorithm Says You Did It: The Use of Black Box Algorithms to Analyze Complex DNA Evidence , 31 Harv. J.L. & Tech. 275, 277 (2017) ) (citations omitted) (hereinafter "Kwong").³ By contrast, human judgment is required to analyze complex mixtures with three or more DNA profiles because "all of the individual DNA profiles [are] superimposed atop one another." Id. at 278. An analyst must decide between "different interpretations that might be equally or similarly valid - and those decisions may have significant impacts on the ultimate results of the analysis." Id.

"It is frequently impossible to tell how many individuals' DNA is present within a complex mixture"; a greater number of contributors only increases the rate of error, which usually comes in the form of an underestimate. See id. ; Declaration of Dan Krane ("Krane Decl.") [Dkt. No. 2008-5], 15. For example, a 2005 study found that analysts mischaracterized known four-person mixtures as three-person mixtures at a rate of 70%. Id. These errors likely occur because of allele⁴ sharing:

Some alleles at some loci are relatively common and therefore likely to overlap between contributors to a mixture. Thus, the more individuals present in a mixture, the more likely it is the mixture will hide identifications of subsequent individuals, as the relative proportion of present versus absent alleles at each locus increases with each new contributor.

Id. As Krane testified at the hearing, a five-person sample can present very similarly to the way four-person mixtures do.

Advancements in amplification technology have improved analysts' ability to accurately determine the number of contributors because they amplify the alleles at more loci. See Krane Decl. ¶¶ 13, 14. For example, SERI previously relied on the Identifiler Plus kit, which amplifies the alleles at 15 loci. The newer GlobalFiler kit, which SERI validated in December 2016, amplifies the alleles present at 21 loci,⁵ and some of the additional loci are *931polymorphic. Hopper testified that polymorphic loci are helpful to the analysis because they are more discriminating, meaning there are many possible alleles at that locus and thus two people are less likely to have the same genotype.

GlobalFiler has improved the reliability of the conclusions regarding the number of contributors for known three-person mixtures. See Krane Decl. ¶¶ 13, 14 (citing a study finding that no lab underestimated the number of contributors to known three-person mixtures); Hopper Decl. ¶ 6. But known five-person mixtures were mischaracterized as originating from four or fewer individuals in approximately 61-75% of samples.⁶ Krane Decl. ¶ 13 (citation omitted). When SERI validated GlobalFiler, it tested two-, three-, four-, and five-person mixtures. Hopper Decl. ¶ 8. It experienced the same difficulties. In fact, it underestimated all of the known five-person mixtures tested:

In each five-person mixture tested, the electropherograms showed no indication of more than four contributors. This was not due to a shortcoming of GlobalFiler or the testing process, but rather because, by coincidence, the contributors used to create the test mixture shared alleles. Given the genotypes of the contributors, no more than eight alleles could appear at any one locus.

Id. Hopper explained in his testimony that SERI often uses DNA profiles of employees and friends during validation studies. A 2018 study found that analysts underestimated 64% of known five-person mixtures and 100% of known six-person mixtures-and characterized all of the mixtures as containing DNA from four individuals.⁷ Krane Decl. ¶ 14.

Even with the improvement in amplification technology, other factors present challenges to accurately identifying the number of contributors. The challenge of allele sharing is "frequently exacerbated by samples that have degraded or which originally contained only a small amount of DNA." Kwong at 278. As Hopper testified at the hearing, degradation occurs when DNA breaks off between the bases, which usually happens to larger pieces first. This process occurs naturally over time, although freezing DNA can slow it down. Amplification kits are unable to copy DNA past the point where the breakage has occurred.

B. Probabilistic Genotyping and Bullet

Probabilistic genotyping is a statistical technique used to analyze DNA mixtures. In 2018, SERI validated the probabilistic genotyping program Bullet for use with two-, three-, and four-person mixtures. Hopper Decl. ¶ 14. SERI did not seek to validate Bullet for use with five-person mixtures. According to Hopper's testimony, one key advancement of Bullet is its ability to consider the possibility that some alleles are not detectable because they have dropped out of the mixture. Due to degradation or simply low quantities of DNA, not all of a contributor's alleles will necessarily be detectable at every locus. Old technologies excluded individuals as contributors to a sample if just one of that individual's alleles was missing from the sample. By considering dropout, Bullet can account for the likelihood that a person *932might have contributed even if some alleles are absent.

Bullet calculates a dropout rate based on the serologist's conclusion about how many individuals contributed DNA to the mixture being analyzed. By default, Bullet takes the sum of all the detected peaks and divides it by the number of contributors that the serologist has input, resulting in a probability that dropout has occurred at each individual locus. The serologist can also set a different dropout rate. Very high peaks, which indicate little or no degradation and few if any minor contributors, might suggest that the probability of dropout is zero. With a probability of zero, Bullet will not consider any genotypes that include alleles not detected in the mixture. By contrast, a probability of one will allow for the possibility that any number of alleles might have dropped out and make it effectively impossible to exclude any genotype as a possible contributor.

Taking this information into account, Bullet then analyzes the genotypes that are possible at a given locus. The program assigns a probability that a given genotype is present compared to others and ultimately produces a likelihood ratio. The likelihood ratio is a ratio of two probabilities: (1) the probability of the DNA evidence originating from the suspect and unknown persons, and (2) the probability of the DNA evidence originating entirely from unknown persons other than the suspect. Hopper Decl. ¶ 11. Bullet calculates a likelihood ratio at each locus tested and then multiplies them together in order to arrive at an overall likelihood ratio. Id. SERI developed an internal scale to assign a word statement to explain the likelihood ratios Bullet produces. Id. ¶ 33. Unlike other probabilistic genotyping programs, Bullet consistently produces the same likelihood ratio if given the same inputs. Id. ¶ 12.

C. 2012 Analysis in this Case

During the first round of testing the swabs from the car, Harmor used the Identifiler Plus amplification kit, which amplifies 15 loci. Declaration of Gary Harmor ("Harmor Decl.") [Dkt. No. 2031-2] ¶ 3. After analyzing the DNA from all thirteen areas of the car, Harmor excluded Elmore as a potential contributor for ten. 2012 Report 8-10. He was unable to perform a comparison for one swab because of insufficient DNA. Id. at 8. He concluded Elmore could be a minor contributor to another area, but so could one person in two. Id. at 9. For the right door handle, Harmor wrote that the swabs included "a mixture of DNA from at least five individuals." Id. at 8. He concluded that Turner was the major donor. Id. Elmore could possibly be a minor donor, but "approximately four in seven persons could also be a minor contributor considering Reginald Elmore's genetic profile." Id. An internal reviewer at SERI signed off on Harmor's analysis.

D. 2018 Analysis in this Case

In April 2018, the government asked SERI to perform additional DNA testing on some areas of the car, including the right rear⁸ door handle. Opposition ("Oppo.") [Dkt. No. 2031] 2. Hopper used the recently validated Bullet to conduct the second analysis. See 2018 Report. Hopper was aware that a colleague had already analyzed the same items, but he performed his own analysis and did not know that Harmor had concluded that five people contributed to the mixture. Hopper Decl. [Dkt. No. 2031-1] ¶¶ 16, 26. One mixture was weak and incomplete and thus *933unsuitable for comparison, and Hopper excluded Elmore as a possible contributor to others. 2018 Report 8-9. The following sections refer to his testing of the right door handle.

1. Amplification and Number of Contributors

First, Hopper combined the extracted DNA that remained from Harmor's analysis with the DNA that was extracted from the swabs the government sent with its retesting request.⁹ In total, he had 0.65 nanograms of DNA to test; SERI recommends that 1 nanogram of DNA be used with GlobalFiler whenever possible. Next, he amplified the alleles using the GlobalFiler kit. Hopper Decl. ¶¶ 6, 18. The amplified DNA was then loaded onto a genetic analyzer, which displayed each fragment on a graph. Hopper testified that the new graph showed fewer alleles than the 2012 one had, indicating that degradation had taken place in the more than six years since the prior testing.

Hopper then determined the number of DNA contributors, starting by counting the number of peaks on the electropherogram, each of which represents an allele.¹⁰ See Hopper Decl. ¶ 19. There were seven alleles at two loci: D19S433 ("D19") and D22S1045 ("D22"). Gov't Ex. 3; see Hopper Decl. ¶ 19. Because nearly all human beings possess no more than two alleles at a given locus, and because seven was the highest allele count, Hopper concluded that the sample was a mixture of "at least four contributors." Id. ¶ 24. Hopper used the term "at least" because "outside of laboratory controlled conditions, it is impossible to know with certainty the number of contributors to a mixture." Id. The actual number of contributors can be larger than the apparent number because of degradation, allele sharing, and the presence of homozygote contributors. See id.

Hopper ruled out certain signs that might have indicated more than four contributors, starting with peak height imbalance. See id. ¶¶ 20-23. The peak heights of alleles from the same contributor are expected to be within about 60 percent of one another. Hopper Decl. ¶ 20. As a result, when alleles at the same locus have materially different heights, those alleles likely come from different contributors, suggesting more contributors than the number of alleles would otherwise indicate. See id. Hopper did not see peak height imbalance that would cause him to alter his conclusion. At D19, there were two alleles with taller peaks. See Gov't Ex. 3. Because the heights of those peaks were within 60 percent of each other, Hopper concluded that likely came from the same contributor,¹¹ although those allele heights could have resulted from allele sharing or the presence of homozygotes.¹² Hopper concluded that any of the remaining peaks could pair together in a genotype because they were all fairly short.

In analyzing the number of contributors, Hopper also considered stutter peaks and peaks that were below the detection threshold. See Hopper Decl. ¶ 21. Below-threshold peaks can be caused by an allele that is present only in a small quantity or an artificial peak created during the amplification process.¹³ He disregards *934peaks that are below the analytical threshold "even though it is possible that those disregarded peaks represent alleles from minor contributors." Id. There was one below-threshold peak at D22, and considering that peak would have raised the total number of alleles to eight. Id. ; Gov't Ex. 3. But because a four-person mixture can result in a locus with eight alleles, this below-threshold peak did not impact Hopper's estimate of contributors. See Hopper Decl. ¶ 21. At the hearing, I asked Hopper about a red line appearing to the left of the first peak at D22. He indicated that it could be relevant to his analysis, but he was not able to say what it was without being able to magnify it on the computer.¹⁴

Regarding the two persons of interest-Turner and Elmore-Hopper did not see evidence of allele sharing that would cause him to conclude that there were more than four contributors to the mixture (assuming that both Turner and Elmore contributed). See Hopper Decl. ¶ 23. Turner and Elmore shared no alleles at the two loci where seven alleles were present, and "all four of their alleles [were] present" at each of those loci. Id. Regarding any remaining unknown contributors, Hopper could not rule out the possibility of allele sharing.

For all of these reasons, Hopper decided to move forward with his analysis based on his conclusion that there were four contributors to the mixture. He reiterated throughout his testimony that while there could have been more contributors, he decided on the number four because he saw no compelling reason to conclude that it was higher.

2. The Likelihood Ratio

Hopper then used Bullet to calculate a likelihood ratio. Hopper Decl. ¶ 27. He made several inputs before Bullet conducted the analysis. First, he input his conclusion that the mixture contained four contributors. Second, he assumed that Turner was a contributor to the mixture based on his conclusion that "the mixture is at least 18 billion times more likely" if it originated from Turner and three unknown individuals than if it originated from four unknown individuals.¹⁵ Id. ¶ 30. Third, he invoked Bullet's ability to account for dropout at the six loci where Elmore's alleles were absent.

As to Elmore, Hopper concluded as follows:

Assuming Isiah Turner is a contributor to the mixture, the DNA evidence is at least 270 thousand times more likely if it originated from Isiah Turner, Reginald Elmore (item 21A-1), and two unknown, unrelated individuals than if it originated from Isiah Turner and three unknown, unrelated individuals.¹⁶

Id. ¶ 29; 2018 Report 8. Based on SERI's verbal scale,¹⁷ this result indicated "very strong" support for the proposition that *935Elmore is a contributor to the sample. Hopper Decl. ¶¶ 28, 33.

After Elmore moved to exclude this testimony, for the purposes of comparison Hopper asked Bullet to calculate a likelihood ratio based on the assumptions that five individuals contributed to the mixture and one was Turner.¹⁸ Id. ¶ 31. The result-roughly 60,000-shows that "the likelihood ratio would not dramatically change if the mixture in question had five contributors instead of four." Id. Hopper acknowledged in his declaration and his testimony that because Bullet is not validated to perform that analysis, he would not rely on that statistic. See id.

Once Elmore challenged Hopper's conclusion, Harmor reviewed his own conclusion in 2012 that there were at least five contributors to the mixture. See Harmor Decl. ¶ 5. He now disavows it. He testified that he could not remember why he reached that conclusion, and he did not have any documentation that might indicate the basis for it. His "best guess is that [he] made a mistake" because "[t]he number of contributors was not relevant to the statistical analysis [he] was performing." Id. ¶ 7. He testified that his technical reviewer must have missed the error as well. He re-reviewed the electropherogram and saw nothing to suggest that the mixture has five contributors. Id. ¶ 5. He did not consult his original technical reviewer to determine how he reached his earlier conclusion.

LEGAL STANDARD

Federal Rule of Evidence 702 allows a qualified expert to testify "in the form of an opinion or otherwise" where:

(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;

(b) the testimony is based on sufficient facts or data;

(c) the testimony is the product of reliable principles and methods; and

(d) the expert has reliably applied the principles and methods to the facts of the case.

Fed. R. Evid. 702.

Expert testimony is admissible under Rule 702 if it is both relevant and reliable. See Daubert v. Merrell Dow Pharm., Inc. , 509 U.S. 579, 589, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). "[R]elevance means that the evidence will assist the trier of fact to understand or determine a fact in issue." Cooper v. Brown , 510 F.3d 870, 942 (9th Cir. 2007) ; see also Primiano v. Cook , 598 F.3d 558, 564 (9th Cir. 2010) ("The requirement that the opinion testimony assist the trier of fact goes primarily to relevance.") (internal quotation marks omitted).

Under the reliability requirement, the expert testimony must "ha[ve] a reliable basis in the knowledge and experience of the relevant discipline." Primiano , 598 F.3d at 565. To ensure reliability, the court must "assess the [expert's] reasoning or methodology, using as appropriate such criteria as testability, publication in peer reviewed literature, and general acceptance." Id. These factors are "helpful, not definitive," and a court has discretion to decide how to test reliability "based on the particular circumstances of the particular case." Id. (internal quotation marks and footnotes omitted). "When evaluating specialized or technical expert opinion testimony, the relevant reliability concerns may focus upon personal knowledge or *936experience." United States v. Sandoval-Mendoza , 472 F.3d 645, 655 (9th Cir. 2006).

The inquiry into the admissibility of expert testimony is "a flexible one" where "[s]haky but admissible evidence is to be attacked by cross examination, contrary evidence, and attention to the burden of proof, not exclusion." Primiano , 598 F.3d at 564. "When the methodology is sound, and the evidence relied upon sufficiently related to the case at hand, disputes about the degree of relevance or accuracy (above this minimum threshold) may go to the testimony's weight, but not its admissibility." i4i Ltd. P'ship v. Microsoft Corp. , 598 F.3d 831, 852 (Fed. Cir. 2010). The burden is on the proponent of the expert testimony to show, by a preponderance of the evidence, that the admissibility requirements are satisfied. Lust By & Through Lust v. Merrell Dow Pharm., Inc. , 89 F.3d 594, 598 (9th Cir. 1996) ; see also Fed. R. Evid. 702 Advisory Cttee. Notes.

DISCUSSION

The pending motion presents a narrow question: whether Bullet is validated to perform the analysis on the sample at issue. Elmore argues that I must exclude Hopper's opinions because Bullet is not validated to analyze samples like the one at issue here, which according to Elmore likely has more than four contributors.¹⁹ Mot. 14. The government counters that just because five-and six-person mixtures are often mischaracterized as four-person mixtures does not mean that this mixture was mischaracterized. Oppo. 6. Having reviewed the papers, heard the experts' testimony, and considered the argument of counsel, I find that the government has not met its burden to show Bullet performed an analysis that it was validated to perform. For a number of reasons, Hopper did not reliably conclude that only four individuals contributed DNA to the mixture at issue.

First, the body of research in the record before me shows that five-person mixtures are underestimated as four-person mixtures at a troubling rate-a rate much higher than that associated with four-or three-person mixtures. These errors occur because "the more individuals present in a mixture, the more likely it is the mixture will hide indications of subsequent individuals, as the relative proportion of present versus absent alleles at each locus increases with each new contributor." Krane Decl. ¶ 13. In other words, as Krane testified, the difficulty in identifying the number of contributors to a complex mixture increases significantly more with each additional contributor. GlobalFiler has improved analysts' ability to accurately identify some complex mixtures by allowing them to evaluate the alleles at more loci. See Krane Decl. ¶ 13 (citing a study finding that if all alleles from every contributor are visible and there are no other complicating factors, known three-person mixtures are underestimated only in 0.2% of samples). But there remains a high rate of error, particularly for mixtures above four contributors. See id. ¶¶ 14-15. A post-GlobalFiler study revealed that 64% of known five-person mixtures and 100% of known six-person mixtures were underestimated.²⁰ Id. ¶ 14.

*937Most were misidentified as containing DNA from four contributors. Id.

Perhaps more importantly, SERI itself has demonstrated an inability to distinguish five-person mixtures from four-person mixtures. During the GlobalFiler validation study, it did not correctly identify a single five-person mixture. Hopper asserted that these errors occurred because "by coincidence," the individuals who contributed DNA to the study shared alleles. But a 100% rate of error²¹ gives no confidence in SERI's ability to be accurate if faced with the same coincidence in the real world. During his testimony, Hopper qualified his conclusion about the number of contributors many times over, honestly acknowledging the many reasons why the mixture might have more contributors than four. Indeed, SERI's executive director and chief forensic serologist concluded in 2012 that there were at least five contributors to the mixture. There is simply not enough evidence to conclude reliably that this mixture is a four-person mixture accurately identified by SERI instead of a five-person sample underestimated by SERI.

Three facts specific to the sample at issue here cast further doubt on the reliability of Hopper's conclusion. First, he had only 0.65 nanograms of DNA to test, which is 35% below the amount recommended for use with GlobalFiler. Second, more than six years elapsed between the first testing and the second, and Hopper testified that the sample showed signs of degradation. See 2018 Report 8 ("Due to the presence of weak typing results at some loci, it is possible that minor components of the mixture have dropped out in the larger loci."). Indeed, a side-by-side comparison of the fifteen loci amplified in 2012 and the same ones amplified in 2018 shows more alleles in the prior testing. Both the low quantity of DNA and the degradation of the sample impact the detectability of alleles, which in turn impacts the dropout rate, which in turn underlies all of Bullet's calculations. Finally, there are two loci with seven alleles-and one of those loci has a below-threshold peak that could represent an eighth allele. If that is the case, the sample can be a four-person mixture only if no two contributes share alleles at that locus, no contributor is a homozygote at that locus, and no additional alleles have dropped out at that locus.²²

The government argues that exclusion of the testimony is not appropriate; instead, Elmore can challenge Hopper's analysis and conclusions during cross-examination. But the number of contributors is a foundational part of every calculation Bullet performs.²³ If that input is in doubt, the reliability of the entire analysis is necessarily in doubt. To corroborate Hopper's conclusion about the number of contributors, the government put forth the results he obtained after running Bullet with a *938five-person mixture input. See Oppo. 6-7. But Bullet was not validated to test five-person mixtures, and I will not rely on that result for any purpose.

DNA evidence can have a powerful effect on a jury's evaluation of a criminal case. See John W. Strong, Language and Logic in Expert Testimony: Limiting Expert Testimony by Restrictions of Function, Reliability and Form , 71 Or. L. Rev. 349, 367 n.81 (1992) ("There is virtual unanimity among courts and commentators that evidence perceived by jurors to be 'scientific' in nature will have particularly persuasive effect.") (citing cases). If SERI could accurately identify five-person mixtures and if it had validated Bullet to analyze them, then it might have a reliable understanding of how underestimating a five-person mixture impacts the likelihood ratio. That understanding could improve the reliability of Hopper's conclusion on the number of contributors or make it appropriate to allow the government to present two likelihood ratios: one based on four contributors and a second based on five. Then the other problems identified in this Order, such as Harmor's changed testimony, the small testing sample, and the signs of degradation, would be ripe for cross-examination. But there are simply too many reasons to question the reliability of Hopper's conclusion on this foundational issue, which brings the entire analysis outside the parameters of Bullet's validation at SERI. This testimony is not reliable, and it is not admissible.

CONCLUSION

For the reasons set forth above, Elmore's motion to exclude is GRANTED .

IT IS SO ORDERED.

1 The appeal did not impact the case against Elmore, and he requested to be tried separately at the preexisting trial date. Given concerns about judicial efficiency and witness security, I denied his request. See Order re: Trial on July 16, 2018.

2 Citations to the hearing transcript do not accompany references to the testimony I heard because the final version of the transcript was not yet available at the time this order was drafted.

3 Elmore previously provided the entire article with a status report on his anticipated challenge to this evidence. Status Report Ex. 1 [Dkt. No. 1999-1].

4 An allele is a sequence of DNA inherited from a biological parent. With rare exceptions, people have two alleles at each locus. A person is a homozygote if both parents contributed the same allele to a particular locus and a heterozygote if each parent contributed a different allele at that locus. See Krane Decl. ¶ 11.

5 Hopper's declaration and the government's opposition initially indicated that GlobalFiler amplifies the alleles at 24 loci. Oppo. 2; Hopper Decl. ¶ 6. Hopper clarified in his testimony that the number 24 includes three loci that are not used in statistical calculations because they merely identify a person's sex. As relevant for Hopper's analysis, GlobalFiler amplifies the alleles at 21 loci.

6 The study did not take peak height into account. See Oppo. 5 n.2.

7 The government points out that this study included results obtained using GlobalFiler along with kits that amplify fewer loci.

8 The report refers to the area as simply the right door handle, but Hopper indicated in his testimony that he was referring to the right rear door handle.

9 Hopper testified that SERI stored the retained DNA frozen, but he was not aware how the government stored the swabs that SERI returned after the 2012 testing. The 2018 testing consumed all of the remaining samples.

10 There are exceptions, for example stutter peaks.

11 Hopper testified that D22's seven alleles did not have such a simple pairing.

12 When there is more than one of the same allele present at a particular locus, those alleles appear stacked together as one in the electropherogram.

13 Specifically, a stutter peak can appear to the left of a taller peak.

14 He also testified that while conducting the analysis, he was able to zoom in and consider the entire graph. Doing so led him to conclude that there were seven alleles with a possible eighth.

15 He acknowledged during his testimony that he knows of no agreed-upon threshold that a likelihood ratio must meet before making an assumption that an individual contributed to the mixture.

16 Hopper agreed to revise his phrasing from "... 270 thousand times more likely to have arisen from ..." to "... 270 thousand times more likely if it originated from ..." in response to Elmore's critique that prior conclusion was inaccurately stated. See Hopper Decl. ¶ 29; 2018 Report 8 (emphasis added).

17 In response to Elmore's criticism that SERI's verbal scale does not match the one developed by the Scientific Working Group on DNA Analysis Methods ("SWGDAM"), Hopper noted that SERI developed its scale before SWGDAM had published a recommendation. Hopper Decl. ¶ 33.

18 Bullet is only able to calculate a likelihood ratio for five-person mixtures if one or more contributor is known. Id. ¶ 31.

19 Elmore also asserts that the jury is likely to misunderstand the likelihood ratio in a way that will prejudice him. I do not reach this issue because I grant the motion on other grounds.

20 The government asserts that I should not rely on the 64% figure from this study because it included data from kits that do not amplify as many loci as GlobalFiler does. But the government has not pointed me to any evidence showing that the rate of underestimation is less concerning than these studies suggest.

21 The government asserts that we do not know a rate of error appropriate to ascribe to this sample because known four-person samples were accurately identified 100% of the time.

22 Elmore cross-examined Hopper on two profiles from the National Institute of Standards and Technology's database of 1,036 random DNA profiles while cross-examining Hopper. Viewing the five most significant loci according to Hopper, and invoking six instances of dropout just as Hopper did for Elmore, those two individuals (PT83879 and ZT79319) also could have contributed DNA to the mixture, raising the possible number of contributors to six.

23 For example, the number of contributors factors into the dropout rate Bullet uses for each locus. The government highlighted Bullet's ability to account for dropout as a key advance of the technology. As Elmore brought to light during Hopper's cross-examination, a very low dropout rate would exclude Elmore while a very high dropout rate would effectively make it impossible to rule anyone out.