Valador, Inc. v. HTC Corp. ( 2017 )

MEMORANDUM OPINION

T.S. Ellis, III, United States District Judge

At issue in this trademark infringement case is whether plaintiffs expert on the question of likelihood of confusion should be excluded pursuant to Rule 702, Fed. R. Evid., and Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). As the matter has been fully briefed, and because oral argument would not aid the decisional process,¹ this question is now ripe for resolution.

I.

This action arises from defendants’ alleged trademark infringement on plaintiffs registered mark, “VIVE,” which plaintiff claims to use in connection with the sale of its software. Plaintiff, Valador, Inc., is a small Virginia stock corporation headquartered in Herndon. The three defendants in this action are (1) HTC Corporation, a Taiwanese corporation, (2) HTC America, Inc., a Washington state corporation headquartered in Seattle, and (3) Valve Corporation, a Washington state corporation headquartered in Bellevue, Washington. Plaintiff alleges that each defendant has infringed on plaintiffs VIVE mark through defendants’ marketing, advertising, and sale of a headset, the “HTC Vive,” capable of running software that renders three-dimensional images. Yet, plaintiff does not label any of its end products or delivera-bles with its “VIVE” mark; rather, plaintiff inserts its “VIVE” mark in the text of its service contracts, the overwhelming majority of which are for government agencies. See Valador, Inc. v. HTC Corp., No. 1:16-cv-1162 (E.D. Va. Mar. 3, 2017) (Hr’g Tr.) at 32:2-16; 34:6-13.²

To demonstrate trademark infringement, plaintiff must prove “that it ha[d] a valid, protective trademark and that the defendant’s use of a colorable imitation of the trademark is likely to cause confusion among consumers.” Synergistic Int’l, LLC v. Korman, 470 F.3d 162, 171 (4th Cir. 2006) (quotation marks omitted). Of course, the touchstone of a trademark-infringement claim is whether “the defendant’s actual practice is likely to produce confusion in the minds of consumers about the origin of the goods or services in question.” George & Co., LLC v. Imagination Entm’t Ltd., 575 F.3d 383, 393 (4th Cir. 2009) (quoting CareFirst of Md., Inc. v. First Care, P.C., 434 F.3d 263, 267 (4th Cir. 2006)). Courts have recognized two forms of confusion in trademark infringement cases: “forward confusion” and “reverse confusion.” The traditional pattern of forward confusion occurs “when customers mistakenly think that the junior user’s goods or services are from the same source as or connected with the senior user’s goods or services.” 4 McCarthy on Trademarks and Unfair Competition § 23:10 (4th ed. 2017). Thus, a forward confusion case here would mean that con-suiners mistakenly think that the HTC Vive is connected with Valador. By contrast, in a reverse confusion case,³ the junior user overwhelms the market such that “customers purchase the senior user’s goods under the mistaken impression that they are getting the goods of the junior user.” Id.⁴ Here, reverse confusion would occur if someone purchasing Valador’s Vive software was under the mistaken impression that they were buying an HTC Vive. Thus, to prove likelihood of confusion under a reverse confusion theory, plaintiff must show “that because of the similar marks, [Valador]’s customers are likely [and mistakenly] to ... think that [HTC] is the source of, or is sponsoring or backing, [Valador]’s goods or services.” Id. Although plaintiff has not unequivocally chosen a theory, plaintiffs counsel at the summary judgmént hearing relied exclusively on reverse confusion.⁵

To support its claim of likelihood of confusion, plaintiff hired Christopher Bonney, who purports to be an expert in marketing, marketing research, and conducting market surveys. At plaintiffs request, Bonney performed a survey ostensibly to determine whether there is or likely will be confusion regarding the source of the parties’ products using the “Vive” marks.⁶ Specifically, “[t]he purpose of this study was to determine whether there is or will be confusion between two companies using similar marks to promote similar products; namely, computer software applications for three dimensional (3D) presentation of information and event simulation[.]’” Bon-ney Report (“Report”) at 5. The universe' for the survey comprised “men between the ages of 18 and 34 who enjoy virtual reality entertainment,” which -is, in Bonnets view, the target audience for the HTC Vive. Id. at 4. The survey defined “virtual reality” as “video games or computer generated simulation of a 3D image or environment that can be interacted with in a seemingly real or physical way.” Id. at 20. Although Bonney testified in his deposition that this definition of “virtual reality” is common, he could not remember where he obtained that definition.

The survey asked four sets of questions:

(1)The, first set of questions “addressed the extent to which the .Valador and HTC VIVE products are perceived by members of the HTC target audience to be described by!’ the Patent and Trademark Office’s' category purportedly “in question” in the case, which Bonney defined as “computer software application for three dimensional (3D) presentation of information and event simulation.” Id. at 13.

a. Respondents were orally presented with one-sentence descriptions purportedly of plaintiffs and defendants’ products, and asked whether they believed these products fit within the trademark category quoted above. Id. The survey described plaintiffs product, identified as “Product A,” as “[a] software program that integrates gaming technology, 3D modeling and realworld or simulation information-to give the user an interactive three-dimensional (3D) experience.” Id. The survey described the HTC Vive, identified as “Product B,’’ as “[a] gaming system that uses a headset and software combined with gaming controllers to give the user a- 3D virtual reality gaming experience.” Id.

(2) The second set of questions claimed to address “perceived similarities between Valador Vive and HTC Vive.” Id. at 14. The survey asked respondents to state whether the parties’ products, as described above, were ■ “very alike,” “somewhat alike,” “not very alike,” “not at all alike.” A respondent could also answer “I don’t know.” Id. at 21. The survey also asked, “How likely do you think it is that there will be confusion between the two products?” and offered the same answer choices. Id.

(3) The third set of questions sought to evaluate “perceived similarities between Valador and HTC VIVE names.” Id. at 14. The survey informed respondents that Product A ' “is called VIVE” and Product B “is also called VIVE.” Id. at 22. Thereafter, respondents were shown two seemingly identical, black-and-white 'images of the word “VIVE,” without any other styling, words, symbols, or images that typically accompany defendants’ alleged use of the “VIVE” mark in the marketplace. The survey also placed the labels, “Product A” and “Product B,” beneath these black-and-white images. Id. Importantly, the “VIVE” images respondents saw were not the actual images that plaintiff and defendants use with respect to their “VIVE” marks.⁷ Nor did the survey provide the participants with the actual marks as used in commerce.

a. The respondents were then asked “How alike or different would you say these two names are?” and provided the answer choices, “Very alike,” “Somewhat alike,” “Not very alike,” “Not at all alike,” and “I don’t know.” Id. at 22.

b. The survey offered the same answer choices to the question, “How likely do you think it is that there will be confusion between the two product names?” Id.

c. Next, the survey asked, “Do you think these two ways of showing the VIVE name look like they’re coming from the same company, or do they look [like] they come from two different companies?” Id. The respondents could answer “The same company,” “Different companies,” or “I don’t know.” Id.

d. The respondents were then asked, “How likely do you think it is that there will be confusion between the two VIVE names if they are used by different companies selling similar products?” The answer choices were “Very likely,” “Somewhat likely,” “Not very likely,” “Not at all likely,” and “I don’t know.” Id.

(4)The last set of questions asked participants about their understanding of the ® and ™ symbols.

Based on these questions and the survey responses, Bonney offered the following conclusions:

(1) “The persons interviewed in the Survey adequately represent the opinions of those consumers most likely to purchase virtual reality products,” id. at 6;

(2) “The Defendants’ use of the VIVE mark in commerce is likely to cause confusion among consumers as to the source of the parties’ respective VIVE-branded products,” id. at 7;

(3) “The Defendants’ use of the VIVE mark in commerce is likely to cause confusion among consumers as to whether HTC and Valador are the same company,” id.;

(4) “The Defendants’ use of the VIVE mark in commerce has caused confusion among consumers as to the source of VIVE branded products; the Survey respondents believe that Valador’s VIVE products and HTC’s VIVE products come from the same company,” id.;

(5) “There is likely to be confusion among consumers with Valador and HTC using the VIVE term as they sell similar products,” id.; and

(6) “Consumers of virtual reality products believe Valador’s and HTC’s products are similar,” id.

Bonney also filed a rebuttal report challenging defendants’ expert’s use of the standard “Eveready”⁸ methodology for analyzing likelihood of confusion in trademark cases.

Defendants have moved pursuant to Rule 702, Fed. R. Evid., to exclude Bonnets report, rebuttal report, opinions, and testimony. Specifically, defendants contend (1) that Bonney is unqualified, and (2) that Bonney used unreliable principles and methods to reach his conclusions. In the alternative, defendants contend that Bon-ney’s testimony must be excluded as unfairly prejudicial and misleading, pursuant to Rule 403, Fed. R. Evid.

For the reasons that follow, Bonney’s survey, findings, reports, and conclusions must be excluded pursuant to Rule 702, Fed. R. Evid.⁹

II.

Rule 702, Fed. R. Evid., contemplates a gatekeeping role for the district court. Specifically, “trial judges act as gatekeepers ‘to ensure that any and all scientific testimony ... is not only relevant, but reliable.’” Cooper v. Smith & Nephew, Inc., 259 F.3d 194, 199 (4th Cir. 2001) (quoting Daubert, 509 U.S. at 588, 113 S.Ct. 2786). This gatekeeping role is essential because juries are often unequipped to discern flaws in an expert’s methods or findings. Even a rigorous cross-examination may be inadequate to ensure that a jury looks behind the veneer of expertise and affords the testimony what weight it deserves. See id. (“[D]ue to the difficulty of evaluating their testimony, expert witnesses have the potential to be both powerful and quite misleading.” (quotation marks omitted)). Thus, “given the potential persuasiveness of expert testimony, proffered evidence that has a greater potential to mislead than to enlighten should be excluded.” Westberry v. Gislaved Gummi AB, 178 F.3d 257, 261 (4th Cir. 1999). The proponent of expert testimony therefore has the burden of establishing admissibility by a preponderance of proof. Cooper, 259 F.3d at 199.

To begin with, Rule 702 requires that the putative expert be qualified by “knowledge, skill, experience, training, or education[.]” Rule 702, Fed. R. Evid. If qualified, the witness may offer an expert opinion only if “(a) the expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; (b) the testimony is based on sufficient facts or data; (c) the testimony is the product of reliable principles and methods; and (d) the expert has reliably applied the principles and methods to the facts of the case.” Id.

In determining whether expert testimony is reliable, the trial court has “considerable leeway,” Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999), and a “great deal of discretion,” United States v. Dorsey, 45 F.3d 809, 814 (4th Cir. 1995). Indeed, there are several factors that “bear on the inquiry” whether expert opinions are reliable, including:

(1) whether the particular scientific theory “can be (and has been) tested”; (2) whether the theory “has been subjected to peer review and publication”; (3) the “known or potential rate of error”; (4) the “existence and maintenance of standards controlling the technique’s operation”; and (5) whether the technique has achieved “general acceptance” in the relevant scientific or expert community.

United States v. Crisp, 324 F.3d 261, 266 (4th Cir. 2003) (quoting Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786). Ultimately, “the objective of [the] gatekeeping requirement is to ‘make certain than an expert ... employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Cooper, 259 F.3d at 200 (quoting Kumho Tire, 526 U.S. at 152, 119 S.Ct. 1167).

Consumer surveys are generally admissible in trademark infringement cases. Indeed, “[u]sually, objections based on flaws in the survey’s methodology are properly addressed by the trier of fact.” PBM Prods., LLC v. Mead Johnson & Co., 639 F.3d 111, 123 (4th Cir. 2011) (holding that the district court did not abuse its discretion in admitting flawed expert testimony). Thus, the Fourth Circuit has observed that although “there will be occasions when the proffered survey is so flawed as to be completely unhelpful to the trier of fact and therefore inadmissible, such situations will be rare.” Id. (quoting AHP Subsidiary Holding Co. v. Stuart Hale Co., 1 F.3d 611, 618 (7th Cir. 1993)). Importantly, however, an expert’s survey and conclusions may be excluded under Rule 702 if the survey “was fundamentally flawed” and suffers from “fatal flaws,” as opposed to “mere technical flaws.” Citizens Fin. Grp., Inc. v. Citizens Nat’l Bank of Evans City, 383 F.3d 110, 121 (3d Cir. 2004) (holding that the district court properly excluded a likelihood of confusion survey that (1) used vague and imprecise language and (2) surveyed consumers outside of the relevant customer base).¹⁰ A likelihood of confusion survey is properly excluded if the witness does not qualify as an expert, or if the survey was not “conducted according to accepted principles” and in “a statistically correct manner.” M2 Software, Inc. v. Madacy Entm’t, 421 F.3d 1073, 1087 (9th Cir. 2005) (upholding exclusion of proffered likelihood of confusion survey). In addition, in analyzing the reliability of consumer surveys in Lanham Act cases, the Fourth Circuit has relied on the well-known treatise, McCarthy on Trademarks. See PBM Prods., 639 F.3d at 121-23.

These principles, applied here, point persuasively to the conclusion that defendants’ motion must be granted.

III.

Defendants’ motion must be granted bn two independent grounds: first, Bonney is not qualified to present his proffered opinions; second, Bonney’s survey — and his conclusions based on that survey — are “so flawed as to be completely unhelpful to the trier of fact and therefore inadmissible.” PBM Prods., 639 F.3d at 123.

A.

An expert may be qualified by “knowledge, skill, experience, training, or education[.]” Rule 702, Fed. R. Evid. In evaluating a proffered expert’s qualifications, “[t]he trial court ha[s] to decide whether this particular expert ha[s] sufficient specialized knowledge to assist the jurors in deciding the particular issues in the case.” Kumho Tire, 526 U.S. at 156, 119 S.Ct. 1167 (quotation marks omitted). In this regard, “[t]he district court should ... consider the proposed expert’s full range of experience and training, not just his professional qualifications.” Belk Inc. v. Meyer Corp. U.S., 679 F.3d 146, 162 (4th Cir. 2012) (cautioning against reading Rule 702’s qualification element too narrowly).

To be sure, Bonney appears to have four decades of experience as a market research consultant. Yet plaintiff has not made .the requisite showing that Bon-ney is qualified to opine on consumer confusion or proper survey methods in a trademark case. Bonney has no prior experience conducting surveys regarding likelihood of confusion involving claims of trademark infringement.¹¹ Indeed, Bonney admitted that he does “not have any specific knowledge or specialty in trademark cases.” Bonney Depo. at 121:25-122:9. Nor did Bonney review any likelihood of confusion surveys from previous trademark infringement cases before conducting his own survey in this matter. Bonney has never testified as an expert or performed work for anyone testifying as an expert in a trademark dispute or Lanham Act case. Similarly, he has never published on the topic of trademark surveys or likelihood of trademark confusion. Bonney further admitted at his deposition that he is unaware how courts analyze trademark infringement claims. Bonney Depo'. at 131:20-22.

As these facts disclose, Bonney is unqualified to offer opinions regarding likelihood of confusion in. this trademark infringement action. Not surprisingly, courts have excluded similarly unqualified witnesses from offering expert opinions regarding likelihood of confusion regarding trademarks.¹² So too, must Bonney be excluded.

In opposition to this conclusion, plaintiff argues (1) that Bonney has consulted .with trademark attorneys on trademark compliance issues,-(2) that Bonney has prepared other surveys used in litigation and made part of legal documents, and (3) that Bon-ney has previously served as a consulting expert in litigation. None of these arguments is persuasive. First, Bonney’s experience with compliance issues is irrelevant and insufficient, because likelihood of confusion — including the distinction between reverse and forward confusion — in the trademark context carries a particular meaning. And, as outlined infra, the failure to understand that meaning is likely to render a 'survey unreliable. Second, Bon-ney’s other surveys used in another legal proceeding had nothing to do with trademark infringement and are therefore inap-posite. And finally, although plaintiff asserts that Bonney previously served as a consulting expert in litigation, Bonney testified at his deposition that this “consultation” occurred over a decade ago, that he was not hired to consult on any trademark issues, and that the client “determined early on that [Bonney] wasn’t needed.” Bon-ney Depo. at 52:21-22. In fact, Bonney could not even remember whether the case on which he had consulted involved any trademark claims.

Put simply, Bonney lacks the necessary experience with trademark infringement claims to pass muster under Rule 702, Fed. R. Evid. In this regard, plaintiff has “not demonstrated that [Bonney] understands the intricacies of surveys prepared for trademark litigation or surveys testing the particular issue[ ] of ... likelihood of consumer confusion.” Radiance Found., 27 F.Supp.3d 671, 675 (E.D. Va. 2013). In other words, Bonney’s general expertise in the area of surveys and marketing is not “sufficiently] specialized knowledge to assist the jurors in deciding the particular issues in” this trademark infringement case. Kumho Tire, 526 U.S. at 156, 119 S.Ct. 1167; see also JFJ Toys, 2017 WL 679219, at *4 (marketing expert was unqualified to offer testimony about likelihood of confusion where his purported expertise was “devoid of specifics pertinent to the relevant market ... or the use of the marks ... in that same market”). Indeed, as detailed in Part III.B, infra, Bon-ney’s flawed survey methodology reflects his inexperience and lack of qualifications to offer opinions in this trademark infringement case.

Accordingly, because plaintiff has not shown by a preponderance of proof that Bonney is qualified to offer expert testimony on the likelihood of confusion about the origin of the goods or services in question, the Rule 702 motion to exclude must be granted. See Cooper, 259 F.3d at 199.

B.

Even assuming, arguendo, Bonney were qualified under Rule 702, his opinions and survey results would nonetheless be inadmissible. This is so because Bonney’s survey (1) failed to evaluate the proper universe of respondents, (2) did not replicate market conditions, (3> neglected to use a control group, (4) eschewed the recognized methodologies for conducting trademark confusion surveys, and (5) asked improperly leading questions. These fundamental flaws, taken together, render the survey so unreliable as to be inadmissible. '

(1) The Survey Did Not Cover the Proper Universe

As the McCarthy treatise makes clear, “[t]he first step in designing a survey” to gauge actual confusion “is to determine the ‘universe’ to be studied,” that is, the segment of the population whose perceptions and state of mind are relevant in this case. 6 McCarthy on Trademarks, § 32:159. This first step is crucial — even asking “proper questions” in a “proper manner” is likely to render irrelevant results if the survey probes .the “wrong persons.” Id.¹³Ultimately, the survey sample need not “exactly match” the correct universe; the survey need only cover a “sufficiently close approximation.” PBM Prods., 639 F.3d at 123.

Here, the choice of the correct or approximately correct survey universe turns on whether plaintiff is pursuing a forward -confusion or reverse confusion theory.¹⁴ Here, plaintiff appears to rely solely on reverse confusion — namely, that defendants (the junior users) have saturated the market with the HTC Vive and overwhelmed Valador’s use of its “VIVE” mark.¹⁵ And where, as here, a plaintiff asserts reverse confusion, the proper universe for a likelihood of confusion survey is the senior user’s potential customer base. 6 McCarthy on Trademarks, § 32:159. Thus, if plaintiff claims reverse confusion, Bonney should have focused on VhZodor’spotential customers, namely, government agencies. By contrast, if plaintiff asserts a forward confusion theory — that defendants have attempted to trade on plaintiffs goodwill and reputation — the “proper universe to survey is the potential buyers of the junior user’s goods or services.” Id. (first emphasis added). Thus, to the extent plaintiff alleges forward confusion, Bonney’s survey should have targeted potential purchasers of the HTC Vive.

It is clear that under either theory of trademark confusion, Bonney failed to survey the correct population — or even a “sufficiently close approximation” of the correct population. PBM Prods., 639 F.3d at 123. In designing his survey Bonney focused on what he believed to be the target audience for the HTC Vive. This was plainly the incorrect universe for a reverse confusion case. Indeed, Bonney’s surveying defendants’ audience for reverse confusion would be akin to the government’s counting heads in China for the U.S. Census. To obtain results pertinent to a reversion confusion claim, Bonney should have surveyed Valadofs customers — the vast majority of which comprise government agencies — or at least a “sufficiently close approximation” of those customers. See PBM Prods., 639 F.3d at 123. Because the survey did not even attempt to gauge or contact Vala-dor’s customer base, Bonney’s report and opinions regarding a likelihood of reverse confusion are unreliable.

And to the extent plaintiff alleges forward confusion — that HTC Corporation, HTC America, and Valve sought to trade on Valador’s goodwill — Bonney again failed to survey the correct universe. Put succinctly, Bonney failed to survey prospective buyers of the HTC Vive. Nevertheless, plaintiff emphasizes that that Bon-ney targeted those most likely to view HTC Corporation’s advertising and thus, in plaintiffs view, Bonney appropriately limited the survey to men aged 18-34 who “enjoy virtual reality entertainment.” Report at 4. There are several defects in this argument. To begin with, relevant case law correctly shows that the proper population for a forward confusion claim comprises prospectivepurchasers, not just those most likely to view an advertisement,¹⁶ This is so for the obvious reason that “[r]espondents who are not potential consumers may well be less likely to be aware of and to make relevant distinctions ... than those who are potential consumers.” Weight Watchers Int'l, Inc. v. Stouffer Corp., 744 F.Supp. 1259, 1272 (S.D.N.Y. 1990); see also Citizens Fin. Grp., Inc. v. Citizens Nat’l Bank of Evans City, 383 F.3d 110, 121 (3d Cir. 2004) (upholding the exclusion of a likelihood of confusion survey where the respondents were not part of the relevant customer base).

The survey further missed the mark because it was over- and under-inclusive. See 6 McCarthy on Trademarks § 32:161 (collecting cases holding over- and under-inclusive likelihood of confusion surveys unreliable). Thus, the survey does not sufficiently approximate the population of prospective purchasers. Specifically, the survey was over-inclusive because it targeted 18- to 34-year old men who generally “enjoy virtual reality entertainment.” Report at 4. That is far too broad.¹⁷ Indeed, Bonnets survey resembles the survey rejected in Weight Watchers, a trademark infringement suit over unauthorized use of plaintiffs “Lean Cuisine” mark. See 744 F.Supp. 1259. Notably, the district court in Weight Watchers held that a survey was unreliable and over-inclusive where the survey universe comprised women between the ages of 18 and 55 who (1) had purchased frozen food entrees in the past six months and (2) had tried to lose weight through diet or exercise in the past year. Id. at 1272. The court concluded that because “some of the respondents may not have been in the market for diet food of any kind ... the study universe therefore was too broad.” Id. This same logic holds here: simply because survey respondents are men who “enjoy virtual reality entertainment” and are likely to see an HTC Vive advertisement, that does not mean that the respondents were “in the market” for a product like the HTC Vive. See id. The survey’s population was therefore too broad.

In addition to being over-inclusive, Bon-nejfs survey was also under-inclusive. Specifically, the survey (1) excluded all women and (2) failed to cover the “average” purchaser of the HTC Vive — men between 29 and 45 years old, not men aged 18-34. See Def s Interog. Response (stating that relevant sales were made “on average” to men aged 29 to 45).¹⁸ Thus, because the survey did not cover the prime market for defendants’ product, some courts might go so far as to call Bonney’s survey “worthless.” See Ty Inc. v. Softbelly’s Inc., 353 F.3d 528, 531 (7th Cir. 2003) (observing that a trademark survey was “worthless” where the relevant “prime market consisted] of girls 5 to 14,” but the survey’s universe comprised girls aged 13 to 18). Even more fundamentally, it is unclear how Bonney derived his definition of “virtual reality entertainment,” and the survey never asked any questions to determine how respondents understood that term. Plaintiff has therefore failed to show that the survey identified a proper population.

In sum, because Bonney failed to survey a sufficiently close approximation of the correct universe, his findings and opinions are too unreliable to pass muster under Rule 702 and the principles set forth in Daubert.

(2) The Survey Did Not Replicate Market Conditions

Not only did the challenged survey fail to focus on the correct consum-er universe, it also failed to replicate matv ket conditions, This constitutes an added and adequate ground for excluding Bon-ney’s findings under Rule 702. See Water Pik, Inc. v. Med-Sys., Inc., 726 F.3d 1136 (10th Cir. 2013) (upholding exclusion of a likelihood of trademark confusion survey that did not show the parties’ marks as they actually appeared on packaging in the marketplace).¹⁹ To meet Rule 702’s requirement that the expert evidence be reliable and helpful to the jury, Bonney’s survey should have replicated market conditions for the simple reason that trademark infringement occurs if ..“the defen-' dant’s actual practice is likely to produce confusion in the minds of consumers about the origin of the goods or services in question.” CareFirst of Md., Inc. v. First Care, P.C., 434 F.3d 263, 267 (4th Cir. 2006).²⁰ Finally, in determining “whether a survey methodology sufficiently simulates marketplace conditions, the focal point must be the specific products tested by the survey.” THOIP v. Walt Disney Co., 690 F.Supp.2d 218, 236-37 (S.D.N.Y. 2010).

Here, Bonney’s survey is unreliable because it failed to replicate actual market conditions in which consumers might encounter the parties’ marks. To begin with, the survey did not show the marks as they appear in commerce. Instead, the survey showed participants two seemingly identical, black-and-white images of the word “VIVE,”, without any other, styling, words, symbols or images that typically accompany the word, “Vive” as either Valador or defendants use that term in the marketplace.²¹ Notwithstanding the fact that the HTC Vive mark typically appears in commerce accompanied by a triangular symbol, blue coloring, and a stylized “HTC” logo, Bonney excluded those images and font. In this respect, Bonney’s deposition reveals that he consciously chose not to represent the parties’ marks accurately in the survey:

Q. And so you thought it was fine to just disregard , any surrounding . symbols or., .company names that typically are accompanying the word ‘VIVE’?

A. Yes.

[[Image here]]

Q. Your focus was not on portraying even just the words ‘VIVE’ the way those might appear in the marketplace, it was just to show how the word looks disregarding any other features; is that right?

A. That’s correct, because that would have given them different advantages. One — if I recall, the Valador is not used with a whole a lot [sic] of things around it while the HTC is used with [a] little diamond logo and with HTC and sometimes- stream, Steam, or' whatever, Steam, are underneath it, but not always

Bonney Depo. at 242:2-15; 243:13-18 (emphasis added). In other words, Bonney deliberately altered defendants’ mark to make it appear more similar to plaintiffs before asking questions about the marks’ likelihood of confusion.²² That approach is antithetical to trademark infringement analysis. See George, 575 F.3d at 393 (“In assessing whether [a likelihood of] confusion exists, ‘we look to how the two parties actually use their marks in the market-placef.]’” (quoting CareFirst, 434 F.3d at 267)).

In summary, Bonney’s failure to replicate market conditions is another adequate ground for granting the motion to exclude.

(3) The Survey Did Not Employ a Control

As courts routinely hold, a survey’s lack of a control group or control questions constitutes yet another ground for granting a Rule 702 motion to exclude.²³ As the Tenth Circuit has recognized, where, as here, a survey reflects “raw figures” and the “raw confusion rate” among respondents, those figures “are usually unhelpful ... in predicting likelihood of confusion, because [these figures] can be inflated by background noise, or false positives arising from something other than the particular confusion that is alleged and that the survey aims to capture.” ²⁴ Water Pik, 726 F.3d at 1148 (upholding the district court’s conclusion that a survey was “devoid of any probative value and therefore irrelevant ... because [of] several serious methodological flaws in the survey” (citation omitted)). The McCarthy treatise echoes this principle, noting,

“[A] properly constructed survey has at least two groups of respondents: one group (the “test cell”) is shown, the allegedly infringing mark; the second -group (the “control cell”) is shown a mark similar in appearance to the test cell, except for the designation whose influence is being tested.

6 McCarthy on Trademarks § 32:187. In other words, a properly constructed likelihood of trademark confusion survey will “isolate confusion arising specifically from the contested mark [by] substituting] for the contested mark a control mark that shares as many characteristics with the contested mark as possible, with the key exception of the characteristic whose influence is being assessed,” Water Pik, 726 F.3d at 1148, Accordingly, “[b]y discounting for confusion arising from the control, the survey can measure net confusion, or ‘the difference between the raw confusion, percent and the control confusion percent.’ ” Id. at 1148-49 (quoting 6 McCarthy on Trademarks § 32:187). Put succinctly: a control helps ensure reliable results.

It is apparent here that Bonney’s survey did not include a control to account for potential error, false positives, or background noise. Rather, Bonney appears to rely solely on the raw confusion rate among all survey respondents without having used a control of any kind. Moreover, Bonney admitted that he neither used nor considered using a control in his survey. See Bonney Depo. at 151: 13-17. (Q. Did you use a control in any way in the survey submitted in this case? A. No. Q. Did you consider using a control? A. No.”). Thus, it is impossible to gauge and correct for the significant potential error rate. See, e.g., United States v. Crisp, 324 F.3d 261, 266 (4th Cir. 2003) (noting that courts examining reliability under Rule 702 should consider the “known or potential rate of error” and the “maintenance of standards controlling the technique’s operation”). As a result, Bonney’s findings are unreliable.²⁵

(4) The Survey Did Not Employ a Recognized Methodology

There are two common methods for surveying the likelihood of confusion. 6 McCarthy on Trademarks § 32:173.50. Bonney used neither.

The first is the “Eveready”²⁶ method; the second is the “Squirt”²⁷ method. The “Eveready” survey, which is particularly apt where the plaintiffs senior mark is strong and widely-recognized, does not inform respondents what the senior mark is, but assumes that they are aware of the mark from their prior experience. The Squirt format, on the other hand, presents the respondent with both of the conflicting marks. See 6 McCarthy on Trademarks § 32:173:50-174. Importantly, however, in both survey methodologies the respondent is shown the entire mark as used in the marketplace. See id.

Bonney’s survey did not follow either of these standard methodologies, nor did his survey show respondents the parties’ actual marks as they appear in the marketplace. See supra Part III.B.2. Bonney did not even have a clear idea as to which products he was attempting to compare. Bonney did not have a particular Valador product in mind when designing his survey; indeed, he did not know if Valador even had a device called the “Valador VIVE.” Bonney Depo. at 162:23-25. This is unsurprising, given that Valador does not appear to brand any of its end products or deliverables with its mark, and instead simply dubs one of its software applications the “VIVE” and refers to its mark in some contracts. Thus, because the survey is unclear as to the products and marks being compared, Bonney’s conclusions regarding the likelihood of confusion regarding those products and marks are hardly reliable. Cf. THOIP, 690 F.Supp.2d at 236-37 (“When ascertaining whether a survey methodology sufficiently simulates marketplace conditions, the focal point must be the specific products tested by the survey.”).

Bonney’s methodology is further unreliable because he created his methodology specifically for this litigation.²⁸ And since this litigation-driven methodology appears to be novel, it has never enjoyed general acceptance, been tested, or faced peer review. See Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786. Thus, although a putative expert opinion is not automatically inadmissible if it relies on a novel method, plaintiff has not provided enough indicia of reliability here.

Seeking to avoid this result, plaintiff contends that Bonney used “best practices” for marketing research and that Bonney conducted a “blind” survey. These arguments are unpersuasive, because Bon-ney does not identify what those best practices are, how he complied with them, how they translate to a trademark infringement case, or whether his approach to finding a likelihood of confusion between trademarks has ever been tested, subjected to peer review, or enjoyed general acceptance. See Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786. And the survey’s blindness is does not save it. A misleading survey that asks unreliable questions to the wrong universe without a control is unreliable regardless whether the respondents know who to thank for the enterprise.

Thus, Bonney employed an unreliable methodology that warrants exclusion pursuant to Rule 702, Fed. R. Evid.

(5) The Survey’s Questions Were Leading or Suggestive

Finally, the survey is also unreliable because the questions were improperly suggestive. Suggestive questions render a survey unreliable by creating “demand effects” or “cues” from which a respondent can “infer the purpose of the survey and identify the ‘correct’ answers.” 6 McCarthy on Trademarks § 32:172. In other words, leading questions “bias the survey by suggesting to respondents, at least implicitly, that they should believe there is at least some sort of relationship between the different items when the possibility might not even have occurred to the vast majority of consumers who see the items.” Simon Prop. Grp. L.P. v. mySimon, Inc., 104 F.Supp.2d 1033, 1048 (S.D. Ind. 2000). Thus, “[a] survey question that begs its answer by suggesting a link between plaintiff and defendant cannot be a true indicator of consumer confusion.” Scott Fetzer Co. v. House of Vacuums Inc., 381 F.3d 477, 488 (5th Cir. 2004) (rejecting a survey that asked, “Looking at this ad, would you say that this company is in any way affiliated with, connected with, sponsored by, associated with or authorized by the Kirby Company?”).²⁹

Here, the survey does precisely what it should not: It improperly suggests to participants a link between plaintiff and defendants. In this regard, Questions 7, 9, and 11 of the survey ask “How likely do you think it is that there will be confusion between” the two products, the two product names, and the two “VIVE” names “if they are used by different companies selling similar products[.]” See Report at 21-22. Thus, these questions imply that there is confusion, and then ask respondents to estimate how severe the confusion will be. Similarly, Question 10 creates a “demand effect,” or cue, by asking “Do you think these two ways of showing the VIVE name look like they’re coming from the same company, or do they look [like] they come from two different companies?” This question creates a demand effect because it indicates that the two “VIVE” labels in the survey come from the same company. See Hornady Mfg. Co., Inc. v. Doubletap, Inc., 746 F.3d 995, 1005 (10th Cir. 2014) (holding that plaintiffs survey was improperly leading when it asked if the three packages shown individually were of the same, affiliated or unaffiliated sources).³⁰ Moreover, the survey’s questions do not accurately reflect how the parties use their “VIVE” marks in commerce; plaintiff typically refers to its mark in contracts, without branding any products or deliverables with that mark, and defendants’ “Vive” almost always appears with “HTC” adjacent to “Vive,” a unique logo, and stylized font. In other words, the survey feeds respondents a false, suggestive premise, that the “VIVE” are more similar than they actually are.

Thus, because the survey is improperly suggestive, it must be excluded as unreliable.³¹

IV.

In sum, it is apparent that plaintiff has failed to meet its burden to demonstrate that Bonney’s testimony is admissible under Rule 702, Fed. R. Evid. Defendants’ motion to exclude must therefore be granted.

An appropriate Order will issue.

1 . See Rule 78(b), Fed. R. Civ. P. (“By rule or order, the court may provide for submitting and determining motions on briefs, without oral hearings.”); E.D. Va. Local Civ. R. 7(J) (“In accordance with Fed. R. Civ. P. 78, the Court may rule upon motions without an oral hearing.").

2 . Plaintiffs mark is an acronym for a "Vala-dor Immersive Visual Environment,” a software environment plaintiff uses to develop applications.

3 .The Fourth Circuit has not explicitly adopted the reverse confusion theory. See Dick's Sporting Goods, Inc. v. Dick’s Clothing & Sporting Goods, Inc., 188 F.3d 501, 1999 WL 639165, *12 (4th Cir. 1999) (Table Decision) (“To date, this Court has not adopted the doctrine of reverse confusion.”). Yet, the Fourth Circuit has favorably cited it. See MicroStrategy Inc. v. Motorola, Inc., 245 F.3d 335, 347 (4th Cir. 2001) (describing the facts presented as “a classic case of ‘reverse confusion.’ ”). In addition, the Patent and Trademark Office and district courts in this circuit have applied the reverse confusion theory. See, e.g., In re Shell Oil Co., 992 F.2d 1204 (Fed. Cir. 1993); Coryn Grp. II, LLC v. O.C. Seacrets, Inc., 868 F.Supp.2d 468 (D. Md. 2012); Buzz Off Insect Shield, LLC v. S.C. Johnson & Son, Inc., 606 F.Supp.2d 571 (M.D.N.C. 2009). Because plaintiff has invoked the reverse confusion theory, analysis proceeds assuming the validity of a reverse confusion claim.

4 . In a reverse confusion trademark case, "[t]he public comes to assume the senior user's products are really the junior users or that the former has become somehow connected to the latter.” Ameritech, Inc. v. Am. Info. Techs. Corp., 811 F.2d 960, 964 (6th Cir. 1987). In this respect, the senior user "loses the value of the trademark—its product identity, corporate identity, control over its goodwill and reputation, and ability to move into new markets.” Id.

5 . See Valador, Inc. v. HTC Corp., No. 1:16-cv-1162 (E.D. Va. Mar. 3, 2017) (Hr’g Tr.) at 5:22-6:2; 6:18-21; 55:22-25.

6 . Of course, as stated supra, plaintiff does not label or brand any of its end products or deliverables with its “VIVE” mark, but simply (and occasionally) refers to the "VIVE” mark in government contracts. See Valador, Inc. v. HTC Corp., No. 1:16-cv-1162 (E.D. Va. Mar. 3, 2017) (Hr’g Tr.) at 32:2-16; 34:6-13.

7 . In fact, it is unclear whether Bonney had an accurate image of the HTC Vive mark in the first place, as Bonney admitted to having obtained defendants’ VIVE mark from a Reddit webpage, Id. at 5.

8 . See Union Carbide Corp. v. Ever-Ready, Inc., 531 F.2d 366 (7th Cir. 1976), Over time, courts, commentators, and practitioners have elided “Ever-Ready” into “Eveready.” See, e.g., 6 McCarthy on Trademarks § 32:174. As explained infra, the Eveready method is one of the standard, accepted approaches for surveying likelihood of confusion in trademark infringement actions. See id.

9 . Defendants’ argument that the evidence is unfairly prejudicial under Rule 403 need not be reached or decided.

10 . The Fourth Circuit in PBM Products favorably cited this decision. See PBM Prods., 639 F.3d at 123 (citing Citizens Fin. Grp., 383 F.3d 110); see also Scott Fetzer Co. v. House of Vacuums Inc., 381 F.3d 477, 488 (5th Cir. 2004) ("[Although minor methodological flaws will affect weight rather than admissibility, a survey can be so badly flawed that it cannot be used to demonstrate the existence of a question of fact on the likelihood of consumer confusion.” (quotation marks omitted)).

11 . Although Bonney testified at his deposition that he has conducted consumer awareness surveys regarding confusion, he conceded that none of these surveys involved trademark infringement claims or trademark litigation. See Bonney Depo. at 55:25-56-7, This experience is insufficient, because likelihood of confusion in the trademark context has a specific meaning, and unless an expert understands that meaning, his survey may — as here—fail to target the proper population or ask the right questions. See, e.g., Radiance Found., 27 F.Supp.3d at 675 (excluding witness testimony despite witness's "extensive experience researching consumer trends and some degree of experience in trademark litigation” because "[p]laintiffs have not demonstrated that she understands the intricacies of surveys prepared for trademark litigation or surveys testing the particular issues of dilution and likelihood of consumer confusion”).

12 . See, e.g., JFJ Toys, Inc. v. Sears Holdings Corp., Civil Action No. PX-14-3527, 2017 WL 679219, at *4 (D. Md. Feb. 21, 2017) (holding that a marketing expert’s qualifications were “wholly inadequate” to offer testimony about likelihood of confusion where the witness’s purported expertise was "devoid of specifics pertinent to the relevant market... Or the use of the marks ... in that same market”); Radiance Found. Inc. v. NAACP, 27 F.Supp.3d 671, 675 (E.D. Va. 2013) (excluding a witness's opinions regarding likelihood of consumer confusion because (1) the witness lacked "sufficient specialized knowledge related to the core issues pertaining to ,.. confusion,”. (2) "her general expertise-in the area of surveys and marketing [wa]s not sufficiently specialized to assist in deciding the particular issues of the [trademark infringement] case,” and (3) she was “unfamiliar with basic trademark law concepts, commonly used trademark litigation survey methods and interpretation of statistical tables to measure dilution and confusion”).

13 . See also Amstar Corp. v. Domino’s Pizza, Inc., 615 F.2d 252, 264 (5th Cir. 1980) ("[0]ne of the most important factors in.assessing the validity of an opinion poll is the adequacy of the ‘survey universe,’ that is, the persons interviewed must adequately represent the opinions which are relevant to the litigation.”).

14 . See Part I, supra, for a discussion of the forward confusion and reverse confusion theories.

15 . See supra note 5 and accompanying text.

16 . See, e.g., Amstar Corp. v. Domino’s Pizza, Inc., 615 F.2d 252, 264 (5th Cir. 1980) ("The appropriate universe should include a fair sampling of those purchasers most likely to partake of the alleged infringer's goods or services.” (emphasis added)); Paco Sport Ltd. v. Paco Rabanne Parfums, 86 F.Supp.2d 305, 322 (S.D.N.Y. 2000) ("The courts have held that to be probative on the issue of actual confusion, a survey must rely on responses of prospective purchasers of the products in question.” (collecting cases)).

17 . See, e.g., Water Pik, Inc. v. Med-Sys., Inc., 726 F.3d 1136, 1146 (10th Cir. 2013) (upholding exclusion of over-inclusive survey in a trademark dispute between two sellers of sinus-irrigation products, because the expert "selected respondents from those whom he regarded as potential purchasers of sinus remedies in general, not potential purchasers of sinus-irrigation products in particular”).

18 . See also Universal City Studios, Inc. v. Nintendo Co., 746 F.2d 112 (2d Cir. 1984) (holding that a telephone survey of people who had purchased defendant's video game was too narrow to cover the correct universe, because the survey should also have covered those who were contemplating a purchase).

19 . See also Malletier v. Dooney & Bourke, Inc., 525 F.Supp.2d 558 (S.D.N.Y, 2007) ("A shr-vey that uses a stimulus that makes no attempt to replicate how the marks are viewed by consumers in real life may be excluded on that ground alone.” (citing Am. Footwear Corp. v. Gen. Footwear Co. 609 F.2d 655 n.4 (2d Cir. 1979) (observing that the district court properly rejected a survey that failed to replicate “actual market conditions"))).

20 . See also Universal City, 746 F.2d at 117 ("[T]o determine if confusion is likely, each trademark must be compared in its entirety; juxtaposing fragments of each mark does not demonstrate whether the marks as a whole are confusingly similar.”); Malletier v. Burlington Coat Factory Warehouse Corp., 426 F.3d 532, 538 (2d Cir. 2005) ”[T]he Lanham Act requires a court to analyze the similarity of the products in light of the way in which the marks are actually displayed' in their ptir-chasing context.”).

21 .It is worth observing, once again, that plaintiff does not appear to brand any product or deliverable with its "VIVE” mark. Plaintiff instead appears to list its mark on its website in plain text, and to insert references to "VIVE” in its service contracts with government clients.

22 . In addition, Bonney testified that he obtained defendants’ VIVE mark from a "Red-dit” page, as opposed to defendants’ website, because defendants’ "VIVE logo does not appear very large or in a ... setting that ... presents an apples-to-apples comparison, the[re are] background colors or other confusing elements.” Bonney Depo. at 167:14-22.

23 . See, e.g., U.S. Polo Ass'n, Inc. v. PRL USA Holdings, Inc., 800 F.Supp.2d 515, 534 (S.D.N.Y. 2011) (giving no weight to a survey lacking a control because "[w]ithout a proper control, there is no benchmark for determining whether a likelihood of confusion estimate is significant or merely reflects flaws in the survey methodology"), aff'd, 511 Fed.Appx. 81 (2d Cir. 2013); Nat'l Football League Props., Inc. v. ProStyle, Inc., 57 F.Supp.2d 665 (E.D. Wis. 1999) (excluding a survey, that lacked a control); Ga.-Pac. Consumer Prod. LP v. Myers Supply, Inc., No. 6:08-cv-6086, 2009 WL 2192721 (W.D. Ark. 2009) (disregarding a survey because, among other things, the survey lacked a control), aff'd, 621 F.3d 771 (8th Cir. 2010).

24 .A raw confusion rate is simply (a) the number of respondents who agree that a mark is confusingly similar divided by (b) the total number of respondents. Water Pik, 726 F.3d at 1148.

25 . See 6 McCarthy § 32:187 (“Courts have held that a survey that fails to use a control may be ... excluded from evidence altogether.” (collecting cases))

26 . See Ever-Ready, 531 F.2d 366.

27 . See SquirtCo. v. Seven-Up Co., 628 F.2d 1086 (8th Cir. 1980).

28 . See, e.g., United States v. Tran Trong Cuong, 18 F.3d 1132, 1143-44 (4th Cir. 1994) ("Reports specifically prepared for purposes of litigation are not, by definition, of a type reasonably relied upon by experts in the particular field.” (quotation marks omitted)); Wehling v. Sandoz Pharms. Corp., 162 F.3d 1158, 1998 WL 546097, at *3 (4th Cir. 1998) (Table Decision) ("Another significant fact weighing against admitting the testimony is where, as here, the expert developed his opinions expressly for the purposes of testifying.”); Mike’s Train House, Inc. v. Lionel, L.L.C., 472 F.3d 398, 408 (6th Cir. 2006) (noting that courts should be “suspicious of methodologies created for the purpose of litigation”).

29 . Specifically, the Fifth Circuit in Scott Fetter concluded that such a question improperly "prodded survey participants to search for any connection, no matter how attenuated, between the two companies ... instead of permitting participants to make their own associations.” 381 F.3d at 488; see also Sunbeam Corp. v. Equity Indus. Corp., 635 F.Supp. 625, 630 (E.D. Va. 1986) (“Survey questions [that] beg their answers ‘cannot be a true indicator of the likelihood of consumer confusion,' and cannot be relied on to create an issue of fact so as to make summary judgment improper.” (quoting Universal City, 746 F.2d at 118 & n.8)).

30 . Simon, 104 F.Supp.2d at 1048 (noting that a "question about whether the two items are put out by the same or a related source is likely to generate so-called 'demand effects’ ").

31 . In defense of Bonney's survey, plaintiff emphasizes that Bonney claims to have found a likelihood of confusion between the parties’ trademarks, But this argument is inapposite. The Supreme Court in Daubert emphasized that the decision to admit or exclude expert testimony based on reliability "must be solely on principles and methodology, not on the conclusions that they generate.” 509 U.S. at 595, 113 S.Ct. 2786. In other words, the salient issue is Bonney’s reliability under Rule 702 , not his importance to plaintiff's case.