Stupp Co. v. United States

Slip Op. 23-23
UNITED STATES COURT OF INTERNATIONAL TRADE
STUPP CORPORATION ET AL.,
Plaintiffs and Consolidated
Plaintiffs,
and
MAVERICK TUBE CORPORATION,
Plaintiff-Intervenor and
Consolidated Plaintiff-Intervenor,
Before: Claire R. Kelly, Judge
v.
Consol. Court No. 15-00334
UNITED STATES,
Defendant,
and
SEAH STEEL CORPORATION AND
HYUNDAI STEEL COMPANY,
Defendant-Intervenors and
Consolidated Defendant-
Intervenors.
OPINION AND ORDER
[Sustaining the U.S. Department of Commerce’s third remand redetermination in the
less-than-fair-value investigation of welded line pipe from the Republic of Korea.]
Dated: February 24, 2023
Jeffrey M. Winton and Jooyoun Jeong, Winton and Chapman PLLC, of Washington,
D.C., argued for plaintiff SeAH Steel Corporation.
Consol. Court No. 15-00334 Page 2
Robert R. Kiepura, Trial Attorney, Commercial Litigation Branch, Civil Division,
U.S. Department of Justice, of Washington, D.C., argued for defendant. With him on
the brief were Claudia Burke, Assistant Director, Patricia M. McCarthy, Director,
and Brian M. Boynton, Principle Deputy Assistant Attorney General. Of Counsel
was Mykhaylo Gryzlov, Senior Counsel, Office of the Chief Counsel for Trade
Enforcement and Compliance, U.S. Department of Commerce, of Washington, D.C.
Jeffrey D. Gerrish, Schagrin Associates, of Washington, D.C., argued for plaintiff
Welspun Tubular LLC USA. With him on the brief were Roger B. Schagrin and Saad
Y. Chalchal.
Kelly, Judge: Before the court is the U.S. Department of Commerce’s
(“Commerce”) third remand redetermination in its 2015 less-than-fair-value
investigation of welded line pipe imported from the Republic of Korea (“Korea”). See
Final Results of Redetermination Purs. Ct. Remand, April 4, 2022, ECF No. 208
(“Remand Results”); see also Welded Line Pipe From [Korea], 80 Fed. Reg. 61,366(Dep’t Commerce Oct. 13, 2015) (final determination of sales at less than fair value),
as amended by Welded Line Pipe From [Korea], 80 Fed. Reg. 69,637 (Dep’t Commerce
Nov. 10, 2015) (“Amended Final Determination”) and accompanying Issues &
Decisions Memo, A-580-876, (Oct. 5, 2015), ECF No. 30-3 (“Final Decision Memo”).
In Stupp Corporation v. United States, the Court of Appeals for the Federal Circuit
vacated this court’s opinion, remanding to Commerce to further explain why it is
reasonable to apply the Cohen’s d test as part of its differential pricing analysis if
certain statistical assumptions have not been met. Stupp Corporation v. United
States, 5 F.4th 1341 (Fed. Cir. 2021) (“Stupp III”). For the following reasons, the court
sustains Commerce’s third remand redetermination.
Consol. Court No. 15-00334 Page 3
BACKGROUND
The court presumes familiarity with the facts of this case as set out in this
court’s previous opinions, as well as the Court of Appeals’ decision in Stupp III, and
now recounts only the facts relevant to the court’s review of the Remand Results. On
November 14, 2014, Commerce initiated an antidumping duty investigation of welded
line pipe from Korea. Welded Line Pipe From [Korea], 79 Fed. Reg. 68,213, 68,213
(Dep’t Commerce Nov. 14, 2014) (initiation of less-than-fair-value investigation).
Commerce published its final determination on October 5, 2015 and, finding that
39.72% of SeAH Steel Corporation’s (“SeAH”) U.S. sales passed the Cohen’s d test,
applied the average-to-transaction method to those sales. Final Decision Memo. at 4.
Commerce accordingly calculated a 2.53% dumping margin for SeAH. Amended
Final Determination at 69,638. SeAH appealed, arguing that Commerce’s
differential pricing analysis and application of the Cohen’s d test were contrary to
law and unsupported by substantial evidence. See Stupp Corp. v. United States, 359
F. Supp. 3d 1293, 1302 (Ct. Int’l Tr. 2019) (“Stupp I”), reconsideration denied, 365 F.
Supp. 3d 1373 (Ct. Int’l Tr. 2019). SeAH also argued that Commerce improperly
rejected its case brief, which contained citations to certain academic texts not part of
the administrative record. Id. at 1300–03; Letter from Commerce Rejecting SeAH’s
Sept. 1, 2015 Case Br., 1–2, PD 384, bar code 3302027-01 (Sept. 3, 2015); [SeAH’s]
Case Br., PD 377–79, bar codes 3301610-01–03 (Sept. 1, 2015) (“SeAH’s Rejected
Brief”).
Consol. Court No. 15-00334 Page 4
This court sustained Commerce’s determinations with respect to its use of
differential pricing analysis and rejection of SeAH’s case brief. Stupp I, 359 F. Supp.
3d at 1299–1306. Specifically, the court found that Commerce correctly rejected
SeAH’s brief because the academic authorities cited in the brief constituted new
factual information intended to advance SeAH’s arguments. Id. at 1301. The court
also found that Commerce’s differential pricing analysis was supported by
substantial evidence because, among other reasons, Commerce was not required to
apply the Cohen’s d test in accordance with academic literature. Id. at 1302–06.
The Court of Appeals remanded, instructing Commerce to further explain why
its use of the Cohen’s d test was reasonable in light of “significant concerns” related
to application of the test. Stupp III, 5 F. 4th at 1357. Specifically, the Court of Appeals
questioned the reasonableness of Commerce’s application of Cohen’s d test to data
failing to satisfy the statistical criteria of normality, equal variance, and sufficient
observation size. Id. 1357–60. Citing to academic literature examining the use of
Cohen’s d test to measure effect size, the Court of Appeals expressed concern that
Commerce’s failure to satisfy the statistical criteria assumed by Cohen’s test could
“undermine the usefulness of the interpretive cutoffs,” resulting in artificially
inflated dumping margins. Id. at 1357. The Court of Appeals affirmed the remaining
issues from Stupp I, including this court’s decision to uphold Commerce’s rejection of
SeAH’s case brief. Id. at 1344.
Consol. Court No. 15-00334 Page 5
JURISDICTION AND STANDARD OF REVIEW
The court has jurisdiction pursuant to 28 U.S.C. § 1581(c) (2018), which grants
the court authority to review actions initiated under 19 U.S.C. § 1516a(a)(2)(B)(i)1
contesting the final determination in an antidumping duty order. The court will
uphold Commerce’s determination unless it is “unsupported by substantial evidence
on the record, or otherwise not in accordance with law.” 19 U.S.C. § 1516a(b)(1)(B)(i).
“The results of a redetermination pursuant to court remand are also reviewed ‘for
compliance with the court’s remand order.’” Xinjiamei Furniture Co. v. United
States, 968 F. Supp. 2d 1255, 1259 (Ct. Int’l Tr. 2014).
DISCUSSION
On remand, SeAH challenges Commerce’s application of the Cohen’s d test on
the grounds that (1) assumptions underlying the test have not been met, (2) the large
cutoff prescribed by the test is arbitrary, and (3) random variables such as exchange
rates can cause “false positives.” See Cmts. of [SeAH] on Final Determ. on Remand,
5–36, June 14, 2022, ECF No. 216 (“SeAH’s Cmts.”). Defendant and Welspun Tubular
LLC (“Welspun”) counter that (1) the assumptions are inapplicable, (2) Commerce’s
application of Cohen’s d test leads to reasonable results, (3) the cutoff is supported by
statistical literature, (4) SeAH cannot introduce non-record documents for the first
time on remand, and (5) SeAH failed to exhaust administrative remedies for its
1 Further citations to the Tariff Act of 1930, as amended, are to the relevant
provisions of Title 19 of the U.S. Code, 2018 edition.
Consol. Court No. 15-00334 Page 6
exchange rate-related arguments. See Def.’s Corr. Resp. to Cmts., 9–34, Sept. 22,
2022, ECF No. 230 (“Def.’s Reply”); [Welspun’s] Reply [SeAH’s] Cmts. on Remand
Redeterm., 18–32, Aug. 15, 2022, ECF No. 218 (“Welspun’s Reply”). For the following
reasons, the court sustains the results of Commerce’s remand redetermination.
I. SeAH’s Non-Record Documents
SeAH’s comments to the Remand Results reference several pieces of academic
literature which were not included in the administrative record. See SeAH’s Cmts.
at 6–36. Welspun and Defendant argue that the court should disregard these
materials, as judicial review is limited to the agency record. Welspun’s Reply at 19–
20; Def.’s Reply at 10–12. SeAH argues that the court may take judicial notice, or
otherwise consider, these materials to better understand the statistical principles
behind Cohen’s d test. Reply of [SeAH] to Responses by Def. and [Welspun], 10–11,
Sept. 28, 2022, ECF No. 236 (“SeAH’s Reply”). For the following reasons, the court
need not take judicial notice of SeAH’s non-record documents to understand the
statistical principles they illustrate.
Judicial review is generally limited to the administrative record before the
agency at the time it rendered its decision. See Camp v. Pitts, 411 U.S. 138, 142
(1973). “The purpose of limiting review to the record actually before the agency is to
guard against courts using new evidence to ‘convert the “arbitrary and capricious”
standard into effectively de novo review.’” Axiom Res. Mgmt, Inc. v. United States,
Consol. Court No. 15-00334 Page 7
564 F.3d 1374, 1380 (Fed. Cir. 2009) (quoting Murakami v. United States, 46 Fed. Cl.
731, 735 (2000), aff’d, 398 F.3d 1342 (Fed. Cir. 2005)).
This court previously upheld Commerce’s decision to reject SeAH’s non-record
documents, on the grounds that the submissions constituted new factual information
not on the administrative record. Stupp I, 359 F. Supp. 3d at 1299–1306. SeAH’s
submissions primarily cited academic articles relating to application of the Cohen’s d
test under certain conditions. SeAH’s Rejected Brief at 26–33. On appeal, the Court
of Appeals affirmed this court’s decision rejecting the non-record information,
concluding that SeAH’s materials were not introduced to correct inaccuracies in
Commerce’s reporting, but to support its argument challenging Commerce’s use of
Cohen’s d test. Stupp III, 5 F.4th at 1350. In Stupp III, the Court of Appeals
nevertheless referenced and quoted from several of the non-record texts introduced
by SeAH.2 Id. at 1357–59. On remand, Commerce asked SeAH to place the
2 The Federal Circuit cited the following five works: Grissom, Robert and Kim, John,
Effect Sizes for Research: Univariate and Multivariate Applications (2nd ed. 2012), A-
580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Grissom & Kim”); Coe,
Robert, It’s the Effect Size Stupid: What Effect Size Is and Why It Is Important, paper
presented at the Annual Conference of the British Educational Research Association
(September 2002), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Coe”);
Lane, David, et al., Introduction to Statistics, Online Edition, A-580-876, PRRD 8,
bar code 4181776-01 (Nov. 12, 2021) (“Lane”); Algina, James, Keselman, H.J., and
Penfield, Randall, An Alternative to Cohen’s Standardized Mean Difference Effect
Size: A Robust Parameter and Confidence Interval in the Two Independent Groups
Case, 10 Psychological Methods (2005), A-580-876, PRRD 8, bar code 4181776-01
(Nov. 12, 2021) (“Algina”); Li, Johnson Ching-Hong, Effect Size Measures in a Two-
Independent-Samples Case With Nonnormal and Nonhomogenous Data, Behavioral
Research (2015), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Li”).
Consol. Court No. 15-00334 Page 8
previously-rejected materials on the record, which SeAH did. See Letter from
[Commerce] to Interested Parties, A-580-876, PRRD 1, bar code 4176823-01 (Oct. 29,
2021); SeAH Submission of Publications Requested, A-580-876, PRRD 8, bar code
4181776-01 (Nov. 12, 2021). In its comments on the remand redetermination, SeAH
again cites new academic sources not on the record, arguing that the court may
consider the underlying statistical principles which the texts discuss.3 See SeAH’s
Cmts. at 6–25. SeAH claims that the Court of Appeals considered SeAH’s previous
academic sources in Stupp III, despite upholding Commerce’s rejection of SeAH’s
brief which contained these materials. SeAH’s Reply at 10–11. Although SeAH
states that the Court of Appeals took judicial notice of the texts, it later clarified that
the court may consider the statistical principles regardless of whether the texts
themselves are on the record. Response of [SeAH] to Def’s Sur-Reply, 2–3, Nov. 14,
2022, ECF No. 247 (“SeAH’s Sur-Reply”).4
3 SeAH cites to the following six non-record sources in its comments: Todd D. Little,
Oxford Handbook of Quantitative Methods in Psychology (2013); Ricca and Blaine,
Notes on a Nonparametric Estimate of Effect Size, 90:1 Journal of Experimental
Education 249 (2022); Hedges and Olkin, Overlap Between Treatment and Control
Distributions as an Effect Size Measure in Experiments, 21:1 Psychological Methods
61 (2016); Huberty and Lohman, Group Overlap as a Basis for Effect Size, 60:4
Educational and Psychological Measurement 543 (2000); J. Cohen, A Power Primer,
112:1 Psychological Bulletin 155 (1992); F. Alvarez, A. Atkeson, and P. Kehoe, If
Exchange Rates Are Random Walks, Then Almost Everything We Say about
Monetary Policy is Wrong, Federal Reserve Bank Of Minneapolis Research
Department Staff Report 388 (2007).
4 SeAH argues that the Court of Appeals’ decision “stands for the proposition that,
when an agency purports to be using a statistical test in accordance with widely-
adopted statistical practice, the courts may consider non-record academic materials
to evaluate that claim.” SeAH’s Sur-Reply at 2.
Consol. Court No. 15-00334 Page 9
Consistent with the approach of the Court of Appeals, the court may recognize
the basic statistical principles discussed in these texts. The idea, for example, that a
skewed statistical sample may yield inaccurate results is inductive reasoning—not
an assertion of fact. The Court of Appeals’ references to academia do not render its
reasoning dependent on academic sources. Thus, the court considers Commerce’s
Cohen’s d methodology in the same way it would review any other methodology, and
may make logical inferences without taking judicial notice of SeAH’s literature.
II. Administrative Exhaustion
SeAH argues that random fluctuations of exchange rates can affect the Cohen’s
d test, and lead to inaccurate results. SeAH’s Cmts. at 24–28. Welspun and
Defendant argue SeAH failed to properly exhaust this argument. Welspun’s Reply
at 20–21; Def.’s Reply. at 27–28. For the following reasons, the court concludes that
SeAH has exhausted this argument.
Pursuant to 28 U.S.C. § 2637(d), the court “shall, where appropriate, require
the exhaustion of administrative remedies,” including at the preliminary
determination stage before the agency. 28 U.S.C. § 2637(d); 19 C.F.R § 351.309(c)(2).
Section 2637(d) grants the court “discretion to identify circumstances where
exhaustion of administrative remedies does not apply.” ABB, Inc. v. United States,
920 F.3d 811, 818 (Fed. Cir. 2019) (quoting Consol. Bearings Co. v. United States,
348 F.3d 997, 1003 (Fed. Cir. 2003)). The court may also excuse exhaustion in certain
Consol. Court No. 15-00334 Page 10
circumstances, such as when a party is raising a “pure question of law.” Agro Dutch
Indus. Ltd. v. United States, 508 F.3d 1024, 1029 (Fed. Cir. 2007).
In its comments on the draft redetermination, SeAH argued that random
factors, such as exchange rates, could cause the standard deviation of test populations
to vary significantly. SeAH’s Cmts. on Draft Redetermination, 17–20, A-580-876,
PRRD 30, bar code 4224356-02 (March 21, 2022). SeAH did not provide an exchange
rate table, or assert that its actual sales during the period of review were affected by
these factors. See id. Subsequently, in its comments on the final remand results,
SeAH again argues that Cohen’s d could be significantly affected by random factors
where the population of data is not normally distributed. SeAH’s Cmts. at 24–28.
SeAH adds that its sales were, in fact, affected by fluctuations in the exchange rate
between the U.S. dollar and Korean won, because its inland freight expenses were
denominated in won. Id. at 24. Welspun counters that SeAH failed to raise its
exchange rate argument and supporting factual information during the draft
redetermination. Welspun’s Reply at 26.
SeAH has exhausted its exchange rate argument. SeAH’s exchange rate
examples provide an illustration of how it believes random factors can render the
Cohen’s d test inaccurate when values are not normally distributed. Normal
distributions is one of the three assumptions that the Court of Appeals remanded to
Commerce to explain. See Stupp III, 5 F. 4th at 1360. Therefore, Welspun’s argument
that Commerce had no opportunity to address SeAH’s exchange rate calculations
Consol. Court No. 15-00334 Page 11
misses the point; these calculations are not a new argument, but an illustration of
the same scenario Commerce was directed to explain.
SeAH separately argues Commerce must “ignore” fluctuations in exchange
rates pursuant to 19 U.S.C. 1677b-1(a). SeAH’s Br. at 8. SeAH concedes that it did
not raise this argument in its comments to the draft remand results; nevertheless, it
argues that this argument may be considered as a “pure question of law.” Oral
Argument at 0:27:39–0:27:53. Defendant argues that Commerce had no opportunity
to consider this argument on remand, and Welspun characterizes the argument as a
mixed question of law and fact. Oral Argument at 0:24:15–0:26:33, 0:27:56–0:28:39.
Whether § 1677b-1(a) is pertinent to Commerce’s differential pricing analysis is a
matter of statutory interpretation, not subject to exhaustion requirements. See Agro
Dutch Indus., 508 F.3d at 1029. However, SeAH’s argument that § 1677b-1(a) directs
Commerce to compensate for exchange rate variations is inapposite. In its full
context the statute directs Commerce to use the exchange rate “in effect on the date
of sale” for valuation of merchandise, and to ignore fluctuations on that particular
date. 19 U.S.C. 1677b-1(a). The plain language does not mandate that Commerce
compensate for a respondent’s decision to report expenses in a foreign currency, as
SeAH suggests.
III. Differential Pricing Analysis
In Stupp III, the Court of Appeals remanded for further explanation of
Commerce’s application of the Cohen’s d test as part of its differential pricing
Consol. Court No. 15-00334 Page 12
analysis. Stupp III, 5 F.4th at 1360. On remand, SeAH renews its argument that
Commerce’s application of Cohen’s d test is flawed because it fails to take into account
assumptions of sample size, distribution, and variance underlying the test, and
further argues Commerce’s choice of Cohen’s large cutoff is arbitrary. SeAH’s Cmts.
at 6–24. SeAH also claims random fluctuations in exchange rates can affect the d
coefficient, causing even test groups with identical prices to pass. Id. at 24–28.
Commerce counters that its Cohen’s d analysis does not operate in a vacuum, and
must be considered with the ratio test and meaningful difference test. See Remand
Results at 26, 28, 30–31, 41–42, 54–60. Commerce also argues the cutoffs are tied
to real-world criteria, that small fluctuations in price will not lead to “false positives”
in Cohen’s test, and that use of the 0.8 threshold results in reasonably infrequent
application of alternative methodologies. Remand Results at 16–19, 32, 54–60. For
the following reasons, Commerce has adequately addressed Court of Appeals’
concerns.
When investigating whether subject merchandise is being sold at less than fair
value, Commerce typically compares “the weighted average of the normal values to
the weighted average of the export (and constructed export prices) for comparable
merchandise” unless it determines another method is appropriate. 19 U.S.C. § 1677f-
1(d)(1)(A)(i); 19 C.F.R. § 351.414(c)(1). This average-to-average (“A-to-A”) method
compares the weighted average of a respondent’s home country sales prices during
the investigation period to the weighted average of the respondent’s sales prices in
Consol. Court No. 15-00334 Page 13
the United States during the same period. See 19 C.F.R. § 351.414(b)(1). One concern
with the A-to-A method is that it may allow a foreign producer or exporter to engage
in “targeted dumping,” which occurs when an exporter sells at a dumped price to
particular customers or regions, while selling at higher prices to other customers or
regions. See Apex Frozen Foods Priv. Ltd. v. United States, 862 F.3d 1337, 1341 (Fed.
Cir. 2017) (“Apex II”). As a result, higher-priced products can mask dumped products
when Commerce averages the sales using the A-to-A method.
Congress addressed concerns over targeted dumping with the passage of 19
U.S.C. § 1677f-1(d)(1)(B). See Apex II, 862 F.3d at 1342. Section 1677f-1(d)(1)(B)
allows Commerce to compare “the weighted average of the normal values to export
prices . . . of individual transactions for comparable merchandise if (i) there is a
pattern of export prices . . . for comparable merchandise that differ significantly
among purchasers, regions or periods of time, and (ii) [Commerce] explains why such
differences cannot be taken into account using [the A-to-A method or transaction-to-
transaction method5].” 19 U.S.C. § 1677f-1(d)(1)(B)(i)–(ii). Targeted dumping is more
likely when export prices fit a pricing model that differs significantly across different
market segments. Apex II, 862 F.3d at 1341–42. Congress has not provided a method
for Commerce to use to determine whether a pattern of significantly different prices
5Commerce’s regulations provide that the transaction-to-transaction method, which
compares prices of individual transactions, will be employed only in rare cases, “such
as when there are very few sales of subject merchandise and the merchandise sold in
each market is identical or very similar or is custom-made.” 19 C.F.R. § 351.414(c)(2).
Consol. Court No. 15-00334 Page 14
exists. However, the Statement of Administrative Action (“SAA”) of the Uruguay
Round Agreements Act explains that Commerce should proceed “on a case-by-case
basis, because small differences may be significant for one industry or one type of
product, but not for another.” Uruguay Round Agreements Act, Statement of
Administrative Action, H.R. Doc. No. 103-316, vol. 1, at 842–43 (1994), reprinted in
1994 U.S.C.C.A.N. 4040, 4178.6
To determine whether the criteria set forth in § 1677f-1(d)(1)(B) are met,
Commerce conducts a “differential pricing analysis” of a respondent’s sales. See
Differential Pricing Analysis; Request for Comments, 79 Fed. Reg. 26,720, 26,722
(Dep’t of Commerce May 9, 2014). This analysis contains three tests. First,
Commerce applies to respondent’s sales what it refers to as the “Cohen’s d test,”
described in more detail below, which measures the degree of price disparity between
groups of sales. Id. Commerce then counts the percentage of sales by value which
“pass” the Cohen’s d test, and applies its “ratio test.” Id. at 26,722–23. If 33% of
respondent’s sales or less pass, Commerce uses the A-to-A method, and if 66% or
more pass, Commerce uses the A-to-T method. Id. If the total percentage of passing
sales is between 33% and 66%, Commerce takes a hybrid approach, applying the A-
to-T method to those sales passing the test, and the A-to-A method to the remainder.
Id. Finally, if Commerce has not selected the A-to-A method for all sales, it applies
6 The SAA is an “authoritative expression by the United States concerning the
interpretation and application” of the Uruguay Rounds Agreement Act. 19 U.S.C.
§ 3512(d).
Consol. Court No. 15-00334 Page 15
the “meaningful difference” test to determine whether the A-to-A method could
nevertheless account for the disparate pricing. Id. at 26,723. Commerce applies the
test by comparing a respondent’s dumping margin using both A-to-A and the selected
method. Id. If the A-to-A margin is below the de minimis threshold and the margin
from the selected method is not, or if both margins are above the threshold and differ
by 25% or more, Commerce continues to use the selected method; otherwise,
Commerce applies the A-to-A method for all sales. Id.As applied by Commerce, the Cohen’s d test involves comparing the product-
specific prices of “test groups” of a respondent’s sales to a “comparison group” by
region, purchaser, and time period. Stupp III, 5 F.4th at 1346. For each category,
Commerce segregates sales into subsets, with one subset becoming the test group,
and the remaining subsets being combined as the comparison group. Id. Commerce
then calculates the means and standard deviations of the test and comparison groups.
Id. Commerce then calculates a Cohen’s d coefficient by dividing the difference in the
groups’ means by the groups’ standard deviation.7 Id. Each subset is thus tested
against the remaining subsets across each category, and assigned a d coefficient by
7 Thus, d = |mean of test group – mean of control group| ÷ standard deviation.
Commerce uses a modified version of this formula, substituting the square root of the
simple average of the groups’ variances for standard deviation. The Cohen’s d test
solves for a coefficient representing “effect size.” See generally Cohen, Jacob,
Statistical Power Analysis for the Behavioral Sciences, (2nd ed. 1988), A-580-876,
PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Cohen”). “Effect size quantifies the
size of the difference between two groups, and may therefore be said to be a true
measure of the significance of the difference.” Coe at 7.
Consol. Court No. 15-00334 Page 16
solving Cohen’s ratio. If the d value of a test group is equal to or greater than the
“large threshold,” or 0.8, the observations within that group are said to have “passed”
the Cohen’s d test. Id. at 1347.
In Stupp III, the Court of Appeals expressed concern that Commerce’s
application of Cohen’s d under certain circumstances could undermine the usefulness
of the test in less-than-fair-value determinations. Specifically, the Court of Appeals
identified three potential scenarios in which use of Cohen’s d could be problematic:
first, when the distribution of a respondent’s sales data is not normal, second, when
the test groups have few data points, and third, when there is minimal variance in a
respondent’s sales. Stupp III, 5 F.4th at 1357–59. The assumption of normality is
satisfied when a fixed percentage of the population falls within each standard
deviation from the mean—in other words, that a population density graph generally
shows a symmetrical, bell-shaped curve. See Starnes, Yates, and Moore, Statistics
through Applications, 116 (2005), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12,
2021). The assumption of size is satisfied when the population is sufficiently large.
See Cohen at 21. The assumption of homogeneous variances is satisfied when the
standard deviations of test and comparison groups are similar. See Grissom at 68–
69. Commerce argued in Stupp III, as it does now, that the three assumptions are
only relevant as a matter of statistical significance, and do not apply when analyzing
a whole population; the Court of Appeals concluded that this explanation did not fully
address its concerns. Stupp III, 5 F.4th at 1360.
Consol. Court No. 15-00334 Page 17
The Court of Appeals illustrated the problems it identified with the Cohen’s d
test through two hypotheticals. First, the Court of Appeals considered a situation in
which Commerce analyzed a group of only eight export sales across four groups, such
that each test group would consist of only two sales. Stupp III, 5 F. 4th at 1358–59.
With groups of such small numbers, the Court of Appeals pointed out that there
would be some upward bias in effect size, such that the test would produce more
“passing” results, and potentially exaggerate dumping margins. Id. at 1359. The
Court of Appeals also observed that a group of only two sales would lack normality.
Id. Second, the Court of Appeals described a test group of five sales of about $100
each, which differed from one-another by up to two cents. Id. Because the standard
deviation of such a group would be so small, the Court of Appeals pointed out that
the denominator in Cohen’s ratio would be drastically reduced, again causing an
increase in effect size, and inflating the resulting dumping margin. Id. The Court of
Appeals noted that an objective examiner looking at these similar sales prices “would
be unlikely to conclude that they embody a ‘pattern’ of prices which ‘differ
significantly.’” Id. (citing 19 U.S.C. § 1677f-1(d)(1)(B)(i)).
Commerce reasonably explains that Cohen’s d test does not operate in a
vacuum, but as part of the differential pricing analysis as a whole.8 Turning first to
8 The parties devote a significant part of their briefings to discussion of (1) the
permissibility of using Cohen’s d test on full populations, and (2) questions of
(footnote continued)
Consol. Court No. 15-00334 Page 18
the assumptions of population size and normalcy, the Court of Appeals questioned
whether small sample sizes without normal distributions could “exaggerate”
dumping margins by introducing an “upward bias” to effect size. Stupp III, 5 F.4th at
1359.9 Addressing the Court of Appeals’ concerns about population size, Commerce
statistical significance versus practical significance. See Remand Results at 11–16,
43–51; SeAH’s Cmts. at 7–10; Def.’s Br. at 12–20; Welspun’s Br. at 21–23; SeAH’s
Reply at 17–27; Def’s Sur-Reply at 12–29, 34–35; SeAH’s Sur-Reply at 16–24. Neither
question is determinative of whether Commerce’s methodology is reasonable. Both
arguments have already been raised before the Court of Appeals, which concluded
that they did not resolve its concern over whether the absence of certain assumptions
forecloses Commerce’s use of Cohen’s d test. Stupp III, 5 F.4th at 1360.
Commerce correctly asserts that a “t-test” for statistical significance is used
with sampled data, and that Dr. Cohen considered normal distribution and equal
variance as necessary assumptions in a t-test. See Cohen at 19; Remand Results at
12–16. However, Commerce improperly reasons that because there is no need for a
t-test, there is no basis for the assumptions. Remand Results at 14. Commerce also
asserts that SeAH’s assumptions are only relevant as a matter of statistical
significance, and that they do not apply because Cohen’s d test determines practical
significance. Remand Results at 14, 43–45. That these assumptions are required
for questions of statistical significance does not answer the question of whether they
are also needed to determine practical significance, as the Court of Appeals suggests.
See Stupp III, 5 F.4th at 1360.
Although SeAH claims that academic sources do not support Commerce’s use
of Cohen’s d in its differential pricing analysis, this argument is inapposite. SeAH’s
decision to substantially advance its arguments using labels taken from statistical
literature does not alter the court’s obligation on review. See Soc Trang Seafood Joint
Stock Co. v. United States, 321 F. Supp. 3d 1329, 1339 n.13 (2018) (“the fact that
Commerce has adopted a methodology based upon a statistical tool known as Cohen’s
d, and chooses to refer to this methodology as Cohen’s d, does not diminish the
discretion granted to Commerce”); see also Mid Continent Steel & Wire, Inc. v. United
States, 31 F.4th 1367 (Fed. Cir. 2022) (“Commerce’s job is not to follow a statistical
test as explained in published literature for its own sake, but to implement the
statutory mandate to determine when prices of certain groups “differ significantly”).
9 Although the parties dispute whether such results are really “false positives,” it is
undisputed that in at least some instances, groups with as few as two sales have
(footnote continued)
Consol. Court No. 15-00334 Page 19
explains that its Cohen’s d analysis does not stand alone, and operates together with
the ratio test and meaningful difference test. See Remand Results at 26, 28, 30–31,
41–42, 54–60. Thus, even if the Cohen’s d values of small test groups were less
accurate than for large test groups, this difference does not by itself render
Commerce’s use of Cohen’s test unreasonable, because the ratio test and meaningful
difference test compensate for inaccuracies. See id. Commerce’s differential pricing
analysis looks at the frequency and impact of effect size to detect targeted dumping—
not the effect size alone. See Cohen at 8; Remand Results at 26–28. As Commerce
points out in its remand redetermination, the “sole purpose of the Cohen’s d test” is
to determine whether prices “differ significantly” across region, time period, or
customer. Remand Results at 41. The “pattern” of export prices which Commerce
must find under 19 U.S.C. § 1677f-1(d)(1)(B)(i) is then determined by the ratio test.
Id. at 42. The ratio test has already been approved by the Court of Appeals, which
found that Commerce’s choice of the 33% and 66% thresholds was a “reasonable
choice.” Stupp III, 5 F.4th at 1355. SeAH’s attacks on Cohen’s d test presuppose that
what SeAH claims are “false positives” automatically affect the accuracy of
passed Cohen’s test. See Stupp III, 5 F.4th at 1357; SeAH’s Cmts. at 17–19; see
Remand Results at 55, 58–59. Identifying results as “false” positives begs the
question of what is a false positive. See Remand Results at 59 (“To label this result
a ‘false-positive’ does not render the variances inaccurate or erroneous”). SeAH
illustrates this situation using data from its own sales, showing how a group of only
two sales to a single customer passed Cohen’s test, despite SeAH’s observation that
a visual comparison of the groups on a graph showed those sales to be near the
average price. SeAH’s Cmts. at 18. Commerce counters that a visual inspection may
be inadequate in situations involving complex calculations. Remand Results at 59.
Consol. Court No. 15-00334 Page 20
Commerce’s differential pricing analysis, when in fact Commerce has allowed for 33%
positives before there is any potential effect on a respondent’s dumping margins.
Commerce also addresses the Court of Appeals’ concern whether samples
without normal distributions will produce an inappropriate number of passes. SeAH
points to numerous academic sources which it claims confirm the usefulness of
Cohen’s test is compromised when comparing data sets with non-normal
distributions. See SeAH’s Cmts. at 7, n.19 (citing Cohen at 13); Id. at 12 (citing Ellis
at 41); Id. at 13 (citing Starnes, Yates, and Moore at 135). The Court of Appeals has
acknowledged some of these sources. See Stupp III, 5 F.4th at 1357–59 (citing Cohen
at 21, Grissom & Kim at 66, Coe at 13, Lane at 645, Algina at 318, and Li at 1571).
The court need not opine on the relevance of these academic observations;10 however,
it logically follows that a relatively large-tailed distribution (i.e., with large standard
deviation) in a test group would tend to decrease Cohen’s d coefficient, while the
opposite would result in an increase. See Remand Results at 29 (“in other words, the
fat-tailed distribution may undervalue the significance of effect”) (emphasis in
10 The task of the court is not to interpret the meaning of literature treating with
correct application of Cohen’s d. Rather, the court must determine whether
Commerce’s methodology is reasonable in light of considerations that run counter to
its decision. See Motor Vehicle Mfrs. Ass’n of U.S. v. State Farm Mut. Auto. Ins. Co.,
463 U.S. 29, 43 (1983); Ceramica Regiomontana, S.A. v. United States, 636 F. Supp.
961, 966 (Ct. Int’l Tr. 1986), aff’d, 810 F.2d 1137, 1139 (Fed. Cir. 1987) (“As long as
the agency’s methodology and procedures are reasonable means of effectuating the
statutory purpose, and there is substantial evidence in the record supporting the
agency’s conclusions, the court will not impose its own views as to the sufficiency of
the agency’s investigation or question the agency’s methodology”).
Consol. Court No. 15-00334 Page 21
original). SeAH focuses on the second of these two scenarios, arguing that even
inputting random data, such as exchange rates, can cause test groups to frequently
pass Cohen’s d test. SeAH’s Cmts. at 25–28. SeAH further argues that the ratio test
does not account for such random fluctuations.11 Oral Argument at 0:42:57–0:43:43.
Commerce addresses these arguments by explaining that even if Cohen’s test can
produce positive results under unusual circumstances, this possibility does not mean
its use of Cohen’s d is unreasonable when combined with the ratio test and
meaningful difference test. See Remand Results at 26, 28, 30–31, 41–42, 54–60.
The Court of Appeals also specifically asked Commerce to explain why it can
use the 0.8 threshold identified by Dr. Cohen as a measure of a significant price
difference, when Commerce evaluates data which fails to meet statistical
assumptions of normality, size and variance.12 Stupp III, 5 F.4th at 1360. Although
Commerce reiterates those assumptions are irrelevant, see Remand Results at 11–
11 Commerce explains that, even if exchange rate fluctuations do affect prices, this
effect is not “random” because a respondent can control in which currency it
denominates its prices. Remand Results at 45; Oral Argument at 0:49:11–0:50:05.
12 Although the Court of Appeals approved the 0.8 cutoff in Mid Continent Steel &
Wire, Inc. v. United States, it explained in Stupp III that it had yet to consider the
reasonableness of the 0.8 cutoff value when the assumptions in question have not
been met. Stupp III, 5 F.4th at 1356–57(“We held that . . . it is reasonable to adopt
that [0.8] measure where there is no better objective measure of effect size. We did
not, however, address SeAH’s second argument [on assumptions] in Mid Continent”)
(citation omitted) discussing Mid Continent Steel & Wire, Inc. v. United States, 940
F.3d 662, 673 (Fed. Cir. 2019); see also Oral Argument at 1:39:45–1:40:30. More
specifically, SeAH argues that it is unreasonable to compare its prices, which are not
normally distributed, using a subjective benchmark that was derived from a
normally-distributed population. SeAH’s Cmts. at 10–12.
Consol. Court No. 15-00334 Page 22
16, it also explains its choice of the 0.8 threshold as a function of its differential
pricing analysis. First, Commerce explains that it employs the 0.8 threshold to
identify where prices “differ significantly” pursuant to 19 U.S.C. § 1677f-1(d)(1)(B)(i).
Remand Results at 11. Second, Commerce states the 0.8 measurement “represents a
difference which is ‘grossly perceptible.’” Remand Results at 52. Reasonably
discernible from this statement is that Commerce considers a significant difference
to be grossly perceptible in the same way that Dr. Cohen identified a large threshold
as one that is “grossly perceptible.” See Cohen at 27. The SAA to the Uruguay Round
Agreements Act directs Commerce to proceed “on a case-by-case basis, because small
differences may be significant for one industry or one type of product, but not for
another,” SAA at 842–43; thus, Commerce’s choice of a measurement that is a
function of standard deviation as a uniform approach to identify differences as
significant is reasonable, even if the absolute difference in means is small.
Commerce’s approach tailors the question of what is a significant difference in price
to the pricing parameters of different products. Third, Commerce adequately
explains its adoption of Cohen’s widely-recognized choice of 0.8 as a large threshold
as significant. Remand Results at 18. It explains that it chose the 0.8 standard
because it was “a conservative standard to determine that the observed price
differences are significant.” Id. Commerce summarizes its reasoning by explaining
that “[u]sing Dr. Cohen’s thresholds is a reasonable approach to interpret whether
the difference in the prices is significant and the further interpretation of the
Consol. Court No. 15-00334 Page 23
difference in the prices in the context of the calculation of dumping margins ensures
the reasonable and limited application of the alternative comparison methodology.”
Id. at 33. Thus, Commerce chose a threshold it predicted would result in limited
application of the alternative methodology.
Although Commerce adopted this yardstick from Dr. Cohen, and did so because
it was widely acknowledged in the statistical literature, Commerce does not rely on
the prominence of this yardstick alone. Commerce elaborates that its “actual
application of the Cohen’s d test in the context of the differential pricing analysis
resulted in the application of an alternative comparison methodology to a relatively
small number of respondents.” Remand Results at 32. Discernible from Commerce’s
explanation is that the 0.8 cutoff produces reasonable passing rates once the ratio
and meaningful difference tests are applied. SeAH challenges Commerce’s reliance
on the 0.8 threshold as large, arguing that Commerce’s only basis for using the
threshold is that it is widely accepted. SeAH’s Cmts. at 10–11. However, in addition
to relying on a widely-accepted standard for “grossly perceptible” to determine what
is significant, Commerce defines “significant” with reference to the impact a price
difference has on a respondent’s dumping margins. Remand Results at 32. Finding
that the 0.8 threshold leads to relatively few determinations of targeted dumping,
Commerce concludes that its choice is reasonable. Id.
Congress delegated to Commerce the authority to determine where a price
difference is significant. 19 U.S.C. § 1677f-1(d)(1)(B)(i). Congress also made clear
Consol. Court No. 15-00334 Page 24
that the definition of a “significant price difference” would depend on the product at
issue. See SAA at 842–43. Thus, Congress entrusted Commerce to use its expertise
and knowledge of pricing to gauge price distinctions. Cf. Fujitsu General Ltd. v.
United States, 88 F.3d 1034, 1039 (Fed. Cir. 1996) (granting Commerce significant
deference in determinations “involv[ing] complex economic and accounting decisions
of a technical nature”). Commerce’s decision to adopt Cohen’s 0.8 (“large”) threshold
as a measure of significance because it is widely accepted in the statistical literature
does not undermine the reasonableness of that choice, if it is based on Commerce’s
expertise and Commerce demonstrates the reasonableness of that choice with
reference to the impact it has on the differential pricing analysis. Thus, Commerce’s
reference to Cohen’s work does not circumscribe its discretion to choose the same
values in a new context, because that choice is itself reasonable.
Commerce addresses the Court of Appeals’ concern that prices with small
variances, which hover around the same value, will produce inaccurate results on
Cohen’s test. As an initial matter, Commerce explains that results which pass
Cohen’s test under these circumstances are not “false positives,” as small differences
in average prices will mean that variances, too, will be small. Remand Results at 59;
Oral Argument at 1:06:32–1:07:02. Thus, it is discernable that a small variance
means a small difference in price will be more significant, and a passing result under
these circumstances is not necessarily “erroneous.” Remand Results at 59.
Nevertheless, the Court of Appeals observed that an objective examiner considering
Consol. Court No. 15-00334 Page 25
a group of sales where prices differed by only a few cents would be unlikely to conclude
that they show a “pattern” of prices that “differ significantly” under the statute.
Stupp III, 5 F.4th at 1359. Commerce responds to this issue by pointing out that an
examiner would indeed conclude that there was no pattern—because Commerce does
not look for a pattern at this stage of its differential pricing analysis. Remand Results
at 41. Again, Commerce explains that the ratio test determines whether a pattern
exists, while Cohen’s d test only shows whether there are significant price differences.
Id. at 41–42. Thus, Cohen’s test would need to generate enough “false positives” to
overcome the 33% threshold, at minimum, and there is no evidence on the record
suggesting that price patterns, such as that proposed by the Court of Appeals, occur
with frequency in SeAH’s sales.
Additionally, to specifically address the hypothetical proposed by the Court of
Appeals, Commerce explains that, in addition to the ratio test, the meaningful
difference test would prevent low-variance sales which pass Cohen’s d test from
impacting a respondent’s dumping margins. See Remand Results at 30–31. Adopting
the Court of Appeals’ example in which all of a respondent’s prices hovered around
$100 and passed Cohen’s test, Commerce explains that even in this extreme scenario,
the respondent would still be assessed under the A-to-A method. Id. Choosing a
normal value for comparison equal to the highest sales price, and thus maximizing
the respondent’s theoretical dumping margin, Commerce observes the margin would
still be under the 2% de minimis threshold. Id.; see 19 U.S.C § 1673d(a)(4),
Consol. Court No. 15-00334 Page 26
1673b(b)(3). SeAH argues that Commerce’s reliance on the meaningful difference
test is misplaced, because even changes of less than 2% in a respondent’s dumping
margin can cross the de minimis threshold and result in a “meaningful difference”
finding. SeAH’s Sur-Reply at 15. Specifically, SeAH argues that when the Cohen’s
d results from small-variance data sets of different products are cumulated,
Commerce may find that a respondent’s sales pass the thresholds for both ratio test
and the meaningful difference test, even if price differences are negligible. Oral
Argument at 1:13:37–1:16:06.13 This argument overstates Commerce’s position. It is
reasonably discernable that Commerce does not rely on the meaningful difference
test to prevent all “inappropriate” passes from affecting a respondent’s dumping
margins. Commerce has explained the meaningful difference test compensates for a
specific concern with low-variance sales which the Court of Appeals identified. See
Stupp III, 5 F.4th at 1359; Remand Results at 30–31. Moreover, SeAH’s argument is
misplaced, because the question before the court is not whether it is possible to
construct an unusual scenario where Cohen’s d test can result in an alternative
comparison method. Rather, the question is whether Commerce’s use of Cohen’s test,
when applied as a component of its differential pricing analysis, is reasonable. See
Ceramica Regiomontana, S.A. v. United States, 636 F. Supp. 961, 966 (Ct. Int’l Tr.
13SeAH does not argue that it received an alternative method because its own
combined sales inappropriately passed the Cohen’s d test. Rather it offers a
hypothetical to challenge the reasonableness of Commerce’s methodology more
generally. SeAH’s Cmts. at 8, 14–21.
Consol. Court No. 15-00334 Page 27
1986). Thus, for the forgoing reasons, Commerce has adequately explained how its
methodology is reasonable.
CONCLUSION
For the foregoing reasons, Commerce’s remand results are supported by
substantial evidence and comply with the court’s Order, Oct. 8, 2021, ECF No. 192,
in conformity with the Court of Appeals’ Mandate, Oct. 8, 2021, ECF No. 191, and are
therefore sustained. Judgment will enter accordingly.
/s/ Claire R. Kelly
Claire R. Kelly, Judge
Dated: February 24, 2023
New York, New York