Rudder v. District of Columbia , 890 F. Supp. 23 ( 1995 )

890 F. Supp. 23 (1995)

Beatrice RUDDER, et al., Plaintiffs,
v.
DISTRICT OF COLUMBIA, Defendant.

Civ. No. 92-2881 (CRR).

United States District Court, District of Columbia.

June 26, 1995.

Leroy T. Jenkins, Jr., Derrick A. Humphries, and Marquita Brooks, Washington, DC, for plaintiffs.

Benjamin J. Blustein, Asst. Corp. Counsel, Washington, DC, George C. Valentine, Asst. Corp. Counsel, Washington, DC and Chief, Gen. Litigation Section I, along with Garland Pinkston, Jr., Acting Corp. Counsel, Washington, DC, and Martin L. Grossman, Deputy Corporation Counsel, Civ. Div., for defendant.

Jeremiah A. Collins, of Bredhoff & Kaiser, Washington, DC, for intervenor.

MEMORANDUM OPINION

CHARLES R. RICHEY, District Judge.

INTRODUCTION

The question presented in this case is whether a promotional examination administered in 1991 for the positions of Captain, Lieutenant, and Sergeant in the District of Columbia Fire Department ("DCFD" or "the Department") complied with the requirements of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq. More specifically, as the Court framed the issue prior to trial: "the only thing before the Court is the second examination, whether there was cheating on that test, and ... whether it was administered in a discriminatory manner to the detriment of African American Fire Fighters in violation of their rights under Title 7 of the Civil Rights Act." (Extract of Transcript of Proceedings, May 26, 1995). Counsel for each party agreed, prior to and during trial, that this statement accurately describes the issue before the Court.

A bench trial was held from June 5 to June 8, 1995. At the close of the Plaintiffs' case-in-chief and pursuant to Rule 50 of the Federal Rules of Civil Procedure, the Court granted the Defendant's Motion for Judgment as a Matter of Law with respect to the 1991 Captains' and Lieutenants' Promotional Examinations, but denied it as to the 1991 Sergeants' Examination. (See Order entered June 8, 1995). Following the trial, both sides filed Proposed Findings of Fact and Conclusions of Law as to the remaining issues, and the Court heard oral argument on the merits on June 22, 1995.

Upon careful consideration of the evidence presented at trial, the arguments of counsel, the relevant law, and the entire record in this case, the Court finds that the Plaintiffs have not met their burden of proving that the 1991 examination discriminated against the Plaintiffs on the basis of their race, in violation of Title VII of the Civil Rights Act of 1964. In particular, consistent with the Court's statement of the issue with which all counsel agreed, the Plaintiffs have not shown that the 1991 examination was administered in violation of Title VII or that there was cheating on that test. With respect to the development and validity or "job-relatedness" of the examination, the Court further finds that the Plaintiffs have not met their burden of proving discrimination under Title VII. The Court shall therefore enter judgment in favor of the Defendant District of Columbia.

BACKGROUND

This case arises in connection with two sets of promotional examinations administered in the District of Columbia Fire Department pursuant to a Settlement Agreement and Consent Decree adopted by this Court in the case of Hammon v. Barry, 752 F. Supp. 1087, 1097 (D.D.C.1990). In October of 1992, this Court conducted a trial in the case of Allen et al. v. District of Columbia, Civil Action No. 92-555, to consider the allegations of six firefighters who alleged discrimination in the administration of the December 15, 1990 examination and in the subsequent promotional process. On February 16, 1993, the Court issued a Memorandum Opinion and Order setting forth its findings of fact and conclusions of law in the Allen case and entering judgment in favor of the Defendant.

Thereafter, the instant Plaintiffs filed the above-captioned case asserting that the July 1991 promotional examinations for the positions of Captain, Lieutenant, and Sergeant adversely affected African-American firefighters. (Third Amended Complaint, ¶ 69).^[1] The Plaintiffs are fifty-nine African-American firefighters who first filed this action on December 28, 1992, and then filed an Amended Complaint on January 25, 1993. On March 19, 1993, the Defendants filed a Motion to Dismiss or, in the alternative, for Summary Judgment, and the Plaintiffs filed a response to that Motion. The Court denied the Defendants' Motion, without prejudice, and granted the Plaintiffs' Motion for Leave to File an Amended Complaint. The Court then dismissed the case, without prejudice to the right of the Plaintiffs to re-open this matter by filing an Amended Complaint within 30 days of the issuance of the Supreme Court decision in the case of Landgraf v. USI Film Products, ___ U.S. ___, 114 S. Ct. 1483, 128 L. Ed. 2d 229 (1994), and ordered that the Plaintiffs' failure to file an Amended Complaint or otherwise re-open this matter within that time period would result in the dismissal of this matter, with prejudice. (See Order dated March 8, 1994). On April 16, 1994, the Supreme Court issued its Opinion in Landgraf, and on May 27, 1994, the Plaintiffs filed a Third Amended Complaint.

On June 29, 1994, the Defendants filed a Motion to Dismiss the Third Amended Complaint or, in the alternative, for Summary Judgment, and the Plaintiffs filed an opposition thereto. By Memorandum Opinion and Order entered September 6, 1994, the Court found that the Plaintiffs were barred by the doctrine of res judicata from relitigating claims regarding the 1990 promotional examination, and that the Plaintiffs' claims under 42 U.S.C. § 1981 must be dismissed because the Civil Rights Act of 1991 does not apply to pre-Act conduct. The Court also set a schedule for the preparation of this case for trial.

Following discovery, on March 27, 1995, the Defendant filed a Motion for Partial Summary Judgment and to Strike Jury Demand, and later filed a Motion to Treat as Conceded its Motion for Partial Summary Judgment. The Court granted both Motions by Order entered May 9, 1995. The Plaintiffs, after receiving an extension of time, were to file an opposition to the Defendant's Motion by April 13, 1995. As no opposition was filed, even weeks after its due date, the Court treated the Motion as conceded, pursuant to Local Rule 108(b). The Court's ruling operated as a finding of summary judgment in favor of the Defendant (1) on the Plaintiffs' claims under 42 U.S.C. § 1983, and the Fifth and Fourteenth Amendments to the Constitution, as discovery had not revealed the existence of any unconstitutional municipal custom, policy or practice of discrimination relating to the 1991 examinations (a point which the Plaintiffs conceded in a later pleading^[2]); (2) on the Plaintiffs' jury demand regarding the Title VII claim, as the events at issue occurred prior to the passage of the 1991 Civil Rights Act; and (3) on the claims of those Plaintiffs who did not take the 1991 examinations,^[3] including the lead Plaintiff, Beatrice Rudder, as they were ineligible based on the criteria established by the Court and the Special Master in Hammon.^[4]

After several continuances of the trial date and further delays prompted largely by the Plaintiffs,^[5] the case finally came to trial before the Court on June 5, 1995.

FINDINGS OF FACT

Prior to trial, counsel for each side filed Proposed Findings of Fact and Conclusions of Law, which were marked up by opposing counsel as follows: those portions which are disputed were underlined; those portions which are admitted were bracketed; those portions which are admitted, but deemed irrelevant or immaterial, were left blank. Following trial, counsel for each side filed another set of Proposed Findings of Fact and Conclusions of Law in light of the evidence admitted at trial. These post-trial pleadings were also exchanged amongst counsel and marked up accordingly.^[6]

The Court observes that its findings of fact are largely consistent with the Defendant's Proposed Findings of Fact which, in turn, were drawn in major part from the Declarations of Dr. O'Leary and Dr. Barrett, and from certain exhibits.^[7] The Plaintiffs' marking of the Defendant's Proposed Findings reveals that they dispute many of the findings of fact contained herein.

At some point in the trial, the Court heard the testimony of the Defendant's opinion witnesses, Drs. O'Leary and Barrett, and the Plaintiffs' opinion witness, Dr. Hoffman. As the trier of fact, the Court may accept or reject such opinions in the exercise of its sound discretion and, as is the case with any witness, must judge their credibility. As the Second Circuit observed, "[w]hile courts should draw upon the findings of experts in the field of testing, they should not hesitate to subject these findings to both the scrutiny of reason and the guidance of Congressional intent." Guardians Ass'n v. Civil Service Comm'n of New York, 630 F.2d 79, 89 (2d Cir.1980), cert. denied, 452 U.S. 940, 101 S. Ct. 3083, 69 L. Ed. 2d 954 (1981).

While the Plaintiffs object to much the account of the job analysis and test development provided by Drs. Barrett and O'Leary and set forth in the Defendant's Proposed Findings of Fact, the Court takes this opportunity to observe that there is no probative evidence in the record to refute the testimony of these opinion witnesses. The Court, in short, found Drs. Barrett and O'Leary to be highly credible witnesses who answered difficult questions in what appeared to be an honest way, conceding problems with the examination where apparently necessary. On the other hand, the Court found the Plaintiffs' opinion witness, Dr. Hoffman, extremely biased; at one point, he went so far as to pause before answering a question so as to respond in a way that was consistent with "his burden." As the Court stated from the bench, Dr. Hoffman's responsibility, like any witness, was to answer questions honestly and accurately — not to act as an advocate by interjecting during the Court's colloquies with counsel and arguing with the Court over the law. Moreover, the Court is troubled by Dr. Hoffman's willingness to accept the test results where they showed adverse impact, but reject the results where they showed no adverse impact. Under such circumstances, the Court gives very little weight to Dr. Hoffman's testimony. Indeed, in light of his readily apparent and extreme bias, much of Dr. Hoffman's representation of the "facts" is entitled little more weight than would be afforded an attorney's words at closing argument which, as this Court has told juries for years, is not evidence in a case.

With that said, the Court shall proceed to set forth its findings of fact.

A. The Test Development Committee

The Settlement Agreement in Hammon v. Barry, which was executed on August 20, 1990 following over six years of litigation regarding charges of discrimination and reverse discrimination in the District of Columbia Fire Department, was composed of two parts: Part A addressed all claims involving hiring and vestiges of discrimination and Part B addressed the issue of future promotions. Hammon v. Barry, 752 F. Supp. 1087, 1090-91 (D.D.C.1990). By agreement of all the parties to the Hammon litigation, a Test Development Committee ("TDC") was established to work with the Hammon Special Master, Professor Stephen A. Saltzburg, in developing the promotional examinations required by the Settlement Agreement. Id. at 1091, 1091 n. 1. The members of the TDC were Dr. Richard Barrett, Dr. Lawrence O'Leary (both of whom testified on the Defendant's behalf in this case), and Dr. Carl Holmes.

On June 19, 1991, following the development and administration of the 1990 examination, the Hammon Special Master ordered that "the City shall use the TDC to prepare the second round of promotional examinations." (June 19, 1991 Order Regarding TDC). In particular, the Special Master charged the TDC with the following duties:

drafting of all examination exercises, training of graders if graders are to be used, preparing a report on the examination when the results are known, doing statistical or other analyses as may be required to validate the tests and provide details for a report, defending the tests if they are attacked, and responding to the City's reasonable requests for information and explanations concerning the examinations.

Id. at 2. The Special Master further ordered that "[t]he City, particularly the District of Columbia Office of Personnel, shall be responsible for administering the examinations...." Id. The Special Master also deemed the TDC "responsible for examinations that are legally sufficient" and ordered the TDC to "be prepared to defend the examinations which it prepares." Id. at 3.

Thereafter, the TDC drafted examinations for three ranks in fire suppression: Sergeant, Lieutenant, and Captain. (Barrett Decl., ¶ 1). The TDC also observed administration of the examinations, participated in the development of scoring keys and grading, and issued a final report and validation study on November 13, 1991. (Id.; Defendant's Exh. 2, Fire Department Promotion Test Battery: Development and Validation for Washington, D.C., Second Administration, July, August 1991, dated November 12, 1991 [hereinafter "Validation Study"]). As discussed in the Validation Study, the TDC concluded that the 1991 examination was valid under the "content validation" method set forth in the Uniform Guidelines on Employee Selection Procedures ("Uniform Guidelines"), 29 C.F.R. Ch. XIV. (Barrett Decl., ¶ 9).

The Court further observes that, as part of the Settlement Agreement — which was signed by the Hammon Plaintiffs' attorney, Joan Burt — $3,500,000.00 was distributed to members of the Plaintiff class. The vast majority of the instant Plaintiffs received monetary compensation as a result of said Agreement. In particular, the Court has examined the sealed materials in Hammon and determined that at least forty-four of the fifty-nine Plaintiffs in the case at bar (and perhaps more by virtue of the similarity of names as between Hammon Plaintiffs and Rudder Plaintiffs) received money from the District of Columbia as part of the Settlement Agreement in Hammon. Thus, very few of the instant Plaintiffs did not receive financial benefit from the Hammon settlement. Moreover, the Court notes that none of the Rudder Plaintiffs would have had an opportunity to take the 1990 and/or 1991 examinations were it not for the benefit they received from the settlement as Hammon class members. Nevertheless, they still filed the instant suit.

B. Components of the 1991 Examination

The 1991 Fire Department promotional examinations were comprised of three parts: (1) the Job Knowledge Test; (2) the Assessment Center; and (3) the Fire Scene Simulation. (O'Leary Decl., ¶ 3).

The Job Knowledge Test was a multiple-choice test of knowledge based on manuals, orders and other material assigned on a reading list. (O'Leary Decl., ¶ 3). TDC member Dr. O'Leary was responsible for developing the Job Knowledge Test, in collaboration with Dr. Barrett, also a TDC member. (Id.)

The Assessment Center was comprised of exercises designed to test for behavioral characteristics essential to Fire Department officers, such as leadership, judgment, decisiveness, and communication skills. (Id., ¶ 4). Dr. O'Leary developed the Assessment Center.

The third portion of the examination, the Fire Scene Simulation, was developed by TDC member Dr. Carl Holmes and reviewed by Drs. Barrett and O'Leary. Two fire scene scenarios were administered as part of the 1991 examination, an engine scene and a truck company scene. (Id., ¶ 5). Each scene presented a drawing of a fire scene and the surroundings, and quotations from the dispatcher. The applicants responded to a series of questions in which they wrote down the decisions that they would make and orders that they would give on the scene, based upon the information presented. (Id.)

C. The TDC's Job Analysis

To develop the 1990 and 1991 examinations, the TDC conducted job analyses of the Sergeant, Lieutenant, and Captain positions. The TDC performed its own job analysis because it determined that the Fire Department's existing position descriptions were inadequate for its purpose. Therefore, in 1987 the TDC conducted interviews with a biracial group of approximately 20 D.C. Fire Department Officers. (O'Leary Decl., ¶ 7). As a result of these interviews, a nine-page narrative entitled "Tasks of Fire Sergeant, Lieutenant, Captain in Fire Suppression," was generated in November 1987. (Id.; Defendant's Exh. 3).

Following these interviews, a Job Analysis Questionnaire was distributed to a larger sample of 54 officers. (Defendant's Exh. 5). These officers were also racially diverse. The purpose of the Questionnaire was to quantify and verify the accuracy of the initial data obtained from the interviews. The Questionnaire allowed a broader sample of officers the opportunity to examine the initial data and correct any inaccuracies regarding their job duties. (O'Leary Decl. ¶ 8; Barrett Decl., ¶ 14). The results of the Job Analysis Questionnaire are summarized at pages 18-21 of the November 1991 Validation Study. (Defendant's Exh. 2).

The TDC determined from the Questionnaire and the interviews with the officers that the three positions at issue — Sergeant, Lieutenant and Captain — were essentially the same in the responsibilities at the fire scene. The only significant differences are that, if there are two officers of different rank present, the one with the higher rank takes charge until the Battalion Chief arrives. With respect to fire house duties, Captains have some administrative duties not shared with the Sergeants and Lieutenants, such as sitting on disciplinary panels. (O'Leary Decl., ¶ 9).^[8]

The TDC further determined that there were basic knowledge areas that were important and common to all three ranks, and determined that the same Job Knowledge Tests would be administered to all three ranks. However, Dr. Holmes created two fire scene scenarios for each of the three ranks. (O'Leary Decl., ¶ 10 (as amended by the witness at trial)).

Although the job analysis was begun in 1987, prior to using the results of the job analysis to develop the 1991 tests, the TDC interviewed officials of the Fire Department who confirmed that the equipment, procedures, and policies of the Fire Department had not changed in the few years since the job analysis was performed in any way that would preclude use of the job analysis. (Barrett Decl., ¶ 5).

D. Development of the Job Knowledge Test

The TDC's goal in conducting a job analysis was to determine what knowledge, skills, and abilities are necessary to function effectively in the D.C. Fire Department. The TDC incorporated its findings into the multiple-choice Job Knowledge Test. This test was based on the principle that those who can recognize the correct answer from among four supplied on a multiple choice of job knowledge are likely to be the better performers in the Fire Department. (O'Leary Decl., ¶ 11).

The development of the Job Knowledge Test proceeded in six phases. In Phase I, Dr. O'Leary met with a bi-racial group of 20 Sergeants, Lieutenants and Captains who served as Subject Matter Experts (SME's). Together they reviewed the Department's manuals and standard operating procedures, and pinpointed those areas which are particularly relevant to their jobs. (O'Leary Decl., ¶ 12). The results of these meetings are summarized in the "Written Test Question Development Process." (Defendant's Exh. 6). In addition, these meetings are summarized in Appendix 3 of the November 1991 Validation Study, entitled "Development of the Job Knowledge Test." (Defendant's Exh. 2).

In Phase II of the development of the multiple choice exam, Dr. O'Leary met with other groups of DCFD officers over a several day period in April 1989 in order to isolate further the exact written material that was important for officers to know. In addition, he visited fire houses to talk to Sergeants, Lieutenants and Captains about managerial and administrative tasks performed at the fire house. (O'Leary Decl., ¶ 13). (See Defendant's Exh. 2, Validation Study at Appendix 3 ("Development of the Job Knowledge Test")).

In Phase III, another racially diverse group of 24 Sergeants, Lieutenants and Captains reviewed the job knowledge areas previously identified by the Subject Matter Experts, adding or deleting areas as appropriate. In addition, these 24 officers were asked to weigh the various knowledge areas by spreading 100 points across the knowledge areas commensurate with his or her opinion of its relative importance. Thus, each participating officer gave a weight to a particular aspect of his job, such as Fire Scene, Rescue, and HazMat. (O'Leary Decl., ¶ 14). The results of this process are summarized in the "Job Knowledge Weighting Form," which is attached as Appendix 1 to the Validation Study. (Defendant's Exh. 2).

In developing the Job Knowledge Test, Dr. O'Leary took into account the relative weights that the officers had given to the different knowledge areas. Proportionately more exam questions were created for those knowledge areas which the SME's had deemed particularly important. For example, if a particular job knowledge area received 28% of the points, an attempt was made to come as close as possible to having 28% of the questions on the test originate from that area. This is shown in the Job Knowledge Test Plan, which is attached as Appendix 8 to the Validation Study. (Defendant's Exh. 2). Although there was some variation in the weights assigned by the individual officers, there were a sufficiently large number of participating officers to avoid undue arbitrariness. (O'Leary Decl., ¶ 15).

In Phase IV, Dr. O'Leary prepared the multiple choice questions. He was assisted by a bi-racial group of fire department officers and test question writers who were all located over 500 miles from Washington, D.C. (O'Leary Decl., ¶ 16).

During Phase V, Dr. Barrett analyzed the completed questions item-by-item over a period of months. Any "defective" questions were either replaced or modified. (O'Leary Decl., ¶ 17; Barrett Decl., ¶¶ 17-19).

Upon completion of this editing process, the questions were again reviewed by a racially mixed group of Subject Matter Experts from the D.C. Fire Department and the D.C. Office of Personnel. Any question judged poor by these SME's was discarded. These SME's were Deputy Chief Everett Green (Director of the Fire Academy), Deputy Chief Richard Clark (Director of Special Projects), and four Battalion Fire Chiefs. In July 1991, Drs. Barrett and O'Leary again met with the SME's to further refine the content validity of the questions by verifying that each question met the criterion of a psychometrically sound job knowledge multiple choice question. (O'Leary Decl., ¶ 18; Barrett Decl., ¶¶ 17-19). Using a "Content Validation Form," the officials responded to specific questions regarding the content validity of each item. (See Defendant's Exh. 2, Validation Study at Appendix 2 ("Content Validation Form")).

E. Development of the Assessment Center, and Training of Assessors

Dr. O'Leary also developed the Assessment Center portion of the examination. The Assessment Center is designed to test for behavioral characteristics essential to fire department officers. These characteristics are referred to as "dimensions." The Assessment Center is based upon the premise that those who demonstrate proficiency in simulated work activities during the selection procedure are likely to exhibit these behavioral characteristics on the job. A list of the dimensions is set forth in Appendix 9 of the Validation Study. (Defendant's Exh. 2). They include: dealing with people, decisiveness, judgment, leadership, planning and organization, verbal communication, and written communication. (O'Leary Decl., ¶ 21).

All three ranks participated in a Leaderless Group Discussion, designed to test several dimensions such as leadership and dealing with people. In this exercise, a group of six candidates met as a simulated committee with the responsibility for setting priorities for forty-three different work behaviors related to the job of company officer. Each group was observed by three assessors; thus, each assessor was responsible for observing two of the candidates. (O'Leary Decl., ¶ 22). Dr. O'Leary testified at trial that the 2:1 ratio of candidate to assessor is consistent with professional standards.

The assessors were a racially balanced group of Fire Captains or higher-ranking officials recruited from fire departments across the United States and Canada. Dr. O'Leary had previously worked with a number of the assessors, and testified that he knew them to have proven track records. (O'Leary Decl., ¶ 26). The assessors are listed in Appendix 6 of the Validation Study. (Defendant's Exh. 2).

Prior to the Assessment Center exercises, the assessors had participated in an intensive two-day training workshop. According to Dr. O'Leary, the training received by the assessors exceeded the levels established by the International Congress on Assessment Centers. During this workshop, the assessors were trained in the principles of assessing and were familiarized with the organization and management procedures of the D.C. Fire Department, and the duties of the Sergeant, Lieutenant and Captain positions. In addition, the assessors saw videotapes of sample exercises, and completed numerous practice exercises, during which their scoring was analyzed and critiqued in order to ensure consistent scoring among the assessors. The assessors were carefully reviewed during the training sessions. If any assessors demonstrated an inability to grasp the concepts or perform effectively, they were removed. (O'Leary Decl., ¶ 27).

During the workshop, assessors were trained to avoid the "halo effect," which may cause an assessor to award points for one dimension based upon a candidate's score for another dimension. The halo effect, for example, may cause an assessor to award a candidate more points for "judgment" than is warranted because the candidate received a high score for "leadership." The assessors were trained to recognize and avoid this phenomena. (O'Leary Decl., ¶ 28).

F. Development of Fire Scene Simulation

The Fire Scene Simulation ("FSS") was developed by Dr. Holmes and his staff, based on information gathered during the job analysis. This information included the frequency of different types of fires, procedures used in responding to emergency calls, the types of equipment used (e.g., size of hose), and communications procedures. (Defendant's Exh. 2, Validation Study).

The FSS is based on the premise that those who can determine how to attack a fire presented in the drawings, augmented by descriptions of the dispatcher's instruction, and local conditions such as weather and time of day, are likely to know how to attack a fire in real life. Moreover, although the answers to the FSS were written and not orally issued as on a fire scene, TDC member Dr. Barrett compared the results of scoring written notes with analysis of tapes into which the applicants had dictated their responses. The scores were nearly identical. (Defendant's Exh. 2, Validation Study, p. 26).

G. Administration of the Examination

Due to security concerns, the exam was not pre-tested. Such a pre-testing would require administration to a relatively large population of individuals, and the Plaintiff's opinion witness Dr. Hoffman conceded on cross-examination that pre-testing is not required under the Uniform Guidelines. Instead, the exam was reviewed by the SME's prior to administration. (O'Leary Decl., ¶ 19). Afterwards, the TDC sent the examinations directly to the D.C. Office of Personnel ("DCOP") for copying, where they were maintained under lock and key prior the administration. The DCOP maintained all three portions of the test: the multiple choice Job Knowledge Test, the Fire Scene Simulation, and the Assessment Center materials. (Smith Decl., ¶ 5).

The Job Knowledge and Fire Scene Simulation portions of the examination were administered at the same time to all candidates for all ranks. The administration took place in a large convention room at the D.C. Convention Center on July 29, 1991. Upon entering the test site, the candidates were required to present photo identification. At the request of DCOP, top Fire Department officials and Battalion Fire Chiefs were present to maintain order and security on the floor while the test was being administered. In addition, the three TDC members were present. DCOP personnel were also present to administer the exams. (Smith Decl., ¶ 6).

The Assessment Center took place over several days at the Embassy Suites Hotel in Washington, D.C. The candidates arrived at the hotel at their previously scheduled time, and were escorted to rooms where their exercises took place. (O'Leary Decl., ¶ 29).

No problems relating to administration were brought to the attention of TDC members Barrett and O'Leary. (O'Leary Decl. ¶ 29; Barrett Decl. ¶ 21). In addition, DCOP coordinator Billie Smith did not observe any problems relating to the administration at the Convention Center, and no problems were brought to her attention by candidates, monitors, or the TDC members. (Smith Decl., ¶ 7).

H. Grading

Immediately after the administration, Smith took the examination booklets and answer sheets to the DCOP test room, where they were maintained under lock and key. The multiple choice section of the examination was graded by employees of the Metropolitan Police Department, who scanned the multiple choice section with a mechanical scanner. The scanner scans the answers onto a computer file, which is then copied onto a computer diskette. The scoring key for the multiple choice section was provided by the TDC. The DCOP had no input into the development of the scoring key. (Smith Decl., ¶ 8).

The grading of the Fire Scene Scenarios occurred during a two week period in August 1991. The grading took place in a classroom set up in Building T-3. All the graders were present in the same classroom during the grading process. (Arthur Decl., ¶ 5; Smith Decl., ¶ 9).

The majority of the Battalion Fire Chiefs (BFC's) in the D.C. Fire Department were used as graders for the 1991 Fire Scene Simulation. There were approximately forty BFC's at that time, and approximately thirty to thirty-three of them graded the fire scene scenarios. (Arthur Decl., ¶ 6). Several African-American Battalion Fire Chiefs assisted in grading the 1991 Fire Scene Simulation. (Arthur Decl., ¶ 7).

For security purposes, a double-blind procedure was used for grading the Fire Scene Simulation booklets. This procedure ensured that the DCFD officials who graded the examination did not know the race or identity of the candidate whose booklet was being graded. This procedure was followed at the insistence of the FSS developer, Dr. Carl Holmes, who routinely uses this method in other jurisdictions to ensure that persons scoring the test booklets have no way of knowing whose booklets they are scoring. (Smith Decl. ¶ 10). The DCOP was responsible for inserting a second "conversion" identification number on the FSS booklets, in addition to the original identification number which had been inserted onto the booklets by the candidates. A record was made of the two identification numbers and maintained under strict security by the DCOP. The original identification number was then removed from the booklets. (Id.)

Prior to receiving the FSS examination booklets, graders received instructions from Dr. Holmes. Dr. Holmes met with the BFC's as a group to go over the answer key. As a group, they reviewed the two FSS questions, and Holmes asked the BFC's what they thought the correct responses were. (Arthur Decl., ¶ 9).

The answers given by the BFC's were based upon information in the Fire Department study materials, regulations, and procedures. Dr. Holmes wrote down the suggested answers on the blackboard, and this information was typed up and became the grading key. (Arthur Decl., ¶ 11).

Each BFC who participated in the grading of the FSS was given his own official grading key. Dr. Holmes instructed them to make a few modifications or clarifications to the grading key. These modifications were written on the blackboard, and the graders copied them down on their grading keys. The information remained on the blackboard the entire time they were grading. The graders could either refer to the blackboard or to their own grading key. (Arthur Decl., ¶ 12). Although the graders may have written notes on their individual scoring keys, there is no evidence that substantively "different" scoring keys were generated or used. Rather, all the changes to the grading key were authorized by Dr. Holmes. (Arthur Decl., ¶ 13). No evidence was presented that graders made any unauthorized changes to the grading key.

Two BFC's worked as a team to grade a fire scene. One person read aloud the answer key, and the other person graded the exam booklet. Then the grade was recorded on a Rating Sheet. The graders awarded an answer a certain amount of points based upon the quality of the answer. If an examination answer contained all the required information and was completely correct, it was to be given full credit. If there was disagreement between the two graders about whether the answer was correct, or what score to give, the two graders discussed it between themselves and resolved it. (Arthur Decl., ¶¶ 14-16).

The DCOP maintained custody of the examinations at night. Every morning a DCOP official would bring the examinations to the Training Academy and give them to either Chief McCaffrey or Shelton Arthur. In the evening, they were returned to DCOP. The examinations were handed out to the graders by either Chief McCaffrey or Shelton Arthur. The exams were handed out from the top of the pile, and no graders were permitted to choose which examination to grade. (Arthur Decl., ¶¶ 18-19).

I. Results of the Examinations

1) Job Knowledge

The results of the Job Knowledge Test were close as between African-American firefighters and white firefighters. At the Captain's level, the mean score of whites was only slightly higher than that of African-Americans (African-American: 62.58; White: 64.48). At the Lieutenant's rank, African-Americans scored higher than whites (African-American: 61.57; White: 61.12). At the Sergeant's rank, the mean score of whites was higher than African-American candidates (African-American: 47.96; White: 54.88). (O'Leary Decl., ¶ 31). (Defendant's Exh. 2, Validation Study at Table 7, p. 33).

2) Assessment Center

The results of the Assessment Center demonstrate that the scores of African-American and white candidates are also very close at each rank. At the Captain rank, the mean score of African-American candidates was slightly higher than white candidates (African American: 34.24; White: 34.06). Similarly, African-American candidates scored higher than white candidates at the Lieutenant level (African-American: 36.71; White: 33.88). At the Sergeant level, the means score of whites was slightly higher than the mean score of African-American candidates (African-American: 21.48; White: 22.80). (O'Leary Decl. ¶ 30). These results are printed at Table 7 of the Validation study, page 33. (Defendant's Exh. 2).

3) Fire Scene Simulation

On the Fire Scene Simulation, the mean score of African-American candidates was lower than white candidates at each rank (Captain: African-American: 47.33; White: 55.98; Lieutenant: African-American: 48.57; White: 53.32; Sergeant: African-American: 37.71; White: 47.06). (Defendant's Exh. 2, Validation Study at Table 7, p. 33). The Fire Scene Simulation produced more adverse impact than the other tests for all ranks but, according to Dr. Barrett, less adverse impact than the traditional multiple-choice test. (Barrett Decl., ¶ 27).

The number of applicants of each race for each position are presented in Table 6 of the Validation Report of November 13, 1991, and the test results of African-American and white candidates are presented in Table 7 of the Validation Report. (Defendant's Exh. 2). According to Dr. Barrett, the scores on all of the tests developed by the TDC have substantially lower average differences between African-American and white candidates that is typical of multiple choice tests. (Barrett Decl., ¶ 27).

The opinion witnesses for both the Plaintiffs and the Defendant agreed that there was overall no adverse impact upon African-Americans at the Captain rank, because the selection rate for African-Americans exceeded the selection rate for white candidates. Moreover, because all candidates taking the Lieutenants' exam were promoted, there was no adverse impact upon African-Americans at the Lieutenant rank.

J. Weighting of the Exam Components

Prior to the administration of the examination, the TDC assigned weights to the three examination components based upon a review of the job analysis and the competencies required for the positions. The TDC determined that the Assessment Center would be given a weight of 60%, and the Job Knowledge Test and Fire Scene Simulation would each receive a weight of 15%. Thus, the ratio of weight for cognitive skills (measured by the Job Knowledge Test and the Fire Scene) to the supervisory skills (measured by the Assessment Center) was 2:3. (O'Leary Decl., ¶ 32).

While the Plaintiffs' opinion witness, Dr. Hoffman, testified that the weighting of the examination was "arbitrary," the Court credits Dr. Barrett's testimony that the TDC decided to weight the examination components in this manner for several reasons. First, multiple-choice tests have a long record of adverse impact. Nevertheless, it is relevant to test for knowledge of some specific rules and procedures that the applicants can be expected to know. Second, the Assessment Center was given additional weight because it taps skills of managing subordinates, and the ability to make some administrative decisions. (Barrett Decl., ¶ 29). The 2:3 ratio of cognitive (Job Knowledge Test and Fire Scene Simulation) to managerial abilities reflects the TDC's assessment of the relative importance of the two categories of skills for D.C. Fire Department officers. (O'Leary Decl., ¶ 47). Finally, the test results set forth in Table 7 of the Validation Study support the decision of the TDC to place more emphasis on the Assessment Center than on the other tests as a means of reducing adverse impact without compromising the validity of the test battery. (Barrett Decl., ¶ 27).

K. Alleged "Cheating Sheet"

At trial, the Plaintiffs presented a handwritten two-page document which allegedly related to the multiple-choice portion of the 1991 promotional examination. The document is unreadable in substantial part, and no evidence was presented regarding who wrote the document. More importantly, no specific evidence was presented showing that this document, in fact, related to the 1991 examination, and no evidence was presented showing that the document was generated before that test. Moreover, no evidence was presented showing that any firefighters, either African-American or white, saw the document before or during the examination, or that anyone achieved an unfair advantage by reliance on the document. Indeed, several witnesses testified that after examinations are administered, study groups convene to discuss the examination questions and the correct answers. Thus, the document very likely could have been developed in connection with a post-exam study group, if it was indeed related to the 1991 examination at all.

The Plaintiffs presented three witnesses who alleged that Lieutenant (then Firefighter) Michael Willis (African-American) either showed them or informed them about this document. However, Lieutenant Willis testified that he had never seen this document until Plaintiffs' counsel showed it to him on the first day of trial. (Willis Decl., ¶ 9). Lieutenant Willis further testified that following the 1990 examination, rather than the 1991 examination, he had been apprised of the existence of a document apparently relating to the earlier examination. (Id., ¶ 5). In any event, no evidence whatsoever was presented showing that the document in question related to any cheating scheme.

CONCLUSIONS OF LAW

Title VII authorizes the use of "any professionally developed ability test provided that such test, its administration or action upon the results is not designed, intended or used to discriminate...." 42 U.S.C. 2000e-2(h).^[9] "[T]he accepted procedure for Title VII cases is to require a plaintiff to establish a prima facie case, and then to require the defendants to rebut this showing with proof that the test was legitimately job-related." Guardians Ass'n v. Civil Service Comm'n of New York, 630 F.2d 79, 88 (2d Cir.1980), cert. denied, 452 U.S. 940, 101 S. Ct. 3083, 69 L. Ed. 2d 954 (1981). As set forth in detail below, the question of whether the Plaintiffs established a prima facie case regarding the Sergeants' exam is not disputed, and the Court has already ruled that the Plaintiffs failed to meet their initial burden with respect to the Captains' and Lieutenants' examinations. "The real issue in this case, therefore, is whether the defendants have rebutted the plaintiffs' prima facie case by proving that its test was job-related: that the test accurately selected applicants who would be better [Sergeants in the DCFD]." Guardians Ass'n, 630 F.2d at 88.

A. THE PLAINTIFFS ESTABLISHED A PRIMA FACIE CASE OF DISPARATE IMPACT WITH RESPECT TO THE 1991 SERGEANTS' EXAMINATION, BUT NOT AS TO THE LIEUTENANTS' AND CAPTAINS' EXAMINATIONS, OR AS TO THE ASSESSMENT CENTER

In order to prove disparate impact under Title VII, the Plaintiffs must first make out a prima facie case of discrimination, i.e., show that "the tests in question select applicants for hire or promotion in a racial pattern significantly different from that of the pool of applicants." Albemarle Paper Co. v. Moody, 422 U.S. 405, 425, 95 S. Ct. 2362, 2375, 45 L. Ed. 2d 280 (1975). See also Wards Cove Packing Co. v. Atonio, 490 U.S. 642, 650, 109 S. Ct. 2115, 2121, 104 L. Ed. 2d 733 (1989) ("The ``proper comparison [is] between the racial composition of [the at-issue jobs] and the racial composition of the qualified ... population in the relevant labor market.'") (quoting Hazelwood School Dist. v. U.S., 433 U.S. 299, 308, 97 S. Ct. 2736, 2742, 53 L. Ed. 2d 768 (1977).

1) The 1991 Lieutenants' and Captains' Promotional Examinations

As the Court ruled from the bench on June 6, 1995 and by Order of June 8, 1995, the Plaintiffs failed to meet their burden of establishing a prima facie case of discrimination with respect to the 1991 Lieutenants' and Captains' promotional examinations. At trial, the Plaintiffs' own opinion witness could not point to any evidence that there was adverse impact resulting from the 1991 Lieutenants' or Captains' examinations. (See Plaintiffs' Exh. 63, Report of Dr. Hoffman, p. 10 ("There are insufficient numbers of candidates to estimate adverse impact on the Lieutenant and Captain Examinations.")).

In addition, Plaintiffs' counsel admitted at trial that none of the Plaintiffs took the 1991 Lieutenants' examination, and the Plaintiffs' opinion witness conceded that there was no evidence of adverse impact from that examination. (See also Defendant's Exh. 2, Validation Study, pp. 5, 35, 37 ("Since it is anticipated that all the applicants for Lieutenant will be appointed, the test would have no [adverse] effect on this group.")).

With respect to the Captains' examination, the Validation Study shows the impact ratio for African-Americans to be equal to or one and one-half times greater than the impact ratio for Caucasian candidates. (Id. at Table 8, p. 34) In other words, for any ten possible promotions, four out of seventeen African-American Captain applicants were promoted as a result of the 1991 Captains' examination as compared to six out of thirty-six Caucasian applicants. Accordingly, the Court finds that the Plaintiffs failed to meet their burden of establishing a prima facie case of discrimination with respect to the 1991 Captains' examination as well. Wards Cove, 490 U.S. at 650, 109 S. Ct. at 2121.^[10]

2) The Assessment Center

It is undisputed that the Assessment Center produced no disparate impact on African-Americans. Indeed, African Americans scored higher than their white colleagues on this portion of the examination. Accordingly, the Court finds that the Plaintiffs have failed to establish a prima facie case of discrimination with respect to the Assessment Center portion of the 1991 examination.^[11]

3) The 1991 Sergeants' Examination

The Court further finds that, as the Defendant's own opinion witnesses conceded, the 1991 Sergeants' examination resulted in an adverse impact on African-American firefighters, thus establishing a prima facie case of discrimination as to that exam. The Validation Study shows the impact ratio for African-American candidates to be between 28% to 36% of that of white candidates. (Defendant's Exh. 2, Validation Study at Table 9, p. 36). Thus, if ten applicants were selected pursuant to these numbers, two out of 143 African-American candidates would be promoted as a result of the 1991 Sergeants' examination as compared to six out of 120 white candidates. Accordingly, the Court finds that the Plaintiffs succeeded in meeting their burden of establishing a prima facie case of discrimination by showing that a specific employment practice (the 1991 Sergeants' examination) caused an adverse impact on African-American candidates. See Wards Cove, 490 U.S. at 657, 109 S. Ct. at 2124. As "Congress did not intend that the standard used in interpreting Title VII would reject every test with a disparate racial impact," however, the inquiry next becomes whether the Defendant produced evidence showing that the test was valid. Guardians Ass'n, 630 F.2d at 89-90.

B. WHILE THE DEFENDANT PRODUCED AMPLE EVIDENCE THAT THE 1991 SERGEANTS' EXAMINATION WAS VALID AND JOB-RELATED, THE PLAINTIFFS FAILED TO MEET THEIR BURDEN OF SHOWING THAT THE EXAM WAS INVALID UNDER TITLE VII

"Upon demonstration by a plaintiff of a prima facie case of employment discrimination in violation of Title VII's prohibition of selection devices which adversely impact a protected minority, the defendant has an opportunity to demonstrate that the selection device has ``a manifest relationship to the employment in question.'" Frazier v. Consolidated Rail Corp., 851 F.2d 1447, 1454 (D.C.Cir.1988) (quoting Connecticut v. Teal, 457 U.S. 440, 446-47, 102 S. Ct. 2525, 2530, 73 L. Ed. 2d 130 (1982)). "Put another way, the employer may demonstrate that the selection device is ``valid.'" Id. See also McDonnell Douglas Corp. v. Green, 411 U.S. 792, 803, 93 S. Ct. 1817, 1824, 36 L. Ed. 2d 668 (1973); Griggs v. Duke Power Co., 401 U.S. 424, 431, 91 S. Ct. 849, 853, 28 L. Ed. 2d 158 (1971). While the employer has the burden of producing evidence of a business justification, however, the burden of persuasion remains at all times with the plaintiff.^[12]Id.; Watson v. Fort Worth Bank & Trust, 487 U.S. 977, 997, 108 S. Ct. 2777, 2790, 101 L. Ed. 2d 827 (1988) ("[T]he ultimate burden of proving that discrimination against a protected group has been caused by a specific employment practice remains with the plaintiff at all times.").

The employment practice at issue in this case is a promotional exam. Under Title VII disparate impact analysis, an employment test with an adverse impact on racial minorities is prohibited unless the test is "demonstrably a measure of job performance." Griggs, 401 U.S. at 436, 91 S. Ct. at 856. Thus, in a case involving examinations, the business justification that the employer must show is that the test is job-related. See Albemarle Paper Co. v. Moody, 422 U.S. 405, 95 S. Ct. 2362, 45 L. Ed. 2d 280 (1975). The Equal Employment Opportunity Commission ("EEOC") has adopted specific guidelines that define "job-relatedness" and require validation according to professionally developed testing standards for employment tests with an adverse impact. See Uniform Guidelines on Employment Selection Procedures, 29 C.F.R. Ch. XIV. ("Uniform Guidelines"); Albemarle Paper Co., 422 U.S. 405, 95 S. Ct. 2362, 45 L. Ed. 2d 280 (1975); Griggs, 401 U.S. at 436, 91 S. Ct. at 856 (EEOC's construction of § 703(h) to require that employment tests be job-related comports with congressional intent).

Under the Uniform Guidelines and applicable law, the Court finds that the Defendant produced more than sufficient evidence that the 1991 examination was valid and job-related and, in turn, the Court finds that the Plaintiffs failed to meet their ultimate burden of proof on this issue.

1) Validity of the Examination Under the Uniform Guidelines

Dr. Hoffman testified in detail on the Plaintiffs' behalf regarding the ways in which, in his view, the 1991 examination was invalid under the Uniform Guidelines. As a preliminary matter, the Court observes that, while "the Guidelines should always be considered ... they should not be regarded as conclusive," because the Supreme Court "has not ruled that every deviation from any of the Guidelines automatically results in a violation of Title VII." Guardians Ass'n, 630 F.2d at 91. Rather, as the Second Circuit explained, "[t]he [Supreme] Court appears to have applied the Guidelines only to the extent that they are useful, in the particular setting of the case under consideration, for advancing the basis purposes of Title VII." Id. Bearing in mind these principles, the Court acknowledges Dr. Hoffman's identification of arguably imperfect compliance with Guidelines, but emphasizes that it is not clear that strict adherence to the "intricate details" of the Guidelines is mandatory under Title VII. Id. Rather, "the dispositive issue is whether the challenged practice serves, in a significant way, the legitimate employment goals of the employer." Wards Cove, 490 U.S. at 659, 109 S. Ct. at 2126. "The touchstone of this inquiry is a reasoned review of the employer's justification...." Id.

In any event, the Court finds that the 1991 examination meets the requirements of the Uniform Guidelines. For the purposes of satisfying the Uniform Guidelines, an employer may rely upon content validity studies, criterion validity studies, or construct validity studies. 29 C.F.R. § 1607.5(A). Selection procedures which purport to measure knowledge, skills and abilities, such as the examination at issue here, may be validated by a content validity study. Id. § 1607.14(C)(1). Moreover, pursuant to the Uniform Guidelines, a selection procedure may be validated under the content validation method even though adverse impact may exist. Id. § 1607.3(A).

The content validity method was utilized by the TDC in reaching its conclusion that the 1991 examination was valid. (See generally Defendant's Exh. 2, Validation Report). Although the Uniform Guidelines provide for alternate validation strategies, primary reliance was placed on content validity by the TDC because in some samples there were too few subjects to produce stable results for criterion-related validity. In addition, the TDC chose not to use criterion validity because it determined that the divisiveness in the Fire Department made it unwise to rely heavily on the performance ratings of DCFD officials. (Barrett Decl., ¶ 10).

a) Adequacy of the Job Analysis

Evidence of the validity of a test by a content validity study should consist of data showing that the content of the test is representative of important aspects of performance on the job for which the candidates are to be evaluated. 29 C.F.R. § 1607.5(B). In other words, "[t]he central requirement of Title VII [is the] relationship of test content to job content...." Guardians Ass'n, 630 F.2d at 97. Thus, under the Guidelines, "[t]here should be a job analysis which includes an analysis of the important work behavior(s) required for successful performance and their relative importance...." 29 C.F.R. § 1607.14(C)(2).

As set forth in the Court's findings of fact, the TDC performed an extensive job analysis of the important work behaviors of the Captain, Lieutenant and Sergeant positions. (See Defendant's Exh. 2, Validation Study, pp. 11-21). From the interviews with bi-racial groups of Fire Department personnel and the Job Questionnaire, the TDC determined what knowledge, skills, and ability areas are necessary to function effectively in the D.C. Fire Department. (O'Leary Decl., ¶¶ 7-10; Defendant's Exh. 2, Validation Study, pp. 13-21 ("Tasks of Fire Sergeant, Lieutenant, Captain in Fire Suppression")). Accordingly, the Court finds that the job analysis conducted in preparation for the 1991 examination meets the standards set in the Uniform Guidelines, 29 C.F.R. § 15(C)(3).

Moreover, the criticisms of the job analysis by Plaintiffs' opinion witness, Dr. Paul Hoffman, do not convince the Court that the test is invalid. In his critique, Dr. Hoffman states that, to the extent the District did not provide necessary cooperation or personnel to the TDC, the quality of the exam might have suffered. (Hoffman Report, p. 4 (attached to Plaintiffs' Exh. 63, Hoffman Decl.)). However, the evidence shows that the District provided the required cooperation and administrative back-up to the TDC, and Dr. Hoffman has identified no specific defects in this regard. (O'Leary Decl., ¶ 36).

Dr. Hoffman next states that the TDC's job analysis did not specifically inventory the skills required to perform each task. (Hoffman Report, p. 4). Again, the Court cannot credit this testimony. As noted above, the TDC conducted extensive interviews with DCFD officers and compiled a nine-page narrative entitled "Tasks of Fire Sergeant, Lieutenant, Captain in Fire Suppression" in November 1987. (Defendant's Exh. 3). Following these interviews, a Job Analysis Questionnaire was distributed to a larger sample of fifty-four racially-diverse officers, in order to verify the accuracy of the initial data obtained from the interviews. (Defendant's Exh. 5; O'Leary Decl., ¶ 37).

Moreover, at trial Dr. Hoffman expressed disapproval of the job analysis because it was begun in 1987 and may have borrowed from an earlier job analysis. However, Dr. Barrett testified that the TDC interviewed DCFD officials to make sure that the job analysis was still relevant to the 1991 examination and concluded that the jobs had not changed in any way related to the testing procedure. (See also Barrett Decl., ¶ 5).

Dr. Hoffman also criticizes the use of the twenty-two DCFD officers as Subject Matter Experts because "no representation is made in support of the competence of the commanders selected." (Hoffman Report, p. 4). However, there was never any evidence suggesting that these SME's were in any way incompetent, and Dr. Hoffman has not cited any such evidence. (O'Leary Decl., ¶ 38). Indeed, at trial Dr. Hoffman agreed during cross-examination that several of the SME's named by defense counsel were competent to give information to the TDC.

Dr. Hoffman further claims that the job analysis was flawed in that the racially-mixed group of fifty-four SME's who responded to the Job Analysis Questionnaire were biased because they had a personal interest in the questions that would be on the examination. (Hoffman Report, p. 6). There is no affirmative evidence in the record that any of the SME's were biased, however, and Dr. O'Leary explained that the SME's were chosen so as to obtain reliable and dependable information regarding the daily job duties of Sergeants, Lieutenants, and Captains. (See O'Leary Decl., ¶ 42).

The Court notes that Dr. Hoffman also challenges the TDC's decision not to include any questions regarding Basic First Aid in the examination based on the job analysis. Dr. Hoffman states that Basic First Aid is an important function "in every metropolitan fire district." (Hoffman Report, p. 5). However, the TDC's job analysis showed that in the D.C. Fire Department, company commanders are not involved in Basic First Aid because each company has an Emergency Medical Technician assigned to it. (O'Leary Decl., ¶ 39). As Dr. Hoffman concededly has no personal knowledge of the DCFD's policies regarding Basic First Aid, the Court finds his conclusions in this regard unsubstantiated and without merit.

b) Validity of the Job Knowledge Test

The Court's findings of fact demonstrate that the content validity of the Job Knowledge Test was established by the use of a systematic review procedure by bi-racial groups of senior officials in the Fire Suppression Division who served as Subject Matter Experts (SME's). As discussed by Dr. O'Leary and set forth above, the development of the Job Knowledge Test occurred over six distinct phases. During these phases, racially diverse groups of Fire Department officials provided information regarding the skills, knowledge and abilities necessary to perform their jobs. The relative importance of the various knowledge areas was reflected in the "Knowledge Area Weighting Form," which was used by Dr. O'Leary to determine the number of questions to draw from each knowledge area. (O'Leary Decl., ¶¶ 11-18).

Furthermore, prior to the administration of the examination, the TDC met with senior Fire Department officials and carefully reviewed each of the multiple choice items to ensure job-relatedness. Using a Content Validation Form (attached as Appendix 2 to the Validation Study), the officials responded to questions about each item such as "Is there a clear link between the information needed to answer the question and work performance?" and "Is the knowledge required to answer the question required of the incumbent to perform the work?" These SME's were a bi-racial group which included the Director of the Training Academy, and several Battalion Chiefs. (Barrett Decl., ¶ 17-19; O'Leary Decl., ¶ 18). In view of the procedure outlined above and the lack of evidence to the contrary, the Court finds that the content validation of the Job Knowledge Test was sufficient.

Dr. Hoffman claims that the selection of content for the Job Knowledge Test was not guided by the amount of time spent in the various job activities, or the importance of the job activities. (Hoffman Report, pp. 5-6). This, however, was the very point of Phase III of the development of the test. As discussed by Dr. O'Leary, DCFD officers were asked to weight the various knowledge areas by spreading 100 points across the knowledge areas commensurate with his or her opinion of its relative importance. The results of this process are summarized in the "Job Knowledge Weighting Form," which is attached as Appendix 1 to the Validation Study. (Defendant's Exh. 2). In creating questions, Dr. O'Leary took into account the relative weights that the officers had given to the different knowledge areas. More exam questions were created for those knowledge areas which the SME's had deemed particularly important. This is shown in the Job Knowledge Test Plan, which is attached as Appendix 8 to the Validation Study. (Id.) The Court thus finds that the examination meets the Federal Uniform Guidelines requirement that "the work behaviors selected for measurement should be critical behavior(s) and/or important work behavior(s) constituting most of the job." (O'Leary Decl., ¶ 40 (quoting 29 C.F.R. § 1607.14(C)(2)).

Dr. Hoffman also cursorily stated at trial that the job knowledge portion of the test violated thirty standards of testing, and thus has no validity. However, Dr. Hoffman did not identify these standards anywhere in his report or testimony.

The Court further finds that Dr. Hoffman is factually incorrect when he suggests that only two of the six phases of the job knowledge test development were completed. (Hoffman Report, p. 5). All six phases were completed, as explained in Appendix 3 of the Validation Study. (Defendant's Exh. 2; O'Leary Decl., ¶ 41). Accordingly, in sum, the Court finds that the Job Knowledge Test was valid.

c) Validity of Fire Scene Simulation

Finally, the Court finds that the Fire Scene Simulation is representative of important aspects of job performance, i.e., knowledge of strategies for attacking a fire, the ability to develop a plan of attack, and the giving of appropriate orders to subordinates, etc. See 29 C.F.R. § 1607.5(B). Indeed, there is no evidence in the record to compel the opposite conclusion.

Dr. Hoffman testified that the exam was invalid because each person only took one set of FSS exercises while, in his opinion, each individual must take four to ten. As the Intervenor observed at oral argument, however, Dr. Hoffman's views of examination testing are in many ways extreme, and this is one such example. The evidence showed that the administration of one FSS exercise took approximately one hour, and the Court cannot credit Dr. Hoffman's implicit claim that the DCFD must devote four to ten hours per person on this portion of the examination alone before it may be validated. The necessity of such an expenditure of resources has not been substantiated by the Plaintiffs.

Dr. Hoffman also testified that each applicant should have been provided with training prior to the administration of the examination so that they would be familiar with the duties and skills necessary for the officer position for which they were applying. Again, such an expenditure of resources would be extreme, and appears wholly unwarranted. Moreover, as Dr. Hoffman acknowledged on cross-examination, the applicants did have experience taking direction from officers in the positions for which they were applying. The Court fails to see how further training in the positions sought by Department personnel was required in order to validate the examination. Accordingly, the Court finds Dr. Hoffman's critiques of the FSS without merit.

2) Job Relatedness and Fire Prevention

Upon careful consideration of this history of Hammon v. Barry, the Court finds that the Plaintiffs in this action are precluded from challenging the decision not to administer a separate examination for Fire Prevention personnel, as that issue was resolved by the Court in the Hammon case. The question of whether a separate examination should be administered for the Fire Prevention unit arose during the negotiation and approval stages of the Hammon Settlement Agreement. The parties to Hammon, who apparently included all of these Plaintiffs,^[13] consented to a promotional procedure that envisioned that unitary promotional tests would be given in 1990 and 1991. Some of the Hammon Plaintiffs who were members of the Fire Prevention unit objected to the Settlement Agreement because it did not provide for separate examinations for the Prevention and Suppression units. Hammon v. Barry, 752 F. Supp. 1087, 1098 (D.D.C.1990). This Court resolved those objections in its November 1990 Order approving the Settlement Agreement.

In upholding the unitary promotional tests established by the Settlement Agreement, the Court noted, inter alia, that giving a separate exam for the Fire Prevention unit was not economically sound in light of the few officer vacancies in that unit. The Court further noted that administering a unitary test would allow Prevention personnel to compete for positions Department-wide. Id. The Court thus wrote that:

In light of the factors supporting the administration of a unitary promotional test at this time and the assurances by all the parties and the Special Master that they are receptive to the Fire Prevention objectors' concerns and are working diligently to consider alternatives to accommodate the Fire Prevention unit, the Court holds that the objections submitted by members of the Fire Prevention unit are insufficient to preclude this Court from approving the Settlement Agreement as a fair resolution of this litigation.

752 F.Supp. at 1098 (emphasis added).

In addition, the Court finds without merit the Plaintiffs' argument that the examinations are not job related for the Fire Prevention personnel who took them because they focus on fire suppression. The appropriate inquiry is not whether an examination is related to the job which the individual held at the time he or she took the exam, as the Plaintiffs suggest. Rather, the appropriate inquiry is whether the test is job related to the position for which the individual is applying. See 29 C.F.R. § 1607.5(B). In this case, the candidates from both the Fire Suppression and Fire Prevention units were applying for officer positions that were primarily located in the Fire Suppression Division. Thus, the 1991 examination, which focused on fire suppression, was job related because the content of the test related to the job for which the candidates were competing. The Uniform Guidelines promulgated by the Equal Employment Opportunity Commission make this point clear. Section 1607.5(B) of the Uniform Guidelines provides that

Evidence of the validity of a test or other selection procedure by a content validity study should consist of data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.

29 C.F.R. § 1607.5(B) (emphasis added). Thus, the Uniform Guidelines confirm that the appropriate comparison is between the subject matter of the test and the subject matter of the job being competed for — not the job that the test-taker may have held at the time he or she took the test.

In addition, as both the Court and the Special Master have observed, giving a unitary examination actually created more opportunities for advancement by Fire Prevention personnel. The unitary promotional test allowed Fire Prevention personnel to compete for vacancies throughout the Department, rather than only vacancies within the Fire Prevention unit. Hammon v. Barry, 752 F.Supp. at 1098. This fact is very significant because of the small number of officer vacancies in Fire Prevention. As the Special Master noted in his February 24, 1992 Report, there were only approximately fifty personnel working in the Fire Prevention unit, whereas the Fire Suppression Unit has a complement of approximately 1800 personnel, according to the TDC's validation report. (Special Master's February 24, 1992 Report on Fire Prevention Personnel and Promotions at 5; Defendant's Exh. 2, Validation Study, p. 12). There were only approximately five Sergeant slots in Fire Prevention, and even fewer vacancies. However, the results of the examination show that six African-Americans assigned to the Fire Prevention Division were promoted as a result of the unitary promotional examination given in 1991; in particular, four were promoted to Sergeant and two were promoted to Lieutenant. The Court thus finds no merit to the Plaintiffs' challenge to the administration of a unitary examination for both the Fire Suppression and Fire Prevention units.

3) Administration of the 1991 Examination

The Court also finds that the 1991 promotional examination was administered in a manner consistent with accepted practice. The TDC members were present during the administration, and observed no administrative flaws which would undermine the validity of the tests. In addition, no evidence of any administrative problems was presented at trial and, as set forth in its findings of fact, the Court finds no merit to the Plaintiffs' claim that an alleged "cheat sheet" existed so as to invalidate the entire examination. Finally, while the Plaintiffs criticize the TDC for not expressly reviewing any aspect of the administration of the exam, it appears to the Court that such a review was unnecessary as there is no evidence in the record to refute the testimony of Dr. Barrett, Dr. O'Leary and Billie Smith that no problems relating to the administration of the examination were brought to their attention. (O'Leary Decl. ¶ 29; Barrett Decl. ¶ 21; Smith Decl. ¶ 7).

4) Reliability of the Examination

Furthermore, the Court finds that the test meets professionally acceptable standards of reliability for the reasons explained by Dr. Barrett. As Dr. Barrett noted, the reliability of a test battery has a different meaning than the reliability of a homogeneous test such as the multiple-choice test. In building a multiple-choice test of job knowledge, the test author attempts to write questions that get at different specific components of the job knowledge. Therefore, a high level of reliability is a sign of a good test. However, in this case, three different tests were used to get at three different facets of job performance, knowledge of rules and policies, administrative skills, and knowledge of fire suppression. Thus, according to Dr. Barrett, it would be expected that the scores would correlate poorly with each other. If they correlated at a high level, one or two of the tests would be superfluous. (Barrett Decl., ¶ 24).

5) Adequacy of TDC Validation Study

At oral argument, the Court asked the Plaintiffs whether their case would collapse should the Court decide not to credit the testimony of Dr. Hoffman. In response, the Plaintiffs asserted that the Court would have to "struggle with" their argument that the Defendant failed to provide a large volume of information allegedly needed to review the Validation Study. In other words, the Plaintiffs challenge the TDC's Validation Study^[14] on the grounds that it is incomplete under 29 C.F.R. § 1607.15. However, the evidence shows that the November 1991 Validation Study and the appendices attached to it (Defendant's Exh. 2), as well as the information from the Job Analysis (Defendant's Exh. 3, 4, 5, 6), contain the information required by the Uniform Guidelines. The Guidelines broadly state that documentation evidence "should" be maintained, and that the evidence "should be compiled in a reasonably complete and organized manner to permit direct evaluation of the validity of the selection procedure." 29 C.F.R. § 1607.15A(3)(b). See also Id. § 1607.15(C) (setting forth the particular information content validity reports should contain). The Court finds that the Validation Study and related documentation is sufficiently complete under this standard.

It is undisputed that the Defendant provided 16,000 pages of discovery to the Plaintiffs, and there is no serious allegation that the Defendant withheld any pertinent information. To the extent that the Defendant produced documents late (a few days before trial), they were stricken from the record. Moreover, there is no contention that the tardy production of documents was not due to inadvertent error. Finally, while the Plaintiffs argued that the Uniform Guidelines mandated maintenance of the "raw data" underlying the TDC's conclusions set forth in the Validation Report, they offer no authority for that claim. Indeed, while their own opinion witness, Dr. Hoffman, purported to rely on the Guidelines in attacking the validity of the 1991 examination, he further testified on cross-examination that he did not have "much familiarity" with the Guidelines. The Court thus credits the Defendant's opinion witnesses representation that all information required by the Guidelines was contained in the Validation Report. In any event, the Plaintiffs proffer no authority for the proposition that failure to maintain the actual "raw data" underlying the information contained in a Validation Study of a test that has otherwise not been shown to be invalid compels a Court to "throw out" the test entirely. As counsel for the Intervenor observed, such an argument resembles a tacit concession that the evidence presented is insufficient in itself to support a finding that the test was invalid under Title VII.^[15]

C. IN FAILING TO SHOW THE EXISTENCE OF EQUALLY EFFECTIVE, NON-DISCRIMINATORY ALTERNATIVES TO THE CHALLENGED EXAMINATION, THE PLAINTIFFS DID NOT MEET THEIR BURDEN OF PROVING THAT THE 1991 EXAM WAS DISCRIMINATORY IN VIOLATION OF TITLE VII

Once the Defendant produces evidence of a legitimate business justification for the challenged employment practice, the Plaintiffs may still persuade the factfinder that ``other tests or selection devices, without a similarly undesirable racial effect, would also serve the employer's legitimate [promotional] interests;' by so demonstrating, [the Plaintiffs] would prove that ``[the employer was] using [the] tests merely as a ``pretext' for discrimination." Wards Cove, 490 U.S. at 660, 109 S. Ct. at 2126 (quoting Albemarle Paper Co., 422 U.S. at 425, 95 S. Ct. at 2375). See also Watson v. Fort Worth Bank & Trust, 487 U.S. 977, 998, 108 S. Ct. 2777, 2790, 101 L. Ed. 2d 827 (1988). "Factors such as the cost or other burdens of proposed alternatives are relevant in determining whether they would be equally as effective as the challenged practice in serving the employer's legitimate business goals." Wards Cove, 490 U.S. at 661, 109 S. Ct. at 2127 (quoting Watson, 487 U.S. at 998, 108 S. Ct. at 2790).

The Plaintiffs' expert, Dr. Hoffman, mentioned several alternatives to the challenged 1991 promotional examination in his report and at trial. In particular, Dr. Hoffman mentions in his report the possibility of differential weighing of the examination, utilization of different passing points, and rescoring of the examination. (Plaintiffs' Exh. 63, p. 16). The Court finds no merit to Dr. Hoffman's suggested alternatives.

Indeed, the Court observes that Dr. Hoffman himself condemns all of his suggested alternatives. First, he states that differential weighing "cannot improve the situation," and that utilization of different passing points "must be discarded as an alternative" given his conclusion that the 1991 examination is invalid. (Id.) Second, as for his suggested alternative of rescoring the examination, Dr. Hoffman states that, while rescoring of the job knowledge part "might reduce adverse impact," the accuracy of rescoring the Assessment Center and the Fire Scene Scenario is affected by the passage of time. (Id. (emphasis added)). Dr. Hoffman concludes this section of his report by stating:

[t]he specification of suitable alternatives, given that the current (1991) test must be abandoned, is beyond the scope of this report.... There are indeed suitable alternatives; ways of overcoming these problems, but in my opinion, the difficulties in the District of Columbia will not be solved by another set of examinations in the traditional vein.

(Id.)

Dr. Hoffman also testified that one feasible alternative to the job knowledge portion of the examination would be subjective evaluations by supervisory personnel. As it appears that Dr. Hoffman failed to familiarize himself with the history of this case or the prior Orders of this Court, the Court finds this alternative completely without merit. The promotional examinations were administered in connection with the Hammon Settlement Agreement, following six years of litigation over serious allegations of racial discrimination in the Department. Under such circumstances, to posit that subjective evaluations by supervisory personnel would be an equally effective but less discriminatory alternative than use of a three-part examination developed by an independent TDC and a Special Master appointed by this Court borders on the specious.

Finally, at oral argument Plaintiffs' counsel asserted that they met their burden of proving an equally effective non-discriminatory alternative through cross-examination of the Defendant's opinion witness, Dr. Barrett. In particular, Plaintiffs' counsel argued that Dr. Barrett acknowledged that the Job Assessment Center, which had no adverse impact, could have been afforded 80% weight, thereby reducing the adverse impact of the overall examination. However, Dr. Barrett called this alternative "ridiculous" because it would have marginalized the other portions of the examination which tested cognitive skills. In addition, the Plaintiffs' argument contradicts their position throughout the trial that the Assessment Center was invalid, as well as the conclusions of their own opinion witness, Dr. Hoffman, that because "none of the examination components can be considered valid[,] differential weighting cannot improve the situation." (Hoffman Report, p. 16). Accordingly, the Court finds no merit to the Plaintiffs' claim that affording more weight to the Assessment Center would have produced a less discriminatory — but equally effective — examination.

CONCLUSION

For the reasons articulated herein, the Court finds that the Plaintiffs have not proved their case under Title VII. In particular, the Court finds that, although the Plaintiffs established their prima facie case as to the 1991 exam (as it relates to the Sergeants' examination, but not as to the Lieutenants' and Captains'), the Plaintiffs did not carry their burden of proving that the exam was discriminatory. The Defendant made a credible showing that the entire 1991 examination was valid and job-related, thereby furthering the legitimate business concerns of the District of Columbia, and the Plaintiffs failed to prove otherwise. In addition, the Plaintiffs failed to present competent evidence that there was cheating on the examination. Finally, the Plaintiffs did not show that alternative methods of promotion exist that would have a less discriminatory impact. Accordingly, the Court shall enter judgment for the Defendant and shall dismiss the above-captioned case from the dockets of this Court.

Furthermore, as a significant disparity appears to exist between the financial resources of the Plaintiffs and the Defendant, and as the litigation raised issues of public concern, the Court, in the exercise of its discretion, will not assess costs against the Plaintiffs in this case. See Wrighten v. Metropolitan Hospitals, Inc., 726 F.2d 1346 (9th Cir.1984); Badillo v. Central Steel & Wire Co., 717 F.2d 1160, 1165 (7th Cir.1983).

NOTES

[1] The Third Amended Complaint named as Defendants the District of Columbia, Thomas C. McCaffrey, Jr., a former Assistant Fire Chief, and McEldon L. Fleming, a Battalion Chief. By Memorandum Opinion and Order entered September 6, 1994, the Court dismissed the Plaintiffs' claims against individual Defendants McCaffrey and Fleming on qualified immunity grounds.

By Order entered November 9, 1994, the Court granted the Motion of Local 36, International Association of Fire Fighters to Intervene in this case, pursuant to Rule 24 of the Federal Rules of Civil Procedure.

[2] In particular, on May 8, 1995 the Plaintiffs filed a "Precipie [sic] Regarding Defendants' Motion for Partial Summary Judgment," in which they basically set forth several after-the-fact excuses for their failure to file an opposition to the Defendant's Motion, including the lack of evidence revealed through discovery of a municipal policy or custom of discrimination.

[3] The Court observes that those individuals who chose for personal reasons not to take the 1991 examinations do not have standing to sue, and cannot establish a prima facie case of discrimination. In other words, those Plaintiffs who decided, themselves, not to take the 1991 examinations cannot show that any alleged discrimination relating to the exams caused actual injury to them. Such Plaintiffs may have been promoted if they had taken the exams. Accordingly, they do not have standing to challenge the validity of the examinations. Moreover, those Plaintiffs who simply chose not to take the examinations cannot establish a prima facie case that they were discriminated against by the examinations. Pursuant to the standard established in McDonnell Douglas Corp. v. Green, 411 U.S. 792, 93 S. Ct. 1817, 36 L. Ed. 2d 668 (1973), to establish a prima facie case, a plaintiff must prove "(1) that he belongs to a protected minority; (ii) that he applied for and was qualified for a job for which the employer was seeking applicants; (iii) that, despite his qualifications, he was rejected; and (iv) that, after his rejection, the position remained open and the employer continued to seek applicants from persons of complainant's qualifications." McDonnell Douglas Corp., 411 U.S. at 802, 93 S. Ct. at 1824; Cooper v. Federal Reserve Bank, 467 U.S. 867, 104 S. Ct. 2794, 81 L. Ed. 2d 718 (1984). Those Plaintiffs who, like Plaintiffs Willis Joseph Wilson, Anthony Staten, John Briscoe, and George McDuffie, made a conscious decision not to take the 1991 examinations cannot establish a prima facie case of discrimination.

The Court also rejects any argument that Plaintiffs Wilson, Staten, Briscoe, McDuffie or others who chose not to take the exams were deterred or "chilled" from doing so due to alleged discrimination during the December 1990 examination. This Court determined in Allen that there was no discrimination with respect to the 1990 examinations. Thus, the Court finds no merit to the Plaintiffs' argument that they were deterred by a previous examination which the Court found to be valid.

[4] Following the Court's approval of the Hammon Settlement Agreement, an issue arose whether firefighters who received promotions as a result of the 1990 examination would be eligible to take the 1991 examination, in light of the District's "year-in-grade" requirement. See 31 D.C.Reg. 2715 (1984). The Special Master issued a Recommendation to the Court after examining the Settlement Agreement and conducting hearings on this issue. On January 27, 1992, the Court resolved this issue by ruling that those firefighters who received promotions to vacancies existing for more than one year qualified as having served a year-in-grade for purposes of eligibility for the July 1991 examination. (Memorandum Opinion entered January 27, 1992, Defendant's Exh. 13). The issue of eligibility to take the 1991 examination was thus firmly resolved in the context of the Hammon case, and the Hammon parties, which include African-American firefighters who are Plaintiffs in this case, had an opportunity to be heard on this issue during the hearings conducted by the Hammon Special Master. Accordingly, the doctrine of res judicata precludes relitigation of this issue in the present case. The Court observes that the Plaintiffs' claim that this issue is properly before the Court in this case is wholly disingenuous, as the Court's Memorandum Opinion and Order entered January 27, 1992 is quite explicit on this issue.

[5] This case was filed on December 28, 1992. Following the filing of the Complaint in December, the Plaintiffs have made nine applications for continuations of their obligations and have filed, out of time, nine documents required by Court Order or Local Rule. The applications for continuations and the tardy filing of documents have occurred at every stage of this proceeding, including the time for filing of amended complaints, completion of discovery, motions, pre-trial requirements and trial responsibilities. (See Order entered May 26, 1995). In contrast, the Defendant has made six applications for extensions of time and has filed one document out of time. Together, the parties have made two applications for extensions of time.

[6] The Court observes that, while the Defendant's post-trial Proposed Findings of Fact and Conclusions of Law are virtually identical to its pre-trial Findings, the Plaintiffs object to large portions of the Defendant's post-trial Findings where they conceded those same portions in the pre-trial Findings. (See, e.g., Defendant's Proposed Findings of Fact and Conclusions of Law, ¶¶ 49-50). The Court afforded the parties an opportunity to file supplemental Findings following trial in response to the Plaintiffs' request to file post-trial briefs. The Court permitted the filing of post-trial Findings to allow the parties to incorporate any new evidence adduced at trial, and this ruling did not operate to nullify the pre-trial submissions or, more importantly, the purpose and effect of the Court's pre-trial Orders. The Court shall not allow the Plaintiffs to use this opportunity as a "second bite at the apple" for voicing their objections to material that they previously conceded.

Thus, to the extent that portions of the Defendant's post-trial Findings are identical to its pre-trial Findings and were based on Declarations and/or exhibits made available to the Plaintiffs in advance of trial, the Court shall consider the Plaintiffs' marked-up copy filed prior to trial as governing the identification of the Plaintiffs' objections and concessions to the Defendant's Findings of Fact and Conclusions of Law. The same shall hold for any inconsistencies in the Defendant's marked-up versions of the Plaintiffs' pre-and post-trial Proposed Findings of Fact and Conclusions of Law.

[7] The Court observes that, under the relevant law, the testimony of the opinion witnesses and the exhibits directly relating to the preparation, administration, and review of the 1991 exam constitute the critical evidence submitted at trial. The Court further notes, however, that Plaintiffs' counsel devoted the bulk of their trial time to calling individual Plaintiffs to the stand to testify. Notwithstanding the Court's explicit Orders directing counsel to prepare Declarations containing each witness' complete testimony and review the same with opposing counsel to iron out any evidentiary objections prior to trial, at the urging of Plaintiffs' counsel, the Court afforded the majority of the Plaintiffs' witnesses considerable time at trial to add to their Declarations. This was done over the Defendant's objection to allow these individuals an opportunity to be heard by the Court. However, while the Court is sympathetic to the compelling plights of many of these African-American firefighters, this is a disparate impact case, not a disparate treatment case, and their testimony is not highly relevant under the operative legal standards. The Court apparently is not alone in this view, as the Plaintiffs' Proposed Findings of Fact and Conclusions of Law focus primarily on the testimony and report of their opinion witness, Dr. Hoffman, with little mention of the individual Plaintiffs' testimony. The Court has nonetheless carefully considered the testimony of each and every witness called at trial, and was indeed extremely impressed by the qualifications and professionalism of each of these men and women who have dedicated their lives to public service.

[8] The Court observes that, during cross-examination of Dr. Hoffman, a portion of a sworn statement by retired DCFD Lieutenant Romeo Spaulding was read into the record. Plaintiff Romeo Spaulding, who served in a Captain's position for several years, testified that in his sworn statement that a Sergeant performs the same duties as Lieutenant and Captain. Dr. Hoffman indicated he had no information to contradict that statement.

[9] As the events raised in the Plaintiffs' Complaint occurred prior to the enactment of the Civil Rights Act of 1991, the Plaintiffs' claims are properly analyzed under the 1964 Act and relevant case law. See Landgraf v. USI Film Products, ___ U.S. ___, 114 S. Ct. 1483, 128 L. Ed. 2d 229 (1994). The applicable law is not in dispute.

[10] The Court therefore granted Defendant's Motion for Judgment as a Matter of Law pursuant to Rule 50 of the Federal Rules of Civil Procedure with respect to the 1991 Lieutenants' and Captains' promotional examinations, but not as to the 1991 Sergeants' examination. (See Order dated June 8, 1995 at 1). The Court observes that, in opposition, the Plaintiffs argued that the 1991 examination could not be so segregated, and that if any aspect of the 1991 examination was found to be invalid, the entire examination is invalid. As the Court of Appeals for this Circuit has made clear, however, "[w]here possible, the district courts should retain the flexibility to address only those components of selection devices that are implicated by a prima facie case on the part of the plaintiffs." Frazier v. Consolidated Rail Corp., 851 F.2d 1447, 1455 (D.C.Cir.1988). See also Wards Cove, 490 U.S. at 653, 109 S. Ct. at 2122-23 ("Racial imbalance in one segment of an employer's workforce does not, without more, establish a prima facie case of disparate impact with respect to the selection of workers for the employer's other positions, even where workers for the different positions may have somewhat fungible skills....").

[11] The Court nevertheless observes that, in an abundance of caution, it has thoroughly reviewed the evidence presented at trial regarding the validity of the Assessment Center, and finds that the Plaintiffs have not met their burden of showing it was an invalid test under the applicable law. Indeed, while the Plaintiffs attacked the Assessment Center throughout the trial, the Plaintiffs argued following the trial that the Assessment Center should have been afforded more weight by the TDC because it was anticipated by the TDC to have little adverse impact.

[12] Congress changed the allocation of the burden of persuasion with the enactment of the Civil Rights Act of 1991. 42 U.S.C. § 2000e-2(k) (1991).

[13] The Hammon Plaintiffs included those individuals who "are or were a member of the District of Columbia Fire Department" or those who were "unsuccessful applicant[s] for employment with the Fire Department" and who "claim to be ... victim[s] of racial discrimination because [they] are ... black...." Hammon v. Barry, 752 F. Supp. 1087, 1126 (D.D.C.1990).

[14] It is worth noting that, as a general matter, it is "clear that employers are not required, even when defending standardized or objective tests, to introduce formal ``validation studies' showing that particular criteria predict actual on-the-job performance." Watson v. Fort Worth Bank and Trust, 487 U.S. 977, 998, 108 S. Ct. 2777, 2791, 101 L. Ed. 2d 827 (1988). Here, the Hammon Special Master directed the TDC to prepare a report on the examination. (June 19, 1991 Order Regarding TDC at 2).

[15] The Plaintiffs also argue that the examination is invalid because the Validation Study does not analyze the scores as modified with the addition of points for service and education. The Plaintiffs, however, proffer no authority to support their claim that the exclusion of such an analysis from the Validation Report invalidates the entire examination. In support of this argument, the Plaintiffs refer in their post-trial Proposed Findings of Fact and Conclusions of Law to the report of Dr. Hoffman and the Guidelines in general. (Plaintiffs' Proposed Findings of Fact and Conclusions of Law, ¶¶ 50-51). The Court, however, has combed Dr. Hoffman's Report and testimony and finds no mention of the addition of points for service and education. The Court thus finds that there is no evidence on the record to support this argument.