DocketNumber： C-84-4045 RFP
Judges： Peckham
Filed Date： 6/10/1985
Status： Precedential
Modified Date： 11/6/2024
United States District Court, N.D. California,
Ralph B. Saltsman, Soloman, Saltsman & Hart, Marina Del Rey, Cal., for plaintiffs.
Michael C. Killelea, Deputy City Atty., James R. Wheaton, Lois Salisbury, Public Advocates, Inc., San Francisco, Cal., for defendants.
Joel Nomkin, Dept. of Justice, Washington, D.C., for U.S. (amicus curiae).
PECKHAM, Chief Judge.
During 1983, the City and County of San Francisco (the City), under the auspices of *1226 its Consent Decree Unit, administered the selection procedures for Q/35 Assistant Inspector and Q/50 Sergeant. Those procedures, which are the subject of this litigation, included a written multiple choice examination, a written communications examination and a structured oral examination. On September 12, 1983, the Civil Service Commission (CSC) established cut-off scores for the multiple choice examinations and set the weights for all three components of the selection process. On February 7, 1984, the City provided the parties to the Consent Decree with the results in rank order, by race and sex, of the Q/35 and Q/50 selection procedures. Using the weights set by the CSC, the results demonstrated severe adverse impact for minorities in both ranks, and slight adverse impact for women on the Q/35 Assistant Inspector exam.
Because of the adverse impact, the United States and Officers For Justice filed objections under the Consent Decree to the City's use of the results of the Q/35 and Q/50 selection procedures. Extensive negotiations ensued. As a result of these negotiations, the City Attorney recommended to the CSC that it modify its proposed use of the Q/35 and Q/50 selection process. The Commission then ordered a revised weighing system which substantially eliminated the adverse impact of the selection process. After the CSC's decision, the Consent Decree plaintiffs and the City entered into an agreement withdrawing the plaintiffs' objections to the Q/35 and Q/50 selection process.
On June 7, 1984, the Police Officers Association (POA) and three named plaintiffs filed a complaint in the Superior Court of California which challenged the Civil Service Commission's (CSC) decision to reweigh the Q/35 and Q/50 selection procedures. The defendants, the City and County of San Francisco and the Civil Service Commission, removed the action to federal court under the authority of the Civil Rights Removal Statute. The court denied plaintiffs' motion to remand on December 31, 1984. On that date, the court granted the POA's motion for preliminary injunction, enjoining the City from utilizing the reconsidered weights for the Q/50 and Q/35 examinations until the lawsuit is resolved.
On February 28, 1985, the court granted partial summary judgment in favor of the City and the CSC. The court recognized that the Supremacy Clause and the provisions of the Consent Decree and Title VII authorized the CSC to disregard Civil Service rule 5.07[1], when faced with the adverse impact resulting from the promotional examinations. After the February 28, 1985 order, the court still has before it the issue of whether the plan chosen by the City was a permissible affirmative action plan within the meaning of United Steelworkers of America v. Weber, 443 U.S. 193, 208, 99 S.Ct. 2721, 2730, 61 L.Ed.2d 480 (1979).
Based on the extensive briefing on all aspects of the ramifications of the City's decision to reweigh the examinations and a hearing on the issue of whether the exams as reweighed unnecessarily trammeled the rights of the examinees, the court now holds that the plan adopted by the Civil Service Commission to remedy the adverse impact resulting from the promotional examinations was a permissible affirmative action plan under Weber. The court therefore dissolves its preliminary injunction previously issued in this case and orders the City to begin making promotions for the ranks of Assistant Inspector and Sergeant from the eligibility lists derived from the reweighed examinations.
On March 30, 1979, this court approved a Consent Decree which was entered into as *1227 a settlement of a dispute among plaintiffs, Officers For Justice (OFJ) and the United States, defendants, the City and County of San Francisco and the Civil Service Commission, and intervenor, the POA, as to appropriate and valid procedures for the hiring and promotion of police officers for the City and County of San Francisco. The Consent Decree provides for specific, definable and good faith efforts to be made by defendants to achieve certain goals for employment of women and minorities within specified periods of time.
The Consent Decree imposed numerous obligations on the City with respect to the selection of candidates for entry and promotive positions in the San Francisco Police Department (SFPD). Among other things, the Decree prohibited the use of selection procedures which have an adverse impact against minorities and females, unless such procedures have been shown to be valid pursuant to the Uniform Guidelines on Employee Selection Procedures, 28 C.F.R. 50.14. Consent Decree at ¶ 4. With respect to the selection of Q/50 Sergeants and Q/35 Assistant Inspectors, the Decree set an annual objective of selecting minorities and females to those positions in proportion to their representation in the qualified applicant pool. Id. at ¶ 10(a)(3).
To discharge its obligations under the Decree, the City established a Consent Decree Unit charged with, inter alia, the development and administration of examinations. During 1983, the Consent Decree Unit administered the selection procedures for Q/35 Assistant Inspector and Q/50 Sergeant. Those procedures included a written multiple choice examination, a written communications examination, and a structured oral examination. Each component was designed specifically to measure different dimensions found in the job-analysis to be essential for successful performance as a Sergeant/Assistant Inspector. The multiple choice component was considered the best method for measuring technical knowledge and problem solving. The writing test was designed to test candidates' written communication skills, and the oral test measured oral communication skill, supervisorial skills and interpersonal ability. Each examination component was given at different times during 1983.
On September 12, 1983, after the multiple choice examination had been administered but before administering the two other components, the Civil Service Commission (CSC) established cut-off scores for the multiple choice examinations and set the weights for all three components of the selection process. The cut-off scores for the multiple choice examinations were fixed at 55 percent for the Q/35 and 50 percent for the Q/50. The weights set for the Q/35 procedures were:
Multiple choice examination 45% Written communication examination 29% Oral Examination 26%
The weights set for the Q/50 procedures were:
Multiple choice examination 41% Written communication examination 29% Oral examination 30%
On February 7, 1984, the City provided the parties to the Consent Decree with the results in rank order, by race and sex, of the Q/35 and Q/50 selection procedures. Using the weights set by the CSC, the top 75 places on the Q/35 eligibility list would have included six (8.0 percent) minorities and four (5.3 percent) females, out of a pool of 26 percent minorities and 10 percent females. The top 125 places on the Q/50 eligibility list would have included thirteen (10.4 percent) minorities and twelve (9.6 percent) females, out of the same pool of minorities and women as in the Q/35. These percentages represent severe adverse impact for minorities in both ranks, and a slight adverse impact for women on the Q/35 Assistant Inspector exam. Women were not affected adversely for Q/50 Sergeant.
Because of the adverse impact, the United States and Officers For Justice filed *1228 objections under the Consent Decree to the City's use of the results of the Q/35 and Q/50 selection procedures. This court subsequently scheduled a hearing date of June 20, 1984 for consideration of the objections.
Based on discovery conducted after the filing of their objections, the Consent Decree plaintiffs contended that neither the Q/35 nor the Q/50 selection procedures were valid, if used in the manner intended by the City. The primary basis for this contention was the performance of incumbent Inspectors and Sergeants who pretested the Q/35 and Q/50 multiple choice examinations. The Chief of Police and his subordinates had selected these incumbents to pretest the exam because of their superior performance in the police department. Yet, they performed so poorly on the pretest that they would have had no reasonable chance of appointment from the Q/35 and Q/50 rosters.
The court appointed Dr. Joel Lefkowitz, an expert industrial psychologist, to review the selection process. Dr. Lefkowitz reported that insufficient attention was paid to the issue of adverse impact, especially with regard to the use of the multiple choice test and that alternatives which may have minimized such impact were not explored.
The evidence uncovered during discovery and the findings of Dr. Lefkowitz prompted extensive settlement negotiations between the Consent Decree plaintiffs and the City. As a result of these negotiations, the City Attorney recommended to the CSC, in a public session held on May 29, 1984, that it modify its proposed use of the Q/35 and Q/50 selection process. In particular, he recommended that the Commission use the structured oral examination for ranking purposes and the writing skills and multiple choice exams as pass/fail devices.
In making his recommendation, the City Attorney advised the CSC of the "catastrophic" impact of the selection process on minorities. He explained that this result could not be squared with the Consent Decree, both because the selection process was not "completely valid," and because a "reasonable" alternative weighing system was available which would alleviate the adverse impact. Failure to accept this alternative, he said, would demonstrate lack of "good faith" in facilitating the objectives of the Decree and potentially result in the judicial "set aside" of the entire selection process. He further stressed that reweighing was the only way to settle the plaintiffs' objections short of litigation. See generally CSC Transcript (May 29, 1984).
On June 4, 1984, the CSC adopted the City Attorney's recommendation. The Commission ordered a revised weighing system which substantially eliminated the adverse impact of the selection process. Under the revised system, the cut-off scores of 50 percent for the Q/50 and 55 percent for the Q/35 for the multiple choice examination remained the same. A cut-off score was added for the written examination and set at 60 percent. The oral component of the exam was made the sole ranking device. The top 45 appointments from the revised Q/35 eligibility list will include ten (22.2 percent) minorities and five (11.11 percent) females, and the 75 persons appointed from the revised Q/50 eligibility list will include eighteen (24 percent) minorities and eleven (14.6 percent) females.
After the CSC's decision, the Consent Decree plaintiffs and the City entered into an agreement withdrawing the plaintiffs' objections to the Q/35 and Q/50 selection process. This court entered a Stipulation and Order on June 6, 1984 under which it retained jurisdiction "over all issues related to the weighting of the components of the Q/35 and Q/50 selection procedures, the duration of eligibility lists established as a result of those procedures, and the number of persons to be appointed to the Q/35 and Q/50 ranks."
The Ninth Circuit in Johnson v. Transportation Agency, 748 F.2d 1308, 1311 (9th Cir.1984), extracted four criteria from *1229 United Steelworkers of America v. Weber, 443 U.S. 193, 208, 99 S.Ct. 2721, 2730, 61 L.Ed.2d 480 (1979), to aid in determining whether an affirmative action plan is permissible. A plan is permissible if (1) it is designed to break down old patterns of racial segregation and hierarchy; (2) it does not create an absolute bar to the advancement of white employees; (3) it is a temporary measure, "not intended to maintain racial balance, but simply to eliminate a manifest racial imbalance"; and (4) it does not unnecessarily trammel the interests of white employees. Johnson, 748 F.2d at 1311, citing, Weber, 443 U.S. at 208, 99 S.Ct. at 2730.
The court will discuss each of these factors to demonstrate that the plan chosen to remedy the adverse impact of the promotional exams meets these criteria.
There is no doubt that the exams were reweighed because of their adverse impact. The decision to reweigh the exams, which results in more minorities being appointed, was designed to meet the requirements of the Consent Decree. This court approved the Consent Decree only after finding that the selection procedures administered by the City for positions within the Police Department had unlawfully discriminated against minorities. See Officers For Justice v. Civil Service Commission, 371 F.Supp. 1328 (N.D.Cal.1973). In particular, with respect to the position of sergeant, the court found in 1973 that the City's "prior discriminatory practices had virtually excluded minorities from the rank of sergeant." Officers For Justice v. Civil Service Commission, 14 EPD ¶ 7458 at 4715 (N.D.Cal.1977). In the eleven years since this finding was made, the total number of minority sergeants in the Department has increased by only one.
Quite obviously, the decision to reweigh was made in order to break down old patterns of racial segregation and hierarchy in the San Francisco Police Department.
The plaintiff in Johnson argued that the selection of a woman for the position which he was seeking served as a complete bar to his selection for the position. The Ninth Circuit rejected "this narrow view." 748 F.2d at 1314. The court stated that unless it was "shown a distinct pattern of exclusion of non-minority candidates from such position, [it] cannot conclude that a single employment decision serves as a bar...." Id. Certainly, the reweighing of the exams did not create an absolute bar to the advancement of white employees. Any white employee who passed the multiple choice and written components and scored sufficiently high on the oral component of the exam would be entitled to placement on the eligibility list.
The Supreme Court approved the affirmative action plan in Weber, noting that "half of those trained in the program will be white." 443 U.S. at 208, 99 S.Ct. at 2730. Here, white officers do even better than in Weber under the revised weights. Not only will 78 percent of the appointments go to white officers, but whites will be appointed at a rate disproportionately high to their representation in the pool of examinees.
Under the facts of this case, there is simply no basis for maintaining that the reweighing "create[s] an absolute bar to the advancement of white employees."
In Johnson, the court found that the plan was temporary because "[i]mplicit in the plan is the intent to stop taking sex into account once the long-range percentage goals are attained." 748 F.2d at 1313. Likewise, with the instant plan, neither sex nor race will be taken into account once the City administers an exam that does not result in adverse impact. The reweighing plan is limited to removing imbalance in a single set of examinations. New examinations with their own weights will be used for future promotions. Moreover, the adverse impact resulting from a promotional exam presumably will not require remedial *1230 action once the long-range percentage goals of the Consent Decree are reached.
The Ninth Circuit in Johnson summarily concluded that "[u]nless we are shown a distinct pattern of exclusion of non-minority candidates from such positions, we cannot concluded that a single employment decision serves as a bar or unnecessarily trammels the interests of other employees. The record contains no evidence of such a pattern in this case." 748 F.2d at 1314.
The opposite result was reached by the district court in Hammon v. Barry, 606 F.Supp. 1082 (D.D.C.1985). There, the court found that the plan sought to deprive white firefighters of a legitimate and longstanding expectation of an equal opportunity to advance into the supervisory ranks, an opportunity they had earned by serving the requisite minimum of five years in the department and scoring well on the promotional examination. According to the court, "the fact of past discrimination alone is not enough to deprive innocent whites of their legitimate expectation of advancement." It concluded that because the plan made race a mandatory consideration over merit, it unnecessarily trammeled the interests of white firefighters.
Both of these cases are distinguishable from the case at hand. Neither race nor sex was considered before merit under the reweighing plan. In fact, neither race nor sex was overtly taken into account in making up the new eligibility lists. Nonminorities were not deprived of a legitimate expectation of advancement. Instead, minorities and nonminorities alike were advanced in a manner that had not been originally contemplated. The real issue is whether the City could "change the rules after the game had started"[2] without trammeling the rights of the examinees.
The City claims that since being on an eligible list affords no right to an appointment, the candidates have no right which is being trammeled. Similarly, in Kirkland v. N.Y. State Dept. of Correctional Services, 711 F.2d 1117, 1134 (2d Cir.1983), the Second Circuit found that under New York state law a person on an eligibility list does not possess "any mandated right to appointment or any other legally protectible interest."
The Second Circuit, however, further stated that "[t]he only relevant state right intervenors possess is the right to challenge the settlement on the grounds that the manner in which it provides for appointments is unlawful, arbitrary, and capricious or constitutes an abuse of discretion ... This right intervenors exercised in the district court." Kirkland, 711 F.2d at 1134. The Kirkland court cited two New York state cases for this proposition. In Adelman v. Bahou, 85 A.D.2d 862, 446 N.Y.S.2d 500 (1981), petitioners sought upward reclassifications of their positions without examination. The court stated that it could not overturn a commission's determination unless it was without a rational basis. And, in Burke v. Sugarman, 35 N.Y.2d 39 (1974), the Court of Appeals held that persons who pass an examination and are on an eligible list have standing to challenge unlawful appointments or designations to positions for which the list has been established.
California recognizes a similar right. In Fuchs v. Los Angeles County Civil Service Com'n, 34 Cal.App.3d 709, 715, 110 Cal.Rptr. 311 (1973), the court stated that "[b]oth the petitioner and the public have an interest in ensuring that the best selection procedures are used in determining the advancement of public employees to positions of increased responsibility and authority." Fuchs had filed his action attempting to cancel the list of eligible candidates *1231 for the position of head deputy district attorney and to void any appointment to that position made from the list of eligible candidates. He claimed that parts of the examination weighed factors not relevant to a determination of qualifications for head district attorney. He further contended that his oral interview was not conducted fairly. The court found that there was no indication that the Commission or the director of personnel acted arbitrarily or unfairly, within the established framework, to deprive the petitioner of his promotional opportunity. Id. at 716, 110 Cal. Rptr. 311 (emphasis added).
In Almassy v. L.A. County Civil Service Com., 34 Cal.2d 387, 210 P.2d 503 (1949), the petitioner sought a writ of mandate to compel the Los Angeles County Civil Service Commission to annul two promotional civil service examinations and their corresponding eligibility lists. The California Supreme Court recognized that the civil service commission has been given broad discretionary powers in determining the subjects of examination and the qualifications which are to be measured. The court stated that "[j]udicial interference under such circumstances is unjustified except on a showing of arbitrary, fraudulent, or capricious conduct, or a clear abuse of discretion." 34 Cal.2d at 396, 210 P.2d 503. No such showing was made in the case.
These two cases make clear that California recognizes the right of an examinee to challenge the promotional examinations if the Civil Service Commission has acted arbitrarily, capriciously, or unfairly in depriving the examinees of their promotional opportunities. If the exams as reweighed were not capable of fully measuring the qualifications needed to be an assistant inspector or a sergeant, then the Commission acted arbitrarily when it decided to use the oral component as the sole ranking device. If the Commission acted arbitrarily, capriciously, or unfairly in deciding to use the oral component of the exam as the sole ranking device, then a fortiori the rights of the examinees have been unnecessarily trammeled.
The court therefore set a hearing to determine if the CSC's decision to reweigh the examinations was arbitrary and capricious.[3] The testimony received during the hearing establishes that the decision of the CSC was neither arbitrary nor capricious, nor was it an abuse of discretion.[4]
As a preliminary matter, the displacement of candidates from the eligibility lists does not constitute arbitrary action. Once this court recognized that the City had the right to take some action to remedy voluntarily the adverse impact stemming from the originally-weighted examinations, it became clear that the original eligibility lists could not be used. Thus, no matter what remedy the City chose, the rankings on the eligibility lists would change. By reweighing the examinations, the City salvaged the examinations and saved considerable time that would have been lost if new examinations had been constructed. Although displacement in great numbers occurred, displacement may have been considerably less than what would have resulted from new examinations.
At the hearing, the court heard testimony from four witnesses: two "raters"[5] of the oral component, Professor Andrew L. Comrey, a professor of psychology at the *1232 University of California, Los Angeles, and Raymond Wong, the Consent Decree Coordinator. The testimony from the witnesses established the following facts.
First, although the examinations were reweighed, all three components of the examinations were maintained. Cut-off scores originally set for the multiple choice component, the section which tested technical knowledge, remained the same. Those candidates for sergeant and assistant inspector who did not receive minimum scores of 50 and 55 percent, respectively, were not permitted to take the other two examinations. The cut-off scores were set approximately five points above the level of minimum competency. Twenty-five (25) Q/50 applicants and 23 Q/35 applicants[6] were screened out because of their failure to "pass" the multiple choice examination.
Second, the writing skills component, which was designed to test a candidate's ability to communicate effectively in writing, also was established as a screening device. A standard score of 60 was set for the writing skills examination which resulted in 130 candidates for the Q/50 and 160 candidates for the Q/35 being eliminated from consideration.[7]
Finally, the oral component was established as the sole ranking device. The oral examination consisted of a structured oral interview in two parts: (1) a ten minute presentation on a topic area reflecting the department's operating procedures and policies; and (2) one to three scenarios followed by questions from the raters. The ten-minute presentation focused on one of eight topics drawn from the SFPD's general orders and measured a candidate's oral communication and supervisorial skills. A five-minute question period followed the presentation. The scenarios, developed by subject matter experts, were designed to simulate situations which might be encountered typically by a sergeant or assistant inspector in his/her day-to-day work. The scenarios measured interpersonal skills.
Following the reading of the scenario, the raters[8] asked the candidates pre-designated, probing questions and were not permitted to ask follow-up questions. Both Lieutenant Ornelias and Captain Washington, two of the raters during the oral component, complained about the tightly-structured[9] format of the oral interview. They indicated that they had follow-up questions that they would have liked to ask the candidates. However, Professor Comrey, the POA's own expert witness, testified that the structured interview was more objective and enhanced the reliability of the examination.[10]
*1233 Thus, although the oral component originally was not intended to be the only ranking device, the evidence demonstrates that the oral component is capable of performing this purpose. Much more was tested via the oral interview than speaking ability. As mentioned, the oral component fairly measured supervisorial skills, oral communication skills and interpersonal skills. These were three of the five dimensions or skills and abilities identified by the job analysis as important in the successful performance of the job as both Assistant Inspector and Sergeant. The other two dimensions for the position of Q/50 and Q/35 were technical knowledge, which was tested through the multiple choice examination, and writing skills, which obviously was tested during the written examination.
After hearing all of the testimony and reading all of the evidence, this court believes that the promotional examination process was carefully designed and administered. The City met its dual goals of insuring a credible selection process and erasing the adverse impact resulting from the originally-weighed examinations. The court concludes that the CSC did not act unfairly, capriciously, nor arbitrarily in reweighing the examinations, especially since (1) the multiple choice cut-off score was set above the level of minimum competency; (2) a cut-off score was established for the writing skills component, thereby weeding out those candidates who could not communicate effectively in writing; and (3) the structured oral examination seemed to be a full, fair and credible manner of judging the three dimensions designed to be tested by the oral component.
The court, by its holding, in no way suggests that the selection process was the "best" manner the City could have used to promote candidates to the ranks of assistant inspector and sergeant. Rather, in reviewing the actions of the City and the CSC, the court merely approves the defendants' actions under an arbitrary and capricious standard. Therefore, the court finds that the rights of the examinees were not unnecessarily trammeled in the reweighing of the examinations.
Pursuant to Fed.R.Civ.P. 52(a), this Memorandum of Decision shall represent the court's findings of fact and conclusions of law.
The court holds that the reweighing plan adopted by the Civil Service Commission to remedy the adverse impact resulting from the promotional examinations was a permissible affirmative action plan under Weber. The court therefore dissolves its preliminary injunction previously issued in this case and orders the City to begin making promotions for the ranks of assistant inspector and sergeant from the eligibility lists derived from the reweighed examinations.
Let judgment be entered accordingly.
[1] Civil Service rule 5.07 prohibits the CSC from reconsidering matters previously decided by it more than 30 days after notice of the decision of the original vote. Here, the reconsideration vote was taken eight months after the issue had been decided and closed. Under the Supremacy Clause, the civil service rule gave way to the extent it impeded the operation of the Consent Decree.
[2] It is imprecise to imply that the rules were changed after the examinations were given. First, the candidates took the multiple choice examination before any weights had been assigned to the various components. Second, job announcements for both the Q/50 and Q/35 noted that "[t]he Consent Decree Division reserves the right to revise the examination plan in order to meet the requirements of applicable law and the Consent Decree." The Consent Decree Division merely asserted its reserved right.
[3] This was the only factual issue before the court. The POA originally had included a state claim alleging that the examinations, as reweighed, were not job-related. During oral argument on the motion for partial summary judgment, the court asked Mr. Saltsman, counsel for POA: "Do you have a pendant claim that you are wanting to prosecute on the issue of job validity?" Answer: "No, Your Honor." (R.T. February 28, 1985.) The POA therefore has withdrawn the issue of whether the examinations, as reweighed, were job-related.
[4] The court has reviewed the transcripts of the hearings before the Civil Service Commission and has found that most, if not all, of the evidence received during the court's hearing also was before the CSC.
[5] Lieutenant Gabriel Ornelias and Captain Edward Washington of the Los Angeles Police Department testified in court. In addition, the court received into evidence the deposition testimony of Captain Patrick Devlin of the New York Police Department.
[6] The POA contends that the multiple choice component should have been accorded greater weight in the selection process. In this regard, the court notes that the multiple choice examination tested knowledge of the codes that the candidates had been enforcing for the past three years. Perhaps then, it is unsurprising that the great majority of candidates exceeded the level of minimum competency.
[7] Three candidates who were on the original eligibility list did not make the revised list because they failed the written communications test.
[8] Each interview board consisted of three raters who were not members of the San Francisco Police Department. The raters were specifically chosen from other jurisdictions in order to eliminate favoratism and bias from the selection process.
[9] The entire oral examination was highly structured. The raters were told precisely which dimensions were being measured and the method by which a candidate would exhibit that dimension. The Consent Decree Examination Unit developed a "Rater's Report Form" which defined the dimensions for each classification, explained the scoring scale, and provided a framework upon which the raters could base their judgment as to whether a response was unsatisfactory, satisfactory, or good, relative to the specific dimension that was being measured. The raters were given these forms and encouraged to use them. Moreover, an eight-hour orientation and training program for the oral raters increased the chances that the raters would be consistent and accurate in their grading of the candidates.
[10] Professor Comrey also testified that, if the oral interview were to be used as the sole ranking device, at least 70 percent of the applicants should be eliminated on the basis of the first two components. The court does not accept his opinion for three reasons. First, as a general matter, Professor Comrey's expertise was in psychological testing and not employment testing. Second, and more specifically, he had never seen the data underlying the job analysis, had never read the scenarios, had not seen the tests and had not read the instructions to the raters. Finally, although he stated that an oral interview is unreliable and lacking in validity, some of the specific reasons he gave to support his statement were not applicable to this particular oral examination.