
    680 F.2d 248
    Lawrence A. SCHMID, Appellant, v. Robert FROSCH, Administrator, National Aeronautics and Space Administration (NASA).
    No. 81-1884.
    United States Court of Appeals, District of Columbia Circuit.
    Argued March 12, 1982.
    Decided June 15, 1982.
    Joseph B. Scott, Washington, D. C., with whom Irving Kator, and David H. Shapiro, Washington, D. C., were on the brief, for appellant.
    John H. E. Bayly, Jr., Asst. U. S. Atty., Washington, D. C., with whom Charles F. C. Ruff, U. S. Atty., Washington, D. C., at the time the brief was filed, Royce C. Lam-berth, Kenneth M. Raisler and Whitney M. Adams, Asst. U. S. Attys., Washington, D. C., were on the brief, for appellee.
    Before MacKINNON and MIKVA, Circuit Judges, and WILSON CO WEN  Senior Judge, United States Court of Claims.
    
      
       Sitting by designation pursuant to 28 U.S.C. § 293(a).
    
   Opinion PER CURIAM.

PER CURIAM:

Appellant Schmid brought suit in the district court under the Age Discrimination in Employment Act, 29 U.S.C. §§ 621-634 (the Act), alleging that his former employer, the National Aeronautics and Space Administration (NASA), had subjected him to discriminatory treatment, discharge and retaliation on account of his age. The district court on cross-motions for summary judgment granted NASA’s motion, having concluded that there were no issues of material fact in dispute on any of the claims of refusal to place Schmid as a meteorologist, of improper reduction in force, or of retaliation. 515 F.Supp. 1260. We conclude that summary judgment was improvidently granted and therefore remand the case for further consideration of the disputed issues of fact set forth below.

. I.

Dr. Schmid, who was 49 years old at the time of the disputed actions, worked as a GS-13 at the Goddard Space Flight Center (the Center) from 1958 until his separation in 1978. On July 3, 1977 his position as a GS-13 physicist was abolished when the Theoretical Studies Group of the Sciences Directorate at the Center was disestablished. Nine of Schmid’s professional colleagues were given reassignments within the Sciences Directorate. Schmid was the only one who was not reassigned; he was placed in a holding position for a possible transfer. In late July of that year Dr. Schmid was recommended by officials in the Applications Directorate (Dr. Atlas and Mr. Theon) for a transfer to a meteorologist position — an area of considerable interest to him. A position description was prepared containing Schmid’s classification as a “GS-1330 space scientist.” On August 4, 1977, Dr. Merideth, the official responsible for Dr. Schmid’s potential transfer to a meteorologist position, disapproved of the transfer on the ground that Schmid was not the strongest candidate for the position — presumably because Schmid was trained as a theoretical physicist and not as a meteorologist.

On August 13, 1978, a younger Ph.D. named Uccellini was appointed to a GS-1340 meteorologist position. Dr. Schmid maintains that the position to which Dr. Uccellini was appointed was essentially the same one he was denied. NASA contends that the position Uccellini filled was created in 1977 in connection with a 1976 talent search and was distinct from the position denied Schmid.

The Center held a center-wide reduction in force (RIF) on November 29, 1977 that eventually resulted in Dr. Schmid’s discharge in January 1978. Dr. Schmid claimed that this RIF affected older workers protected under the Act more harshly than younger workers and he presented statistics intended to substantiate his claim. The district court rejected Schmid’s statistics not only because the number of employees holding grades GS-11 or above who were separated was only eleven, but also because it determined that Dr. Schmid’s presentation of raw percentage comparisons was inadequate for a showing of either statistical significance or gross disparity.

The district court correctly found that small samples are of lesser value than large samples but acknowledged that any deficiency might be overcome by a showing of statistical significance or gross disparity. Contrary to the district court’s determination that Dr. Schmid did not proffer a showing of statistical significance, the record suggests that, his submissions to the Department of Transportation and the district court contained useful analysis which amounted to more than mere raw percentage comparisons. See Plaintiff’s Memorandum of Points and Authorities in Support of His Opposition to Defendant’s Motion for Summary Judgment. (J.A. 231, 232-234.)

Dr. Schmid’s analysis focused on the alleged disparate impact of the RIF on those employees at the Center of grade GS-11 and above. The crucial statistic, and the one emphasized by the district court, is that eleven employees of grade GS-11 or above were separated as a result of the 1977 RIF. Dr. Schmid conceded in a submission to the district court that he had “made no use of the pure separation statistic (3 out of 11 [i.e., 3 of the 11 grade GS-11 or above employees who were separated were less than 40 years of age]) simply because of the small sample size.” (J.A. 233.) Dr. Schmid also conceded that the probability that out of 11 separations only three or fewer would be younger than 40 years of age is one in 7.56. (J.A. 232.) The district court cannot be faulted for concluding that this statistic standing alone does not constitute a showing of statistical significance or gross disparity.

In an apparent effort to increase the sample size so that his analysis would be more reliable, Dr. Schmid focused on both those employees who received RIF notices and those who were “hurt” by the RIF, either in terms of downgrading or separation. Of the 151 NASA employees who received RIF notices, 70 held GS-11 or above positions. Of the 70 who received notices, 31 were actually hurt — 20 were downgraded and 11 were separated. Only 13 of the 70 employees who received RIF notices were under 40 years of age. Dr. Schmid alleged that it was highly unlikely that only 13 of the 70 employees who received RIF notices would be under age 40. (J.A. 233.) Theoretically, in an “age neutral” RIF the expected number of employees under age 40 to receive notices would have been 34. Only five out of the 31 employees who were “hurt” by the RIF were under 40 years of age. In an age neutral RIF the expected number would have been 15. Dr. Schmid alleged that the probability that only five of the 31 employees who were “hurt” in the RIF would be under age 40 is also highly unlikely. Of course, none of these mathematical computations take competence and qualification into consideration.

Following the RIF, Dr. Schmid as a long-tenured employee was entitled to “bump” or displace employees of less tenure in positions for which he was qualified. Schmid claims that he was qualified to bump a less senior employee and would have displaced him but for the fact that the other employee was insulated from displacement by his enrollment in the professional intern program, a development and training program designed for new professionals below the journeyman or GS-13 level. The professional intern program is separated from the rest of the workforce so that the next professionals may advance in their career tracks without being displaced. Schmid claims that the intern program violates the Act because it is used as a sham to shield young Ph.D. scientists from displacement— predominantly at the expense of older employees.

II.

It is well settled that summary judgment should only be granted if the pleadings, depositions, affidavits and admissions filed in the case “show that there is no genuine issue as to any material fact and that the moving party is entitled to a judgment as a matter of law.” Fed.R.Civ.P. 56(e); see Poller v. Columbia Broadcasting System, Inc., 368 U.S. 464, 467, 82 S.Ct. 486, 488, 7 L.Ed.2d 458 (1962); National Life Insurance Company v. Silverman, 454 F.2d 899, 908 (D.C.Cir.1971).

Based upon the record before us on appeal, we find that there is manifest evidence of disputed issues of material fact. Compare Defendant’s Statement of Material Facts to Which no Genuine Issues Exist (App. 296-308.) and Defendant’s Reply to Plaintiff’s Opposition to Defendant’s Motion for Summary Judgment (App. 390-404.) with Plaintiff’s Memorandum of Points and Authorities in Support of His Opposition to Defendant’s Motion for Summary Judgment. (App. 231-245.) Accordingly, we set aside the District Court’s grant of summary judgment in favor of Appellee' and remand the case to the District Court to consider the following disputed issues of fact:

(1)the legitimacy and effect of NASA’s professional intern program, with special emphasis on determining whether Ph.D. level scientists are receiving bona fide training and development in the program;

(2) the similarity of the position description for Dr. Uecellini to the position description for Dr. Schmid;

(3) the extent to which the recommendation expressed in the 1978 study on NASA’s Aging Work Force to “over RIF and hire young” was directly or indirectly adopted by NASA and was applied to Schmid’s detriment;

(4) the extent to which Dr. Schmid’s analysis about the 70 employees of grade GS-11 or above who received RIF notices and about the 31 employees who were hurt by the RIF, when considered in conjunction with the critical statistical evidence about the eleven employees of grade GS-11 or above who were separated, are probative of Schmid’s age discrimination claim;

(5) the extent to which the alleged instructions from Dr. Merideth to Drs. Atlas and Dressier to exclude Schmid from logging any hours in the meteorological area might have prevented him from establishing his qualifications for a meteorological position;

(6) whether the procedures followed in convening the panel of experts to review Dr. Schmid’s qualifications for a meteorological position and the subsequent actions of the panel in conducting its evaluation complied with the requirements of the Civil Service Commission’s Amended Educational Requirement; and

(7) if the procedures followed in connection with the panel of experts are found to have deviated from the prescribed regulations, then whether such deviation constituted a reprisal for Dr. Schmid’s filing of an age discrimination claim;

The district court upon remand may also consider any additional material issues of fact that it may find to be in dispute.

So ordered. 
      
      . By specifically referring to employees who hold positions of GS-11 or above, the district court implicitly recognized (J.A. 9.), and we agree, that the relevant class of employees for comparison with Dr. Schmid are the professional-level employees, i.e., the employees at grades GS-11 and above.
     
      
      . “Statistical significance” is a term of art that indicates the likelihood of an association between long-term results of a defendant’s selection (or discharge) process and minority (or age) group status. D. Baldus & J. Cole, Statistical Proof of Discrimination 288 (1980) [hereinafter Baldus],
     
      
      . The district court stated:
      In the 1978 RIF of which Schmid complains, only eleven employees holding grades GS-11 and above were separated. Such a small sample detracts from the value of statistical evidence. Mayor of Philadelphia v. Educational Equality League, 415 U.S. 605, 620-21 [94 S.Ct. 1323, 1333-1334, 39 L.Ed.2d 630] (1974). While this defect could be overcome by a showing of statistical significance, see, e.g., Hazelwood School District v. United States, 433 U.S. 299, 309-11 n.14 and n.17 [97 S.Ct. 2736, 2742-2743 n.14 and n.17, 53 L.Ed.2d 768] (1977) (standard deviation analysis endorsed), or by evidence of gross disparity, International Brotherhood of Teamsters v. United States, 431 U.S. 324, 339-40 n.20 [97 S.Ct. 1843, 1856-1857 n.20, 52 L.Ed.2d 396] (1977), neither are present in this case. The raw percentage comparisons presented by Plaintiff are inadequate for a showing of statistical significance. And Plaintiff does not indicate that the effect of the RIF on older employees was grossly disproportionate to the distribution of older employees in the employee population as a whole. The statistics presented do not raise an inference of discrimination.
      (J.A. 9.) (footnote omitted).
     
      
      . Small samples tend to be less reliable than large samples because of instability and variability caused by unrepresentative samples, measurement error, random selection, and random shocks from unknown or unknowable factors. Baldus at 298-299. This leads to the conclusion that “small samples provide a less reliable basis for making an inference about the treatment of two groups than do larger samples of decisions.” Id. 1981 Supp. at 61. However, small samples should not be rejected altogether. This is particularly true in the employment discrimination context where sample sizes are frequently small. Tests of significance and confidence intervals are specifically designed to estimate the extent to which the ■ reliability of inferences are threatened by small samples. Id.
      
     
      
      . The government conceded that agencies have more control over which employees receive RIF notices than they do over subsequent downgrading and separations. Defendant’s Reply to Plaintiffs Opposition to Defendant’s Motion for Summary Judgment. (J.A. 390, 392.)
     
      
      . The figures Dr. Schmid employed in the Plaintiffs Memorandum of Points and Authorities in Support of His Opposition to Defendant’s Motion for Summary Judgment, supra, are slightly different than those used by the government and the district court. For example, Dr. Schmid’s analysis was based on the premise that 68 rather than 70 employees at the GS-11 and above levels received notices in the 1977 RIF. Accordingly, Dr. Schmid further assumes that 30 rather than 31 of these employees were actually hurt by the RIF — 12 rather than 13 downgraded and four rather than five separated. Because this marginal difference was neither contested by the parties nor addressed by the district court, we will likewise disregard the discrepancy.
     
      
      .The provisions of the Age Discrimination in Employment Act do allow employers to make employment decisions “where the differentiation is based on reasonable factors other than age.” 29 U.S.C. § 623(f)(1).
     
      
      . In remanding this case to the district court to reconsider Dr. Schmid’s statistical evidence regarding those employees who received RIF notices or who were “hurt” by the RIF, we make note of the Supreme Court’s language in International Brotherhood of Teamsters v. United States, 431 U.S. 324, 339-40 [97 S.Ct. 1843, 1856-1857, 52 L.Ed.2d 396] (1977), which, although it was given in a Title VII context, is relevant here:
      Statistics are equally competent in proving employment discrimination. We caution only that statistics are not irrefutable; they come in infinite variety and, like any other kind of evidence, they may be rebutted. In short, their usefulness depends on all of the surrounding facts and circumstances, [footnote omitted]
     
      
      . Dr. Schmid claims in his brief that “the principal adverse action taken was the convening of a supposed panel of experts to judge whether plaintiff was qualified as a meteorologist and the issuing of a report to OPM that he was not so qualified.” Brief for Appellant at 67. If indeed the “convening” of the panel was the “principal adverse action,” then there is little merit to his position because the decision to convene the panel was made in April of 1978, at least three weeks before Dr. Schmid raised his age discrimination claim on May 12th. Presumably having understood Dr. Schmid to be complaining of the “convening” of the panel, the district court made an express finding that the evaluation conducted by the panel and its subsequent submission of a report were not retaliatory because the decision to hold the evaluation preceded the filing of the age discrimination complaint.
      Dr. Schmid’s unartfully stated claim can be reformulated: All of Schmid’s allegations of retaliation concern the way this panel functioned rather than NASA’s earlier promise that it would be convened, a promise that Schmid apparently believed would lead to his vindication. We believe that this point deserves consideration by the district court. We emphasize, however, that the proper focus is on how the panel functioned as measured against the requirements set forth in the Amended Educational Requirement. Dr. Schmid complains, for example, of lack of notice, no opportunity to address the panel, and an improperly constituted panel. Brief for Appellant at 67. These arguments presume a right to notice and hearing and a right to have experts selected who have no affiliation with NASA. The district court should determine whether the controlling regulations which provide for review of qualifications by a panel of experts extend such rights. His rights may not be that extensive.
     