An Example of Unfairly Changing the Rules
It's also something that plays out in adult life as well. I may have mentioned in past posts my experience as a faculty member at the University of Wisconsin-Milwaukee. At the time I was on the PhD faculty, MIS doctoral students were given two attempts to pass a written comprehensive qualifying exam. One student, well-liked by the faculty, had failed his first attempt.We advised him to take an extra semester to study for the exam, but he decided to take it again at his next opportunity. I was the tie-breaking vote with two pairs of senior faculty at odds with each other. One of the pairs was the informal MIS faculty lead (we didn't have a departmentalized business faculty) and the other happened to chair the PhD program committee. These faculty members tried to bias the student's score upward under our grading criteria, but the student still failed the exam by a significant margin. The area lead then told the other pair that they would have to tell the student that he had failed because the professor had already told the student that the latter could do his dissertation under him. The other pair wavered, and then the 4 faculty members, over my objections, decided to suspend the hearing; the PhD program chair called a meeting and introduced a policy change that allowed a conditional pass with agreed-upon remedy work. The grading meeting reconvened with the four faculty members, over my objections, elected to give him a conditional pass. In my judgment, that was manifestly unfair to other students whom had played by the rules and a violation of professional ethics by the four senior faculty members: they let their positive feelings towards the student outweigh their professional judgment. I personally liked the student, but he had to accept responsibility for his failed effort. The conditional pass rule was not in effect at the time the student wrote his exam.
[Postscript: At a national conference two or 3 years later, I was going through a pre-screen interview with another school in a setup interview area with tables and folding chairs; the topic of UWM came up and I mentioned this story. After the interview was over, I turned around and found myself confronted with my old area lead, whom had eavesdropped on the conversation. He tersely told me, "He doesn't have his PhD yet, does he!"]
A Brief Discussion of Test Construction and Validation
I have a special research interest in test construction. The reader may think that a teacher or professor would naturally have expertise in the area, but in fact I would apply rigor, intentionally attempting to sample course material of knowledge, skills and critical thinking. I was reading some lawyer mocking the concept of multiple-choice questions on firefighter or other exams. What you are trying to do is thoroughly cover the subject matter, and it's very difficult to do that in a limited testing period without the use of short-answer test questions. Writing a good multiple-choice question is an art, and I had a knack for writing good questions. (I sometimes used test-bank questions, depending on time constraints, but at least 85% of my test questions over my teaching career were original.) In fact, I often allowed students to bring in one cheat sheet (and you will not believe how small some students can write on a cheat sheet).
In writing a good test, ideally you want a number of discriminating questions, meaning that good students will get it right and poor students will not. For example, a question which seemed to draw random responses from good as well as bad students would get factored into any exam curve. The reason I mention this is the criterion that the EEOC often applies artificial standards for testing that can result in watering down exams (as was done in the City of Chicago) so almost everyone scores high. But if everybody scores high, it may simply reflect not testing higher-order knowledge or other factors. Just like nobody is doing a favor by giving an illiterate person a high school diploma, cities are not well-served where high-ranking fire officials only need to meet a minimum level of competence.
Why tests versus leadership or job ratings, etc.? Well, first, there's no reason you can't have multiple hiring/promotion criteria or factors, just as the Supreme Court described for university admissions. But why do you have standard aptitude tests (e.g., SAT, GRE, GMAT, LSAT, etc.)? Because grades can widely vary among schools and even among teachers within schools. The standard tests provide a common objective baseline for comparing performance. There can be significant variability among subjective ratings, and some feel that that can be a back-door approach to rationalize a de facto promotion quota.
The EEOC Rule and Standard Tests
The EEOC implements a fairly liberal guideline (meaning there's a fairly low threshold to trigger an accusation of "cultural bias"):
In writing a good test, ideally you want a number of discriminating questions, meaning that good students will get it right and poor students will not. For example, a question which seemed to draw random responses from good as well as bad students would get factored into any exam curve. The reason I mention this is the criterion that the EEOC often applies artificial standards for testing that can result in watering down exams (as was done in the City of Chicago) so almost everyone scores high. But if everybody scores high, it may simply reflect not testing higher-order knowledge or other factors. Just like nobody is doing a favor by giving an illiterate person a high school diploma, cities are not well-served where high-ranking fire officials only need to meet a minimum level of competence.
Why tests versus leadership or job ratings, etc.? Well, first, there's no reason you can't have multiple hiring/promotion criteria or factors, just as the Supreme Court described for university admissions. But why do you have standard aptitude tests (e.g., SAT, GRE, GMAT, LSAT, etc.)? Because grades can widely vary among schools and even among teachers within schools. The standard tests provide a common objective baseline for comparing performance. There can be significant variability among subjective ratings, and some feel that that can be a back-door approach to rationalize a de facto promotion quota.
The EEOC Rule and Standard Tests
The EEOC implements a fairly liberal guideline (meaning there's a fairly low threshold to trigger an accusation of "cultural bias"):
A selection rate for any race, sex, or ethnic group which is less than four-fifths (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact.
Sailer claims that he has recalibrated some race-based means data on major graduate-level standard exams (GRE, GMAT, MCAT, LSAT, DAT) using a 50% percentile for white applicants and found (roughly) blacks at a comparable 13.5% and Hispanics at roughly 25%. If we used the four-fifths rule, the standard exam results would be considered "evidence of adverse impact". Sailer elsewhere argues that the controversial results under the Ricci case are comparable in distribution. He points out that fire departments all over (e.g., Chicago and New York City) have repeatedly run into similar exam distributions, despite hiring consultants to control for "cultural bias", to the point some cities (like Chicago and New Haven) had refused to administer subsequent exams for years. (Chicago then finally had a watered-down test administered.)
The Ricci Case
The City of New Haven is 40% black. The city charter specifically forbids race-based preferences, but obviously there would be adverse public reactions if it was seen that blacks were disproportionately denied promotion; effectively, the only way to practically respond to these activist demands would be to implement promotional quotas, which are unconstitutional, violating equal protection--which is essentially what Rizzo and others would be arguing in terms of reverse discrimination.
The Ricci Case
The City of New Haven is 40% black. The city charter specifically forbids race-based preferences, but obviously there would be adverse public reactions if it was seen that blacks were disproportionately denied promotion; effectively, the only way to practically respond to these activist demands would be to implement promotional quotas, which are unconstitutional, violating equal protection--which is essentially what Rizzo and others would be arguing in terms of reverse discrimination.
The city contracted with I/O Solutions to devise a race-neutral test. I/O Solutions designed exams that were 60% written and 40% oral. Thirty fire department managers across the country, two-thirds of them minorities, were brought in to score the oral exams. (25 white, 8 black and 8 Hispanic). The exams, for captain and lieutenant, were administered in late 2003.They got sign-off on source materials for the exam from the black assistant chief.
The city had 7 captain promotions and 41 applicants. The top 7 applicants included 6 whites and 1 Hispanic. At least 5 white and up to 2 Hispanics could be selected under the original rules (it allowed a choice of 3 candidates for each position). The highest ranking black was #16. (The individual scores/ethic classification are available here.)
The city had 8 lieutenant promotions and 77 applicants (43 white, 19 black, 15 Hispanic). The top 13 were white (followed by 3 blacks).
There doesn't seem to be evidence of any systematic bias; it just seems to be a case blacks didn't score high enough to place, despite attempts to facilitate minority input into the exam. The distribution could simply reflect individual differences, not cultural differences, in test preparation. For instance, dyslexic Ricco studied almost 8 hours daily for 3 months before taking the exam.
The city refused to certify the results given hostile reaction from local civil rights leaders to news that no blacks would be among the 15 promotions.
The city effectively threw out the exam claiming racially disparate outcomes and cancelled the promotions claiming no discrimination took place because no one got promoted over the 15 qualifying firefighters. (In the interim, the city has gotten around things by establishing temporary "acting lieutenants" and "acting captains". I do not know if the acting positions were filled exclusively from the top 15 exam scorers. I suspect not, because the local civil rights leaders would probably see that as a work around.)
Cancelling the test results/promotions because some blacks didn't score at the top of the grade distribution, qualifying for promotion, seems arbitrary. It's another thing to argue, e.g., blacks did not have access to the same study materials, did not take the exams under the same conditions (e.g., time limits), or were given different, harder exams. There is no evidence I/O Solutions short-shrifted its methodology in devising a race-neutral exam. A number of blacks scored in the top 50% of the exam distribution and a number of whites scored in the lower 50%. But we don't hear, for instance, of cancelling bar exams or medical board exams because of "disparate outcomes".
Frank Ricci and others sued on the basis of reverse discrimination. The argument that several of the Clinton appointees (including Justice Ruth Ginsburg ) advanced--that there was no discrimination because no promotion event occurred is manifestly circular and absurd. THEY LOST THEIR PROMOTION SIMPLY BECAUSE OF THEIR SKIN COLOR. Start the game over again? To the people that played by the rules and lost and have nothing else to lose, they get a do-over. Simply because of the winners' skin color, which is a factor beyond their control.
Yes, there was discrimination--discrimination against those whom played by the rules and won their promotions fair and square. In any case, over 50% of the test takers were white, and under a racial quota, most of those positions would have gone to whites. Holding back the results simply because blacks didn't make the cutoff constitutes a de facto race-based preference for blacks, which is unconstitutional, as Justice Anthony Kennedy pointed out.
You can't change the rules just because you don't like the way the results played out. In my view, the key question of dicrimination is NOT guaranteed outcome, but on having a fair chance to compete under the same rules. Canceling the exam was unjust for those whom played by the rules and won their promotions, like Frank Ricci. You can't keep testing people until you finally get race-based results you like and then stop the game.
The district court judge, a Clinton appointee, Janet Arterton Bond, ruled against Ricci v [New Haven Mayor] DeStefano. I'll simply quote an amicus brief from the National Association of Police Organizations, which argued:
Among the clearest of rules emerging from the last three decades of this Court’s Equal Protection jurisprudence is that, without exception, “all governmental uses of race are subject to strict scrutiny.” See Grutter v. Bollinger, 539 U.S. 306,326-27 (2003)...The lower court [Judge Bond] squarely found that the Civil Service Board’s refusal to certify the exam results was motivated by the race of those who performed best....The lower court, however, did not apply strict scrutiny.The failure to subject the Board’s decision to strict scrutiny by itself constitutes reversible error. But the lower court’s decision should also be reversed because a straightforward application of strict scrutiny shows the Board’s decision cannot endure...In any event, regardless of what Title VII requires, that statute cannot trump a public employer’s obligations under the Equal Protection Clause.
The Circuit Appeal panel, including Judge Sonia Sotomayor, gave a perfunctory review, until Judge Jose Carbanes, like Sotomayor a Clinton appointee, red-flagged the case, which eventually resulted in yesterday's reversal. Still, you would have thought Judge Sotomayor would have been more familiar with other precedents, including Biondo v City of Chicago, and especially the relevant discussion in the circuit appeal by Judge Easterbrook:
Still, the premise of the City's argument is that regulations supply a compelling governmental interest in making decisions based on race. How can that be? Then Congress or any federal agency could direct employers to adopt racial quotas, and the direction would be self-justifying: the need to comply with the law (or regulation) would be the compelling interest. Such a circular process would drain the equal protection clause of meaning.Some Post-Decision Reflections
I think certainly Judge Sotomayor's due diligence and yet another reversed decision by the Supreme Court need to be taken into consideration by the Senate. My position on Sotomayor remains unchanged: I am opposed to her confirmation (but I am also opposed to a filibuster). I do not like the way she has introduced identity politics into her judicial philosophy and I'm concerned whether she is short-shrifting the equal protection clause. I normally would be willing to consent to the President's choice, even with my misgivings of a nominee's judicial philosophy. But her infamous public quotes regarding the inherent superiority of a Latina judge in contrast to white male judges seems to me to be but the tip of the iceberg of self-serving, muddled thinking in a profession where words are carefully parsed for meaning.
The Obama Administration's decision to back the losing (DeStefano) side of the lawsuit is not surprising, given the fact that Obama has repeatedly been more concerned with outcome versus process and has espoused nontraditional criteria, e.g., empathy. But Robert Gibbs' insistence that the case proved that Judge Sotomayor followed precedent is pure chutzpah. The idea that a potential frivolous lawsuit against the city of New Haven justifies it to violate the Equal Protection Clause of the Constitution is NOT justified by precedent--and if Judge Sotomayor believes it does, then she has no business not only being on the Supreme Court but on the Circuit itself.
