Many of us learned in high school that the idea of mass, standardised, testing originated in Imperial China, with its civil service examinations. Ascertaining that everyone meets an agreed-upon minimum standard is such obvious wisdom that, over the next 1400 years, standardised testing spread around the world.
Not everyone is happy about it. Mostly the people who fail them: certain minorities and the big teachers’ unions.
A recent article published by the National Education Association (NEA) began by saying, “Most of us know that standardized tests are inaccurate, inequitable, and often ineffective at gauging what students actually know.”
Do ‘we’ know that, though? And, if history has taught us anything, it’s that just because everyone believes it, doesn’t make it so. Let’s critically examine the most common objections to standardised tests, like IQ, PISA, NAPLAN (in Australia) or SATs (in the USA).
Claim 1: Admission tests are biased against historically disadvantaged racial/ethnic groups.
Response: There are racial/ethnic average group differences in admission test scores, but those differences do not qualify as evidence that the tests are biased.
If one or more groups tend to do badly at standardised tests, there are two possibilities: firstly, that the tests are biased toward the successful groups; or, secondly, that the unsuccessful groups just aren’t, as a rule, very good at what they’re being tested for.
Guess which, in today’s Everyone Gets A Prize world, is the preferred explanation?
The differences themselves are not in question. They have persisted for decades despite substantial efforts to ameliorate them. As shown in the table above and reviewed more comprehensively elsewhere, average group differences appear on just about any test of cognitive performance – even those administered before kindergarten. Gaps in admission test performance among racial groups mirror other achievement gaps (e.g., high school GPA) that also manifest well before high school graduation.
Either all of the tests are biased, despite decades of trying to eliminate bias, or the group differences are just a brute fact.
Consider this: a basketball team recruitment test based on ability to score hoops. If, say, Asians mostly fail miserably while Blacks excel, does this mean the test is biased against Asians, or just that Asians as a rule just aren’t that good at basketball, for whatever reason? (For instance, average height, or just a general cultural lack of interest.)
Claim 2: Standardized tests do not predict academic outcomes.
Response: Standardized tests do predict academic outcomes, including academic performance and degree completion, and they predict with similar accuracy for all racial/ethnic groups.
Bear in mind that they don’t guarantee academic outcomes – an individual could be brilliant but simply lazy or nervously unsuited to the pressure of tests – but, in general, good scholars will pass tests. In any case, it’s a curious argument that a standardised test of academic performance doesn’t predict academic performance. It’s like arguing that testing the ability to shoot hoops doesn’t guarantee a high-scoring basketballer.
Standardized tests do predict academic outcomes, including academic performance and degree completion, and they predict with similar accuracy for all racial/ethnic groups.
Research confirming the predictive validity of standardized tests is robust and provides a stark contrast to popular claims to the contrary […]
Students of higher socioeconomic status (SES) do tend to score higher on the SAT and fare somewhat better in college. However, this link is not nearly as strong as many people tend to assume.
Which brings us to:
Claim 3: Standardized tests are just indicators of wealth or access to test preparation courses.
Response: Standardized tests were designed to detect (sometimes untapped) academic potential, which is very useful; and controlling for wealth and privilege does not detract from their utility […]
Standardized tests have a long history of spotting “diamonds in the rough” – underprivileged youths of any race or ethnic group whose potential has gone unnoticed or who have under-performed in high school (for any number of potential reasons, including intellectual boredom). Notably, comparisons of Black and White students with similar 12th grade test scores show that Black students are more likely than White students to complete college.
It’s commonly claimed that wealthier schools ‘teach for the tests’, that is, coach their students to pass, say, NAPLAN. In fact, systematic research shows that the effect of test-preparation programmes demonstrate limited gains.
Average gains are small – approximately one-tenth to one-fifth of a standard deviation. Moreover, free test preparation materials are readily available at libraries and online; and for tests such as the SAT and ACT, many high schools now provide, and often require, free in-class test preparation sessions during the year leading up to the test.
Teacher’s unions and activist groups regularly make another claim:
Admission decisions are fairer without standardized tests.
Response: The admissions process will be less useful, and more unfair, if standardized tests are not used.
In fact, quite the opposite is true. Without standardised testing, what are we going to rely on? The claim is that ‘teachers know their students best’, which may be true, but as Levitt and Dubner showed, teachers also cheat. Especially when having a high pass rate affects their careers. Moreover, teaching is a notoriously biased profession: overwhelmingly female and nearly completely left-wing. No room for personal bias, there.
Standardised tests are blind to anything other than academic ability. If your preferred interest group has a high failure rate, it’s not the test’s fault.
