Science Progress | Where science, technology, and progressive policy meet

PISA Test Scores and the Mathematics of Inequality

PISA report coverPolicy makers are responding to reports that students in the United States on average scored lower than their peers in 16 out of 30 other wealthy industrialized nations on an international science exam, predictably arguing that the U.S. performance on the test (the Programme for International Student Assessment) indicates that U.S. students cannot compete in the international workforce. But a recent analysis from the Urban Institute previously discussed on Science Progress suggests otherwise. Talking about “competitiveness” makes it easy to gloss over inequities in the educational system connected to race and class.

News outlets are eager to frame the results as U.S. backsliding in the international economy. The Washington Post quoted former Colorado governor Roy Romer: “How are our children going to be able to compete with the children of the world? The answer is not well.” The Associated Press likewise presented the ranking in a dim light: “In math, U.S. students did even worse — posting an average score that was lower than the average in 23 of the other leading industrialized countries.”

In the Urban report, Into the Eye of the Storm, Harold Salzman and B. Lindsay Lowell acknowledge that policy makers often cite the results from PISA and TIMSS, another international exam, “supposedly showing U.S. students lagging the performance of most other countries.” But using the results to make such sweeping comparisons “stretches the PISA far beyond its appropriate or even intended use.” They go on to make several critical points about the test.

Achievement varies significantly by socioeconomic class and race

The majority of U.S. students, who are white, “actually rank near the very top on international tests.” But minority and low-income students face obstacles to such achievement because of differences in the quality of educational systems and household income. Salzman and Lowell conclude: “The test results indicate that, rather than a policy focus on average science and math scores, there is an urgent need for targeted educational improvement to serve low performing populations, such as recent immigrants and some minorities.”

The diversity of the U.S. population both contributes to economic competitiveness and lowers the average score of students on the test

They point out that the United States “has a large population and the most diverse demographics of any industrialized nation,” and that averaging across such a mixed group of students ignores the size of the population and the distribution of student performance within that population:

What does one infer from comparing the average test score in a nation of over 300 million with that of a nation of 4.5 million (Singapore) or using educational performance as an indicator of economic performance? We would expect India’s 39 percent illiteracy rate and its secondary school enrollment rate of less than 50 percent (World Bank 2007) to make it an inconsequential global power. Of course, that is not the case because rather than average performance it is the small percentage of high performers in a nation of 1 billion that is the more important indicator of its relative science and engineering strength. The use of average rates across a diverse group of nations and diverse populations is of limited use in drawing conclusions about global standing economically or educationally.

The rankings are not a comparison of education systems

They quote the Organization for Economic Cooperation and Development: “If a country’s scale scores in reading, scientific or mathematical literacy are significantly higher than those in another country, it cannot automatically be inferred that the schools or particular parts of the education system in the first country are more effective than those in the second.” Rather, the test measures the cumulative impact of all factors effecting a student’s abilities in the subject areas, including their experience outside the classroom and at home. In the United States, both experiences are closely related to race and class.

The rankings do not indicate the magnitude of difference between average scores in each each country

Salzman and Lowell: “Without knowing the magnitude of the actual raw score differences on the PISA, we can use the test results to rank countries and populations but not know the importance of differences in rankings.”

Worries over the decline in STEM (Science, Technology, Engineering and Math) education propelled passage of the America COMPETES Act, which authorized $44 billion for research and education. More resources for better STEM education, properly allocated, is a good thing. But merely decrying a loss of competitiveness in U.S. students means willfully ignoring the inequities in our education system and in our economy. Smart science and education policy will close the gaps in each.

Tags:

Comments on this article

By clicking and submitting a comment I acknowledge the Science Progress Privacy Policy and agree to the Science Progress Terms of Use. I understand that my comments are also being governed by Facebook's Terms of Use and Privacy Policy.