January 2, 2011
The PISA Rankings and the Role of Schools in Student Performance on Standardized Tests
The PISA Rankings and the Role of Schools in Student Performance on Standardized Tests—Posner
International comparisons are tricky, as Becker points out, but the PISA (Programme for International Student Assessment), which tests 15-year-olds for proficiency in reading, math, and science by well designed standard tests conducted in thousands of schools all over the world, is a careful and responsible program, the results of which deserve to be taken seriously. The latest results (which are for 2009) reveal among other things that although the United States spends more money per student on secondary school education than any other country except Switzerland and Austria, Americans' performance on the PISA tests is mediocre. In the latest tests Americans ranked 17 in reading, 24 in science, and 30 in math. 15-year-old kids in East Asian nations (including Australia and New Zealand), along with Finland, Switzerland, the Netherlands, Belgium, and Canada, outperform the United States in all three subjects. Since 2000, when the PISA tests were first given, the United States has fallen in rank in reading and science, and is unchanged in math.
The rankings tend to be interpreted as measures of the quality of a nation's pre-collegiate school system (primary and secondary education, since primary education influences performance in secondary schools). But this may be a mistake. Schooling is only one, though doubtless an important, input into performance on the PISA tests. Another is IQ. There have been some efforts to compare IQ across countries, notably by Richard Lynn and Tatu Vanhanen; see their 2006 book IQ and Global Inequality. Their results cannot be regarded as definitive, given significant limitations in the data, but they are suggestive. The authors find that the East Asian countries, which generally rank highest on the PISA tests (including reading—not just math and sciene), have the highest average IQs; the average IQ of Americans is lower because of our large black and Hispanic populations, which have lower average IQs than whites and Asians.
IQ is understood to reflect both genetic endowment and environmental factors, particularly factors operative very early in a child's life, including prenatal care, maternal health, the educational level of the parents, family stability, and poverty (all these are correlated, and could of course reflect low IQs of parents as well as causing low IQs in their children). The case for very early intervention in children's development, powerfully urged by the distinguished University of Chicago economist James Heckman, can be understood as an effort to lift IQs in the black and Hispanic communities and by doing so improve the educational performance of black and Hispanic children, including performance on the PISA tests. It is true that Heckman emphasizes noncognitive skills that facilitate learning, but these skills could also increase performance on IQ tests, indicating a positive effect on IQ.
The 2009 PISA test scores reveal that in American schools in which only a small percentage (no more than 10 percent) of the students receive free lunches or reduced-cost lunches, which are benefits provided to students from poor families, the PISA reading test scores are the highest in the world. But in the many American schools in which 75 percent or more of the students are from poor families, the scores are the second lowest among the 34 countries of the OECD; and the OECD includes such countries as Mexico, Turkey, Portugal, and Slovakia.
If IQ is playing a significant role in America's mediocre showing on the PISA tests, improvements in secondary school education are unlikely to have dramatic effects. The white and Asian kids in American schools are already doing fine, for the most part; the black and Hispanic kids may not do much better until their early childhood environment is improved to the point at which black and Hispanic IQs are raised significantly.
Analysis of the PISA results has revealed some other interesting facts. One is that higher teacher salaries dominate small class size as a factor in high PISA scores. This is a reassuring finding because it suggests that secondary school education can be improved at no net increase in cost, since higher teacher salaries are offset by larger classes—if class size is raised in proportion to increases in teacher salaries, there is no net increase in the school's cost, and there should actually be a reduction in cost in the long term because a reduction in the number of classrooms reduces the size and therefore cost of a school even if each classroom is larger. Another reassuring finding, in light of all the agitation over charter schools and voucher systems, is that private schools on average do not outperform public schools after adjusting for the quality of students upon entrance and that competition for students does not seem to improve average performance either. Of course these are generalizations across many countries and America's individualistic culture may not fit them.
This observation is especially pertinent to another finding in the PISA report, which is that poor kids do better in a school that has mostly middle-class kids. Our education system, both public and private, tends as a practical matter to be segregated according to family income and social class. This is a reflection of economic inequality, which is great in the United States and growing.
Becker points out that despite the imperfections of its educational system, America remains preeminent in innovation. This is important but it appears to be due in part to the nation's attractiveness to immigrants. Many of our innovators are foreign born; increasingly they are Asian. The United States, as a result of generous immigration policies in the nineteenth and early twentieth centuries, also has the largest Jewish population in the world—larger than Israel's, and a higher percentage of American Jews are Ashkenazi—that is, descendants of European Jewish immigrants. This is significant because Ashkenazi Jews have a significantly higher average IQ than other Americans, including (though the margin is small) East Asians, and, as important, a very strong cultural orientation toward high achievement in business, science, and intellectual fields generally. From the standpoint of innovation, a wide distribution of IQs is more important than the average IQ, because most innovations will come from persons with an above-average IQ, and in scientific and other technical fields from persons with a way-above-average IQ. Moreover, because of the bell shape (normal distribution) of IQ across persons, a higher average IQ translates into a much longer upper tail—the part of the distribution that contains the highest IQs.
If as I am speculating (and I emphasize that it is speculation), IQ is a major factor in school performance, we should hesitate to place too much weight on variance in educational investments, methods, etc. in explaining differences in that performance, relative to genetic and cultural factors and also (and importantly) to economic inequality.
Implications of International Comparisons of Student Performance-Becker
Results on standardized tests are becoming a common way to evaluate student performance in different school systems, such as in different states of the United States. These tests are especially important in international comparisons of student performance since school systems differ so much across countries. International tests in mathematics, science, and reading have been given to high school students in different countries every few years since about 50 years ago. The well-know PISA tests started in 2000, and the number of countries included has been expanded by 2009 to over 60.
Scores on standardized tests have many well-discussed problems from the viewpoint of measuring how much students learn. For example, teachers may concentrate on materials that are likely to be on a test rather than on other possibly more important knowledge- this is called "teaching to the test". Still, standardized tests are a much better way to compare achievements among students, and especially among students in different countries, than are various measures of average years of schooling- the most commonly used measures in international comparisons.
The US spends much more per student than any other country (with a few small exceptions). Yet from the very beginning, American students have generally performed below average compared to students from the richer countries that comprise the OECD countries. In 2009, the average American student did a little better on the PISA tests: they were much above students from the great majority of the 65 countries that participated, and were at, but no better than, the average level of OECD countries.
Another interesting case is that of Chile. Some critics have used Chile's below average performance on the PISA tests to oppose Chile's extensive use of school vouchers. Chile's performance may be disappointing, but in 2009 Chile did the best of all Latin American countries (Chile was a little behind Uruguay in math), and much better than Brazil, Argentina, and Mexico (except again in math where Chile did about the same as Mexico), even though Argentina has a much longer tradition of emphasizing education.
Japan and other Asian countries typically place much greater emphasis on rote learning and memorization than does the United States and some European nations. South Korea, Japan, and Hong Kong rank high on the PISA tests, much higher than the US. However, America is the world leader in game-changing creativity and innovations, far above these other countries. Perhaps America's lead is related in part to the emphasis that the American education system places on creative thinking rather than memorizing what famous people have said.
Russia provides another interesting example. Russia has produced many outstanding mathematicians and scientists, yet the Russian Federation ranks quite low on the PISA tests in mathematics and science.
Modern economies have enormous specialization by skill, type of industry, and various other characteristics. This implies that an economy can perform at a high level even though not everyone is well educated in mathematics, science, reading, or other fields. It is sufficient if enough young persons are prepared in these subjects to supply enough trained men and women for the jobs and other tasks that require specialized training. Such specialization implies that not only are the average scores for different countries relevant, but so too are measures of the distribution of scores among students within a country. Calculations by William Hubbard of the University of Chicago Law School for the 2003 PISA results do show that the US ranks somewhat higher relative to other countries when comparing the variances in scores among students in the same country than when comparing average scores.
That the distribution of scores is important does not deny that average scores may also matter a lot in modern economies that rely extensively on the labor force's command of knowledge and information. to test this hypothesis, a few economists have related average scores on various international tests in math and science, along with average years of schooling and other variables, to subsequent growth in per capita incomes. Eric Hanushek of the Hoover Institution pioneered this approach, but the latest study I know of is by Appleton, Atherton, and Bleaney, "International Test Scores and Economic Growth". They show that higher scores by different countries are followed in later years by moderately higher growth rates in GDP, holding constant initial per capita incomes, average years of schooling, and some other variables. Test scores are far more important than average years of schooling in predicting subsequent economic growth.
International comparisons are always somewhat suspect because differences across countries that are not observed by economists rather than the variables that are measured may be producing the results found. For this reasons, Appleton, Atherton, and Bleaney also relate changes over time in test scores within countries to subsequent changes in their economic growth rates. They find much weaker results in these within-country comparisons, which is not uncommon.
Despite all the qualifications, scores on standardized tests, like the PISA tests, are much more effective ways to compare student achievements in different countries than are average years of schooling and other measures of school years completed. As I indicated earlier, the mediocre performance of the average American student on the PISA and other tests may indicate only that the US school system emphasizes creativity and independent. However, I strongly believe that it is also evidence of the mediocre performance of many American schools. International test scores have been used for decades to push for improvements in the schools available to the lower third of American students through the spread of charter schools, school vouchers, and stronger incentive to teachers, students, and parents. The 2009 PISA test results indicate that some progress is being made in American schools, but considerable room for improvement still remains.