#Education Week in Review: PISA-OECD, Testing

Students taking a standardized test.

Education Week in Review: PISA-OECD Reactions, and Growing Opposition to Standardized Testing

PISA-OECD Reactions

PISA-OECD releases Volume V results from 2012 tests.The Education Week in Review recaps the biggest stories of the week in education.

Every three years, a set of standardized tests is administered to students across the globe. These tests are administered by PISA (the Program for International Student Assessment) which is a program of OECD (the Organization for Economic Cooperation and Development).

PISA’s most recent tests were administered in 2012. It has been releasing results from those tests in so-called volumes. Volume V was released last week, so PISA is back in the news.

Results from the Creative Problem Solving Section

The results from Volume V revealed that American students fared poorly on the creative problem-solving section of the tests.

Faring poorly on PISA tests has become a trend for American students. And when the U.S. fares poorly, a common refrain from education reformers is, “We need to emulate the high-scoring countries.” After the 2009 tests, for example, the target country was Finland.

We have found two rebuttals to the “we need to emulate other countries” refrain. They were both in the news this week.

Emulating Other Countries Doesn’t Make Any Sense

The first rebuttal is this: emulating other countries doesn’t make any sense.

At the blog Ordinary Times, Vikram Bath argues that it’s unrealistic to copy part of one education system and apply it to another. Education systems are just too complex. They evolve over time based on culture, politics and economics. The U.S. could not successfully emulate Finland even if it found the political will to do so. As an example, he points to the fact that Finland has a poverty rate of 3% whereas the U.S. has a poverty rate of 20%: it’s apples and oranges. Instead of looking internationally for examples, he says, we need to look closer to home, like to the Commonwealth of Massachusetts. If the U.S. had scored as high as Massachusetts did, the country would have jumped an average of 19 spots in reading, science and math.

At Mother Jones, Kevin Drum agrees: trying to emulate Finland would be fruitless. Although Drum likes Bath’s Massachusetts suggestion, he argues that it’s not perfect because of the commonwealth’s poor track record with minority students. Drum points out that the U.S. will soon be majority-minority therefore we need a system that does well with minority students. To this end, he puts forth New Jersey, a state that does very well with minority students, which make up 40% of the population.

So that’s the first rebuttal: emulating other countries doesn’t make any sense.

Methodology of the PISA Tests Is Flawed

The second rebuttal to the “we should emulate other countries” refrain is this: in some cases the methodology of the PISA tests is flawed.

At the Brookings Institution, Tom Loveless dissects the methodology that led Shanghai to top the math rankings. He exposes Shanghai’s exclusion of certain students and chastises PISA-OECD for putting forth Shanghai as an example for the rest of the world.

At Real Clear Education, Daniel Willingham questions what the creative problem-solving tests are actually testing. Students were tested on how to solve problems they might encounter in real life, like operating an MP3 player or finding yourself on a map. Although practical examples such as these do require problem-solving skills, Willingham makes the case that they are not actually the skills that matter. The skills that matter — and the ones we should be teaching our children — are those needed to achieve a goal when the process for doing so is unknown. These skills are not tested by familiar examples.

On his blog, Yong Zhao says that some of PISA’s claims are misleading. Zhao is in the middle of a series, actually, called “How Does PISA Put the World at Risk.” Last week he released Part 4 of the series, “Misleading the World.” (His argument is long.) Essentially, he says that students in the U.S. are learning very important skills that have real value in the future economy, and that we would be foolish to try to emulate another country based on PISA results. “Everyone can be forced to memorize Hamlet, but it is unlikely to force anyone to invent the iPhone.”


After all that, where does it leave us? Here’s the mind-meld: although the U.S. has fallen behind globally and our education system needs improvement, there are some American states doing a good job of educating their students and if we want to improve as a country we should look to those states for our answers.

Growing Opposition to Standardized Testing

A number 2 pencil.Now we shift from an international context to a domestic context.

Standardized tests are the tests taken by all public school students across a state for the purposes of comparison and evaluation. (They are entirely separate from subject-matter tests.)

Standardized tests are under fire in the U.S. from a growing minority. The recent crescendo is in part due to the fact that “testing season” began last week.

One indicator of dissatisfaction is the growing visibility of the opt-out movement. Another indicator is the popularity of the website Testing Talk, where educators can share their observations of standardized tests (i.e., air complaints).

Current Testing Riddled with Problems

It seems the opposition has reason to be upset. Testing — at least in its current form — appears to be riddled with problems. Here are five of them:

  1. The tests are poorly constructed.
  2. The tests are not evaluating some of the skills our students will need in the future like communication, collaboration, critical thinking and creativity — so-called 21st century skills. Therefore these skills are not being developed in schools as much as they should be, if at all.
  3. The tests do not provide helpful feedback to teachers.
  4. The tests do not provide helpful feedback to families. (In fairness, standardized tests were not designed with families in mind. However, given the amount of time spent on test-prep, it’s understandable that families would expect some actionable feedback.)
  5. The role of testing in the evaluation of districts and teachers is too significant (“outsized influence,” “high-stakes testing”). This causes two problems. First, it sets up perverse incentives: districts and teachers spend an inordinate amount of time prepping (because it’s in their self-interest) which means the students are actually losing valuable learning time. Second, the pressure stresses out the kids unnecessarily.

So the opposition is armed with good reasons.

Independent Efforts Across the Country

Although a few organizations are trying to unite the opposition (e.g., United Opt Out and iRefuse) the movement at the moment does not appear to be cohesive. Efforts are organized independently across the country, often on the school level or the district level. Some districts are agreeing to take the tests but protesting loudly and writing public letters. Other districts are agreeing to take the tests but working with elected officials to change the laws. Some parents are taking independent action and opting their children out of the tests entirely.

Standardized Testing Does Have Value

Despite the different efforts, there appears to be general agreement across the opposition that some form of standardized testing has value. Here are three reasons:

  1. Looking at cross-sections of the data helps officials identify important patterns, like the gap in achievement between whites and minorities.
  2. It compares schools across districts and states so administrators can benchmark their achievement.
  3. It helps schools chart their progress over time.


So, if parents and educators are unhappy with the status quo but acknowledge that testing has value, where do they go from here? We see two prescriptions:

  1. Improve the tests. This will require resting control from the learning companies.
  2. Develop alternative assessments of districts and teachers, i.e., add additional methods of evaluation. This will reduce the relative importance of standardized tests so they’re not the be-all, end-all. As a result, it will eliminate the perverse incentives that currently exist.

More Agreement Than Disagreement

In the coverage, you may find hostility among the opposition. For example, Michelle Rhee, the school choice advocate and former DC schools chancellor, is a lightning rod. However, in spite of the hostility, we found far more agreement amongst the opposition on the substance than disagreement. Although parents and educators do disagree on the “how” (protest vs. opt-out vs. write letters vs. take the tests), they largely agree on the “what” (they want effective standardized testing balanced with alternative assessments) and ultimately the “why” (they all just want what’s best for the kids).

Featured photo is “Taking a Test” by Renato Genoza licensed under CC BY 2.0. Two gentlemen is “OECDs ministerrådsmøte” by Statsministerens kontor licensed under CC BY 2.0. Pencil is “#2 Pencils” by vanhookc licensed under CC BY 2.0.