In 1974, physicist Richard Feynman derided education research as a form of pseudoscience.
He wrote: “I found (pseudoscientific) things that even more people believe, such as that we have some knowledge of how to educate. There are big schools of reading methods and mathematics methods, and so forth, but if you notice, you’ll see the reading scores keep going down. … There’s a witch doctor remedy that doesn’t work. It ought to be looked into: How do they know that their method should work?”
Four decades later, social scientists might have found an answer to Feynman’s question. That answer is field experiments. By applying education policies carefully, researchers can see the effect of those policies very clearly. In recent years, as concern over U.S. educational performance rose, more and more such experiments have been tried.
Finally, we’re getting some consistent results. Roland Fryer, a Harvard economist who specializes in education and racial issues, has collected the outcomes of 196 policy experiments, and found some consistent lessons.
Fryer, like Feynman, has harsh words for the state of education research before field experiments came along. Earlier studies either invoked “various assumptions, many of which are not verifiable,” or were “nonexperimental studies” whose results weren’t borne out in controlled environments. In the genteel language of today’s economics literature, this is as close as it gets to calling out pseudoscience.
For example, lots of previous studies suggested that smaller classes didn’t help students learn. But in 1999, a randomized experiment found that it does, in fact, often make a difference. Furthermore, small class size can be very helpful in helping black students close the gap with their white counterparts.
Fryer’s survey paper is intended to elevate the status of research by picking out the studies that use randomization to control for outside factors. He looks at interventions in schools, homes and early childhood education, focusing only on rich countries like the U.S. He examines educational performance — test scores and graduation rates — and future income as measures of success.
So what does Fryer find?
First, the big lesson: If you want to improve student outcomes, the school — not the community or the home — is the place to make changes.
Fryer writes: “Experiments in early childhood and schools can be particularly effective at producing human capital. … Interventions that attempt to lower poverty, change neighborhoods, or otherwise alter the home environment in which children are reared have produced surprisingly consistent and precisely estimated ‘zero’ results.”
This result is unlikely to please conservatives who focus on the need to “fix” families first. But it’s also going to annoy liberals who think that communities have to be improved, or inequality reduced, for education to improve. The evidence is mounting that school is really where the magic of learning happens.
So what do schools need to do? Fryer’s most consistent finding is that high-intensity tutoring — basically, extended one-on-one mentoring — is extremely effective. That’s true both for children and for adolescents, contradicting the claims of some education researchers who believe that only early childhood intervention is important. Give students a tutor, and they improve a lot. (Incidentally, that’s consistent with my own experience as a math tutor, years ago.)
That finding decisively rejects the gloomy assertions of those who claim that nothing works in education. Yes, something does work. Getting every poor black kid in the U.S. a personal tutor for his or her entire childhood is probably prohibitively expensive, but it’s nice to know that disadvantaged students are capable of improving their performance a lot when given sufficient help and attention.
What about charter schools and vouchers, the two main institutional reforms suggested by conservatives? The answer is that charters can work, especially if implemented in the right way, but that school vouchers basically don’t do anything. The latter result fits with other studies — at this point, we can probably say with reasonable confidence that vouchers are not the best approach to improving the U.S. educational system.
Charters are a trickier proposition. The literature essentially shows that charters are not effective for the average student, but are often very effective for poor and minority students. Charters might therefore be an important targeted tool for helping poor minorities close the gap. Fryer also reports that certain policies, such as frequent teacher feedback, can improve the performance of charters — and, probably, of other schools as well.
One important policy I wish Fryer had investigated is an increase in school funding. A bunch of recent research suggests that simply spending more on poor schools, especially on renovations and building new facilities, can be very effective in boosting student performance.
Nevertheless, Fryer’s paper is a gold mine for education policymakers, and anyone interested in school reform. Finally, education research is becoming more of a science than a pseudoscience. The answers we get from experiments may be less bold and confident than the answers we’d get from simply stating convictions or doing sloppy, compromised research. But in the end, if anything will lead us to truth, it’s careful science.
By Noah Smith
Noah Smith is an assistant professor of finance at Stony Brook University and a freelance writer for a number of finance and business publications. He maintains a personal blog, called Noahpinion. — Ed.
(Bloomberg)