Thursday, March 5, 2009

RCTs in education

Esther Duflo has an article in NBER’s quarterly newsletter Reporter about how randomized controlled trials (RCTs) were used to find out what works and what does not work in education sector reform in the developing countries. She discusses the RCTs done in India and Kenya to find out what requires (what does not) for high quality learning (good in heterogeneous student setting), lower teacher absentee, and re-empowering parents (not much effect) in the education reform process. She argues that a likely case would be that the government could be better in getting the schools work better for the poor (rather than giving ownership to local communities or parents). Strange finding given the fact that huge amount of resources are spent by multilateral donors in handing over responsibility of managing schools to local communities in countries like Nepal and India.

Does better access to inputs (textbooks, teachers) affect school outcomes (attendance, test scores) — and if so, by how much? The motivating theoretical framework was very simple, but the results were surprising. For example: Glewwe, Kremer, and Moulin found that lowering the student-textbook ratio from 4 to 2 had no effect on average test scores. Banerjee, Jacob, and Kremer found that halving the student-teacher ratio also had no effect on test scores.

These negative results prompted new reflection on the barriers to education in poor countries: If simply providing inputs does not increase the quality of education in poor countries, then it must be necessary to change the organization of teaching in schools, both the pedagogy and the incentives faced by students and teachers. This led to a new round of field experiments motivated by the general question: Can changing the organization of teaching in schools affect education outcomes? For the most part, these more recent projects have varied more than one factor at a time in different experimental groups, making randomization a powerful tool for examining the role of incentives, spillovers, and other key questions in the economics of education.

Together, a series of randomized evaluations of education programs in developing countries have taught us something about how education in developing countries can be improved: focus teaching on skills students need to progress further; find ways to motivate teachers. Neither of these is necessarily an easy, ready-to-implement prescription. Much more work is needed to develop programs that can achieve these two objectives on a large enough scale, especially given the political economy of education in developing countries. While neither suggests plug-and-play prescriptions, they do give us ample direction about where to search.

What’s more, these experiments have also taught us something about how to search, how we can learn about learning. Each experiment answers some questions and asks new ones; the next study builds on the previous one, progressively suggesting a model of education which is ready to be enriched over time.