mEducation Evaluation Abstracts

The publication draws on four research reports produced collaboratively under the Mobiles for Education (mEducation) Alliance, which aims to foster cost-effective and scalable uses of technology to support education in developing countries. The reports from which evaluation abstracts have been culled for this volume include Mobiles for Reading: A Landscape Research Review, 2014, commissioned by USAID and prepared by JBS International Inc., which includes information about 25 evaluations; Landscape Review: Mobile Education for Numeracy, 2014, commissioned by the German Federal Enterprise for International Cooperation (GIZ) and prepared by Garth Spencer-Smith and Nicky Roberts, Kelello Consulting, describing 8 evaluations; Landscape Review: Mobiles for Youth Workforce Development, 2013, commissioned by USAID and prepared by the Aguirre Division of JBS International Inc., including 23 evaluation descriptions; and Educational Technology Topic Guide, 2014, commissioned by the British Government's Department for International Development (DFID) and prepared by the Health & Education Advice & Resource Team (HEART), which describes 7 evaluations. Evaluations included in this volume include both performance and impact evaluations, which report on a number of outcomes relevant to ICT4E ranging from attitudes towards the technology to measurements of learning gains. The abstracts have been culled into groups based on the internal validity of their evaluation design, with the most robust examples including a comparison or control group mimicking the counterfactual. Studies with higher internal validity impact the ability of researchers to draw causal conclusions, which gives a clear picture of what works and what does not. The publication highlights the importance of impact evaluations for advancing learning about "what works" in the field of ICT4E. Of the 58 evaluations found in the ICT4E Landscape Reviews, 38 percent used "impact evaluation" methods to rigorously examine the degree to which reported outcomes can be attributed to ICT4E interventions. The publication also includes two impact evaluation case studies from among the abstracts included in this volume. The evaluation abstracts in this volume are presented in sections linked to four ICT4E Landscape Reviews from which they were extracted. To help readers locate evaluations by the type of education activity or evaluation design involved, the 58 evaluations found in these reviews have been indexed by both topic and evaluation type. Table 1 shows the frequency with which evaluation abstracts describe studies by type of activity and by evaluation design, while Table 2 identifies each of these same 58 evaluations by title and sponsoring organization, and provides the page number where it can be located. The publication also discusses the importance of impact evaluations for advancing learning about "what works" in the field of ICT4E. The evaluation design was determined by applying the following criteria: Designs Examining Causality, Designs Examining Change (No Causality), Pre-Post Designs, and Single Time Designs. The publication highlights the need for rigorous studies that document the effect size (or difference between pre and post intervention outcome measures for treatment and non-treatment groups) to make serious comparisons regarding the impact between two or more ICT4E interventions. The publication concludes by welcoming suggestions for improving this volume in subsequent iterations, as the authors begin the process of archiving evaluations for the education technology field in a manner that can be easily accessed by anyone who hopes to learn from these evaluations, or to plan a new one themselves.