Radio education project evaluation

The project is being implemented by Fe y Alegria, a Jesuit-run PVO, with TA from the Education Development Center (EDC). The project is broadcasting on a daily basis to an estimated 36O second-grade and 236 third-grade classes with an enrollment of approximately 20,000 in the Departments of Cochabamba, Santa Cruz, the Beni, Sucre, and Tarija. Children in grades 4 and 5 receive lessons in the health pilot. The schools receiving the broadcasts are a mix of public schools administered by Fe y Alegria and schools administered directly by the Ministry of Education. The plan is to produce lessons for one additional grade each year until 1991, when it is expected that lessons will have been developed and validated for grades 2-5. Thus far, only second-graders have been tested in both experimental and control groups. Effect size was used as a measure of the comparative achievement between control and experimental groups. For the first year of the project, an effect size of .909 was obtained in the second-grade, year-end evaluation, which indicates a gain significantly higher than in studies in other countries of the effect of radio, textbook provision, and teacher training. For example, the effect size for second-grade mathematics in a similar project in Nicaragua, from which the Bolivian lessons were adapted, was only .55. An effect size of .909 would mean that where the average control-group student would rank at the 50th percentile, the average radio student would rank at about the 81st percentile, a gain of 31 percentile points. The ten lessons which have been developed in the health pilot concentrate on diarrhea and dehydration, the largest cause of infant death in Bolivia, as well as on hygiene, clean water, and nutrition. Although the post-test results are not yet available, based on field observations a substantial gain in health knowledge is expected. Both the project and the evaluations conducted within it have been well designed and carried out. Throughout the project, problems have been addressed as they occurred, and there has been very little slippage in the production schedule. The intensive TA from EDC, which has had extensive experience with similar projects in other countries, appears basic to the development of quality lessons. The active transfer of technology, essential for sustainability, is apparent in talking with staff from the implementing organization. Many Fe y Alegria staff members are thoroughly familiar with the project and evaluation design and are learning day-to-day operations such as the SYSTAT Program used for data analysis. There are also many mechanisms for cost recovery which could be explored to increase sustainability of the project. Fe y Alegria staff have taken the lead in expanding the project, although they could probably do more to expand outside of Fe y Alegria administered schools. Still, they are well on their way toward the significant, A.I.D.-established goal of 25% coverage of enrolled children and appear to be strong enough to eventually administer the project on their own after TA has been phased out. As post-production implementation costs are only a small fraction of production costs, a reduction in A.I.D. funding should not have a significant impact on implementation at grades 2 or 3 or on the student achievement currently being realized at these grade levels. It would, however, have an impact upon expansion and the production of lessons for grades 4 and 5. A weak point, which may need to be addressed with further TA of a more specific nature, is financial management of the project. Recommendations address specific mechanisms for cost recovery, expansion efforts, and opportunities for collaboration with other projects. (Author abstract, modified)