Summative evaluation of the basic education consolidation project

Summative evaluation covers the period from 12/92 to 9/95, when the project was terminated, 18 months early. The MOE, with help from project advisors, has taken solid steps toward the project's objectives of an improved curriculum, better prepared teachers to teach the curriculum, a continuous assessment system, and useful feedback on student achievement. Progress is evidenced by new institutional processes and structures at the MOE's Curriculum Development Division (CDD), Examinations, Research and Testing Division, and Teacher Training and Development Department (TT&D). These processes and structures lay the groundwork for sustainable long-term educational change, address concerns and issues expressed in the 1993 Report of the National Commission on Education, and are consistent with USAID's strategic objective for education (to increase the level and relevance of what students learn, their receptivity to additional training, and their preparedness for further education). The year 1996 will show the collective impact of the processes and structures which the project has facilitated. In that year, a new junior secondary curriculum will be introduced; the first criterion-referenced certification tests will be administered, processed, and reported; and TT&D will have completed first wave training for all primary school teachers. The new 3-year junior secondary core subject syllabi will be introduced in January 1996, and CDD has plans and timelines for completion of addition syllabi and materials. Syllabi include coverage of work, gender, family life/population, HIV/AIDS, and environmental issues. CDD's curriculum development process receives guidance from high-level interdepartmental committees and from the "Curriculum Blueprint," a policy document developed under the project. The project has contributed to a comprehensive overall teacher training policy for Botswana. The teacher training curriculum has been tied to revisions and innovations in the school curriculum. Significant accomplishments include: development of a Management Manual for Teacher Training College, and of a diploma program in Primary Education at one Teacher Training Institute (with plans to expand to others); computer-literate staff at MOE headquarters and the College of Education; and plans for an Evaluation and Management Network. In the area of student assessment, significant accomplishments include development of a criterion-referenced Primary School Leaving Examination, which will be administered in 1996; a Criterion-Referenced Testing (CRT) training program in 41 pilot schools; training of 80 regional trainers in CRT, who then trained all Standard 5 teachers; and development of a feasible national CRT training program for all primary teachers. MOE staff were experienced and competent and project staff were all high-level experienced professionals. Things to be done were clear to the MOE and were done right technically. The prospects for sustainability are excellent. The lessons learned are simple and few. (1) Project managers need periodic formative evaluations. Summative evaluations often turn out to be either post mortem analyses of why things went wrong or irrelevant seals of approval. (2) There needs to be informative interaction among MOE, contractor staff, and USAID. USAID was not kept informed about the nature of work going on or the quality of staff support given. This was largely the result of not having periodic formative evaluations. Reference group meetings are never enough. Day-to-day informal talk is needed to clarify technical issues and solve misunderstandings. MOE and project advisors should meet informally and on an as-needed basis with the USAID Project Manager. The notion that the Chief of Party alone deals with USAID is clearly inappropriate for this type of project. No one person has that much technical ability. (3) In a project such as this, all short-term consultancies must serve identified and expressed needs of MOE units. The end of the project saw a rush of consultancies, some of which appear to have been shaky. (4) When a systematic change as extensive as the one undertaken in this project is attempted, the public and especially the staff and professionals in the system must be informed early about changes that will affect them and their lives. If this is not started early, confusion results and anxieties grow which can impede success. CRT, for example, is a simple concept, yet teachers and even teacher trainers appear confused about it.