Midterm evaluation : developing Mayan-based health care for rural women and children -- cooperative agreement no. 520-A-00-97-00060-00 : Project Concern International

Mid-term evaluation covers the period 9/97-9/99. Project Concern International (PCI) is the implementing agency. To date, PCI has established strategic alliances with six Mayan NGOs, five of which are currently receiving funding, training, and TA to improve their institutional and technical capacity to deliver health services. The first year and a half of the project were devoted to NGO identification and strengthening and to NGO hiring of personnel (usually nurses) to manage service delivery and work with community volunteers (usually educators). The project's Mayan partner, Rxiin T'namet, also conducted introductory training for NGO staff. Clinical services began in March 1999; the NGOs working with PCI currently cover a population of 67,369. The project has a number of strengths. PCI's Guatemala office has good relations in the field and especially with Ministry of Health (MOH) leadership in departments where the project is active. It also has strength in management and administration and in some promising technical areas, including community pharmacies. The project also has a good baseline and a new project director who has personal credibility with USAID partners and Guatemalan health officials in the MOH and in rural areas. Overall, however, the project has not performed as expected, having failed to make significant progress in any of the intermediate results (IRs) and lower-level results (LLRs) specified in the agreement. The project currently has little programming specific to the community or household, access to clinical services has not increased significantly, and there are no indicators to suggest that quality of services has improved. The project has also been unable to report much progress on most indicators; even those for which data exist fall far below the anticipated annual targets. These findings are disappointing, especially in light of the project's size and budget. The reasons for the lack of progress are complex. A number of operational and management problems during the first 2 years had a strong negative impact on the project. Two other factors also contributed significantly: PCI assumed that its successful, long-term relationship with Rxiin T'namet constituted a model replicable with other Mayan organizations and that this replication was feasible as a strategy to meet program objectives. Probably neither was accurate, for reasons both particular to the PCI-Rxiin T'namet partnership and to the characteristics of Rxiin T'namet as an organization. Additionally, the PCI country team restricted itself when it limited counterparts to women-oriented and women-managed grassroots Mayan NGOs in indigenous rural communities. These decisions had long-range implications for performance. Even if PCI had been successful in identifying and strengthening 10 NGOs, as it had originally proposed, the project could never approach the coverage goals described in the proposal, nor could it make much progress on the agreement's results and indicators. PCI's current NGOs still require major investments in creating or upgrading technical skills and are still far from the competency levels necessary for providing quality health services, even in small populations. The evaluation provides options which PCI can use to set priorities and restructure activities to achieve results within the remaining time frame and budget. These include continuing the current program with more realistic expectations; modifying the original strategy to increase coverage while retaining the focus on Mayan NGOs; expanding the program to focus on service delivery; and expanding activities in prevention and family health, while retaining some clinical and administrative activities. (Author abstract, modified)