Midterm evaluation of the quality assurance project [QAP] (cooperative agreement no. DPE-5992-A-00-0050)

The evaluation covers a period ending 6/94. As originally conceived, the project emphasized research aimed at adapting QA and total quality management (TQM) methods to developing country settings. As the project evolved, however, implementation of QA became the major activities. Five long-term institution building projects were carried out, including national efforts in Chile, Nigeria, and Jordan and pilot efforts in Niger (a single district) and Egypt (a single hospital). In general, these projects -- which have adopted a more participatory, "bottom up" approach than is customary in the standard TQM and PRICOR "systems analysis" approaches -- have been dramatically successful in implanting QA approaches into the thinking of health systems managers. Less positively, however, nongovernment health services and facilities have been largely ignored (except in Niger), and more work is needed in such areas as establishing linkages with donor agencies. Nine short- and medium-term TA activities were undertaken: (1) in Guatemala, work with 5 hospitals on issues of client satisfaction and cost effectiveness; (2) in Costa Rica, a national QA course in preparation for a national program; (3) in Guatemala, Ecuador, and El Salvador, development of QA activities in relationship to cholera control; (4) in Malawi, work to improve the quality of child survival activities being carried out by community health workers; (6) in Burkina Faso, Kenya, and Niger, collaboration with two other USAID central projects in improving measles control; (7) in the Philippines, improving child weighing in collaboration with vitamin A work; (8) in Egypt, improving micronutrient related activities; and (9) in the Philippines, improving TB treatment compliance. While these activities were generally effective and useful (except for the measles control effort in Niger, where lack of coordination interfered with effectiveness), there is some concern that project resources might be better applied to more extensive QA efforts. There are numerous examples of QA activities generating concrete improvements in service quality, including: (1) an increase from 9% to 100% of case records showing correct assessment of respiratory rates in acute respiratory patients in Chile; (2) a reduction from 25% to 11% of the rate of infection following Caesarean sections in a Guatemalan hospital; (3) improvement in the stocking of operating room supplies in a hospital in Egypt; (4) a reduction in waiting lines and a concomitant increase in fee revenue in a hospital in Egypt. Training has been a core element in almost all activities and a key to success. Visits by project staff as a routine follow-up to initial workshop training have also been critical to success, and should be given greater emphasis. In addition, given that training curricula and materials require extensive local adaptation to be effective, the project should shift curriculum adaptation largely to the field where collaboration with local staff can be maximized. Other training efforts included a course at Johns Hopkins University (JHU) and development by JHU and the Academy for Educational Development of an interpersonal communication course in Honduras and two computer-assisted training packages. While no longer the focus of the project, methodology development is still an important and appropriate component, especially since the project's TQM approach is substantially different than the systems analysis approach of its predecessors. Activities in this category have included the preparation of seven monographs or manuals and nine field studies. The publications have been of mixed quality; some are quite good, but as a group they seem to address different audiences and would benefit from greater systemization. The studies which were chosen covered appropriate topics, but health economics issues might have received greater emphasis. Tension between implementation of QA versus the research agenda, with its additional demands, has interfered with research. For the remainder of the project, documentation of the project's activities, dissemination of results and syntheses, and activities to market the project and its successes to donor organizations and country officials should receive greater attention.