Zambia quality assurance program final evaluation

The amount and quality of QAP's work is impressive. In only 5 years, a small team of senior staff built a QA structure and capacity throughout the country, generated enthusiasm for QA, and initiated teamwork on quality of care issues by health providers. However, there is no official document describing Zambia's QAP, and despite attempts to establish links with other entities, QAP remains isolated in its efforts to introduce a QA methodology. Developing the "Integrated Technical Guidelines for Frontline Health Workers" (ITG) was an excellent initiative, but did not result in anticipated gains. Neither QAP nor health providers were formally involved in its development. The QAP approach to setting standards consisted of training about 300 staff to use the Dynamic Standard Setting System (DySSSy). Those trained said it helped them develop their district or health center action plans, but the impact of the training on quality remains undocumented. The ITG communication strategy suffered major delays, and very few health workers know it exists. The district staff usually consider its format not user-friendly and its content overly complicated for some. Health workers are unlikely to use existing guidelines during a consultation. The extent to which they would use any kind of job aid is unclear. In Zambia, no formal instrument is used to directly observe health worker performance, nor is there is a formal strategy to differentiate causes of poor performance; hence, supervisors cannot distinguish whether problems are caused by incompetency or something else. In-service training may be seen as a solution when a lack of knowledge and skills is not the actual cause of poor performance. Good quality is not formally recognized and rewarded, but health staff would be receptive to a formal system for doing so. Of 127 health centers, 26 had active QA (or "problem-solving") teams. The main reasons why teams do not form seem to be the small number of staff at a health center, the lack of regular visits by the coach, and the absence of a QA-trained officer in charge. Key reasons why some teams do not successfully complete a first problem-solving cycle are the complexity of the problem and failure to follow the steps of the cycle. The relevance and importance of the problems chosen varied. Waiting times was often chosen, partly because it was an example used in training. About half the teams chose physical and facilities problems, and about 40% chose clinical problems important for the health of patients or community (e.g., malaria, late antenatal booking). About 5% of problems were identified by soliciting user views or by input from neighborhood health committees. In general, specific features of the problem-solving process, such as problem prioritizing and cause-and- effect diagrams, were used correctly. The use of methods for listing data sources and for gathering and interpreting data could have been better, and about 70% of teams could not show or describe any data collected. Many teams had problems implementing solutions, usually because of resource restraints. The results of the teams' work were assessed through measurable changes documented or reported. Five of the 26 active teams achieved measurable changes in quality, and eight others reported doing so, a claim in some cases confirmed by the evaluation. Also evaluated in the report are national support systems for QAP, specifically, QA training, the QA coaches/link facilitators network, documentation and reporting of QA activities, and supervision of QA activities. Important general findings include the following: (1) Support systems, such as coaching and QA training capacity, must be well established at the district level to ensure the continuity of the teams. (2) In a decentralized system, the QA program should first target the districts in order to promote team ownership for QA activities. (3) Numerous factors influence the productivity of the problem-solving teams. The tools developed for this evaluation (difficulty index and team failure index) proved useful. (4) A detailed documentation system of the QA program could help in monitoring the QAP's impact and in making adaptations.