Audit of AID"s evaluation program

Internal audit covered the period 1968-8/86 and was based on review of 43 evaluation reports from 6 Missions. At the time of the audit, AID/CDIE, Bureaus, and Missions were attempting to improve the evaluation process in order to satisfy the requirements of a management information system mandated by a 1968 amendment to the Foreign Assistance Act (FAA). Substantial improvement in evaluation management is nevertheless needed at both the Mission and Bureau levels. First, the poor quality of evaluations - the result of unclear Agency guidance regarding FAA requirements and inadequate management controls - has made it difficult for A.I.D. management to use evaluations for decisionmaking. In particular, evaluations often did not compare actual with planned project results, as required by the FAA. Other deficiencies are: purpose and methodology statements were absent or unclear; conclusions were not adequately supported or were difficult to locate; recommendations were not supported, readily identifiable, or actionable, or were often insignificant or repetitive of action underway; and lessons learned, when presented, provided little guidance to A.I.D. project planners. As a result, at least $3 million of the estimated $10 million spent annually by USAID"s did not achieve the intended benefits. Second, the extensive use of contractors to perform evaluations was not economical or effective and took the place of internal evaluation and monitoring. Reports were, as a result, often of low quality and primarily used for purposes other than those established by the FAA. A.I.D. could save an estimated $2.5 million through a more selective use of outside contractors while improving its internal management system. Agency management agreed with significant portions of the report, and consented to: implement 8 of 9 recommended quality standards and specify when the use of contractors to conduct final evaluations was appropriate; and to provide guidance on using analytical techniques, monitoring the implementation of recommendations, and identifying and reporting evaluation cost. However, unresolved differences remain concerning: one quality standard (measuring progress against established indicators), the use of outside contractors for interim evaluations, the underlying purpose of evaluation, and the need for Assistant Administrators to advise USAID"s to implement recommendations pending revision of the A.I.D. handbok.