Audit of the quality of results reported in USAID operating units" results review and resource request (R4) reports prepared in 1997

The audit was based on a statistical sample of 18 USAID operating units. Results data in R4s prepared in 1997 were not objectively verifiable, supported, accurate, complete, and/or validated. The auditors found problems with 252 of the 302 results reviewed (83%). These problems are due to deficiencies or weaknesses at the following three levels of organizations: (1) Operating units did not always follow, or were not successful in following, prescribed USAID policies and procedures for measuring and reporting on program performance. For example, 90 of the reported results were not objectively verifiable because operating units did not follow prescribed requirements for establishing objective indicators for which results could be objectively verifiable. Nor did operating units always assess data quality as part of the process of establishing performance indicators and choosing data collection sources and methods as prescribed. (2) The Bureau for Policy and Program Coordination (PPC) did not provide sufficient direction, support, and oversight in requiring operating units to follow prescribed USAID policies and procedures -- and federal requirements -- for measuring and reporting on program performance. For example, PPC did not have (a) a system to monitor whether operating units followed prescribed USAID policies and procedures in reporting performance data in their R4s, or (b) an action plan to correct identified problems. (3) Regional and central bureaus need to better carry out their assigned responsibilities for (a) reviewing and assessing the performance of each of their operating units in achieving those units" objectives and (b) managing the review of their operating units" R4 submissions. For example, these bureaus did not require operating units under their authority to specifically discuss in their R4s what the operating unit did to assess the reliability of reported performance data. Such performance measurement problems have been reported in previous audits (see PD-ABL-470 and PD-ABL-504 [June 1995] and PD-ABQ-274 [March 1998]). This audit report includes one recommendation to PPC, aimed at improving USAID"s internal controls to ensure that operating units report quality performance data to measure actual results against what was anticipated. These controls include better directions and oversight by USAID/Washington management. While PPC agreed that meeting the audit recommendation is highly desirable and that the recommendation can be met over time, the auditors could not determine whether PPC agreed to the specifics of the recommendation. Also, while PPC"s memorandum did provide helpful information on what corrective actions were being taken in some cases, in others the auditors were unable to determine exactly what specific actions were being proposed. Furthermore, PPC did not provide the dates for achieving the corrective actions other than observing that the recommendation can be met "over time". Since it is critical for USAID to implement the corrective actions on a timely basis, it is recommended that USAID implement these actions in regards to the R4s prepared in 1999. PPC did not address this part of the recommendation. (Author abstract, modified)