Participant training project for Europe (PTPE) -- monitoring and evaluation contract : final report

participant training in support of Support for East European Democracy (SEED) Act legislation. Interim evaluation covers the period 1991-12/94. The project is largely achieving its stated outputs, and in a timely fashion. As of 12/94, the project had trained 1,587 participants (39% of them female), of which 1,259 had completed training. The largest programs are in Poland, Bulgaria, Hungary, the Czech Republic, the Slovak Republic, and Romania. Training has been mostly technical (72%) rather than academic (28%) and has occurred in all three SEED Act priority areas (economic restructuring, democratic institutions, and quality of life), but mainly in economic restructuring (66% vs. a guideline of 50% for this area). Overall, training programs have been adequately managed, although in-country participant preparation varies substantially among contractors. Most programs provide more comprehensive orientation after participant arrival in the United States. Overall, 80% of participants felt prepared for their training. Participants were least satisfied with orientation services (advance notice of travel, transportation, and stipends). Some organizations were unable to recruit the anticipated number of participants or required more USAID assistance than anticipated. Exit questionnaires revealed that 84% of participants were satisfied with program quality and 80% with program content. Goodwill, Harvard, and U.S. Telecommunications Training Institute (USTTI) received the most consistently high ratings, with the Cooperative Housing Foundation, Home Builders Institute, Institute of International Education (IIE), and U.S. Information Agency (USIA) receiving generally above average ratings. Soros and the East Central Europe Scholarship Program (ECESP) received the most consistently low ratings, with State University of New York (SUNY) and William Davidson receiving generally low ratings. Overall, ratings for relevance were lower than for quality; 86% believed that training would be useful. The programs with the best overall value (quality and cost per participant month) appear to be those of Harvard, Goodwill, Johns Hopkins, and the IIE. The Soros Foundation"s program had the poorest balance of cost and quality. Returnee data to date are insufficient for assessing training impact. Some lessons learned are as follows. (1) Changes in project design or priorities might reduce the management burden and enable a greater integrated strategic focus on country development objectives. (2) The project"s diverse components do not lend themselves to common management and evaluation standards. The program has 22 distinct activities, some of which represent a significant departure from traditional participant training. In particular, distinct programs such as the USIA"s "topping-off" program should not be subject to standard management, data collection, or evaluation criteria. (3) The project"s institutional competition (cooperative agreements) component has been a mixed success. The management burden has been significant, and the program quality has ranged form very good to poor. The program should evolve to build on identified strengths and develop closer linkages to mission strategies. (4) The ECESP program is meeting otherwise unmet training needs, but should be more closely integrated with USAID mission strategic objectives and priorities. (5) USAID"s needs for monitoring and evaluation information and services should be carefully defined and incorporated into the monitoring services contract.