Monitoring Report: The Quality Assessment Methodology, December 2015

The program was implemented by USAID ROLISP, with the goal of improving the quality of training and evaluation processes at the NIJ. The program's monitoring report, prepared by Luciana Iabangi, Team Leader/Judicial Training Specialist, highlights the progress made in institutionalizing the training program evaluation and the use of evaluation feedback to improve the NIJ training during July-December 2015. The NIJ staff, with support from USAID ROLISP, developed a list of amendments to the Quality Assessment Methodology in September 2015. The amendments modified certain provisions of the methodology to reflect the experience gained during its implementation in 2014. The NIJ's Council approved the amendments, which included the monitoring of trainers during training by the NIJ Vice-Director, NIJ's Chief of Training and Research Division, and other NIJ staff appointed by the NIJ Vice-Director. However, the NIJ management decided not to provide trainers with feedback from evaluations conducted during October 2014-July 2015 due to questionable data, as only one NIJ employee conducted the evaluations. In accordance with the NIJ quality assessment methodology, the assessment of the initial training program is performed by two methods: trainees rate the trainer's performance using an electronic questionnaire, and the chief of the Training Methodology and Research Division attends lectures and seminars to monitor trainers' performance. Despite this, the NIJ did not perform the assessment of the initial training program in the first semester of 2015, as it initiated the assessment of the NIJ's trainers' network. The NIJ's In-service Training Division and Initial Training division did not organize training courses in July-August 2015 due to summer vacation. However, the division organized 20 courses in September 2015, with 11 of them evaluated using the paper-based evaluation method. The NIJ's IT staff discussed the electronic evaluation procedure using PC tablets provided by USAID ROLISP in October 2015, and developed a short guide for trainees on how to use the tablets. Despite this, the NIJ continued to use the paper evaluation system, with some courses evaluated on paper and others using the PC tablets. The NIJ's In-service Training Division organized 35 courses in November 2015, with 10 of them evaluated using the PC tablets provided by USAID ROLISP. The division organized 26 courses in December 2015, with 6 of them evaluated using the PC tablets. However, the initial training quality assessment was suspended pending the outcome of the complex evaluation procedure, and the NIJ did not assess the quality of initial training in the first semester. The NIJ started using the PC tablets provided by USAID ROLISP to collect data electronically, but some trainees completed the electronic questionnaires superficially, and the NIJ was slow in implementing the electronic evaluation system. The conclusions drawn from the monitoring report indicate that it is unclear whether the NIJ will restart assessing the quality of initial training and how it will apply the amended methodology. Without feedback to the trainers, the methodology's intended purpose of ensuring an efficient mechanism for improving trainers' performance and the quality of initial and in-service training at the NIJ was not achieved. The planned follow-up for Component 2 includes continuing to monitor and report on the implementation of the methodology by the end of the USAID program.