Final Report: Ukrainian Standardized External Testing Initiative (USETI) Legacy Alliance GDA Project

The alliance united 17 partners, including the Ukrainian Center for Education Quality Assessment (UCEQA), to strengthen UCEQA's capacity to develop and administer tests that meet international standards. The USETI Alliance program conducted activities in four areas: Area One, Reinforcing and building UCEQA's test development and operational capacity; Area Two, Securing the legislative basis for testing and HEI admissions, and institutionalizing the partnership between business and education; Area Three, Building public support and fostering growth of the Alliance; and Area Four, Facilitating informed test preparation demand and a professional test-prep industry. Area One activities focused on strengthening UCEQA and its nine regional testing centers (RCEQAs) in their ability to provide the full range of core standardized testing services to hundreds of thousands of Ukrainian HEI applicants annually. Through "next level" training in item and test development, the USETI Alliance imparted on UCEQA a skills set and institutional capacity to independently develop and administer the full range of tests needed for merit-based, fair admissions to HEIs with enhanced quality and ensured reliability between test forms. The USETI Alliance provided UCEQA senior management with an appropriate roadmap of recommendations over the 2010-2012 term for building its in-house item and test development capacity. A team of international consultants, each a professional in a certain testing area, provided technical assistance (TA) to UCEQA. The consultants included Markus Broer, Zarko Vukmirovic, Steven Bakker, Mark Zelman, Iwa Mindadze, David Ziegler, Algirdas Zabulionis, and Christina Pendzola-Vitovych. The USETI Alliance ensured that UCEQA/RCEQA teams developed and implemented their products informed by leading-edge testing techniques and up-to-date methodologies. To enhance test quality, the USETI Alliance international consultants provided TA focused on intermediate-level item writing and test development workshops. Six sessions of extensive workshops were provided for more than 100 professionals, including item writers in 13 subjects, RCEQA item writers and subject specialists, UCEQA subject specialists, the UCEQA Item Bank Department, and UCEQA management representatives. The USETI Alliance improved the quality of items generated by the UCEQA system and continued development of an industry-standard item bank. As part of TA in Year One, the USETI Alliance purchased "FASTTest" software to support industry-standard item bank development for UCEQA. Subsequent missions of international consultants to UCEQA continued training and support of IB development. Item banking is crucial to multi-form testing in particular, and mastery of this step is an essential incremental development in UCEQA capacity. The USETI Alliance also supported scoring, scaling, and reporting of scores. International consultants ensured that by the end of project implementation, UCEQA was working independently in accordance with industry standards. Throughout the project implementation, international consultants focused on creating capacity of UCEQA to engage independently in and conduct inter-rater reliability studies. Particular efforts of international consultants were applied to develop a manual for those who mark open-ended items that included practical methodologies and could be implemented to improve reliability.