EdData II Task Order 15: Data for Education Programming in Asia and the Middle East (DEP/AME) Cambodia 2016 Khmer EGRA Strengthening and Pilot Report

The assessment aimed to provide MoEYS with an instrument for evaluating curriculum-based early grade literacy outcomes for students in grades 1-3. However, in 2010, MoEYS revised their primary-level Khmer textbooks to adopt a more phonics-based approach to instruction, returning to traditional methods of teaching and learning. As a result, concerns arose about whether the EGRA accurately reflected the learning expectations for early grade students. In 2015, the U.S. Agency for International Development (USAID) requested RTI International to assess the education sector and specifically review the 2009 Khmer EGRA. Following the assessment, USAID asked RTI to review and strengthen the instrument, including a pilot to ensure validity and reliability. The EGRA strengthening process began with a review of Grade 1-3 textbooks to reflect the realities of materials being used. A workshop was conducted in Phnom Penh in August 2016 to review the objectives and purposes of EGRA, critically examine the instrument, make necessary revisions, and conduct a field test to address any major issues with changes made. The workshop was led by RTI's Jessica Mejia, Jonathan Stern, and Keely Alexander, with support from Dr. Ok Amry of SBK Research & Development. The results of the pilot showed an overall strong instrument with high internal consistency and validity. The results also demonstrated a good grade progression across the majority of subtasks, as well as consistency across reading passages. However, some inconsistencies were found, particularly in the listening comprehension and dictation subtasks. The newly revised EGRA instrument is a significant improvement upon the original version, reflecting learning expectations from the current curriculum and adhering to EGRA best practices. However, several recommendations are necessary before using the EGRA at scale. These recommendations include removing or revising problematic items, which were identified through psychometric analyses, and reconsidering the use of listening comprehension passages, which proved unreliable and reduced the overall reliability of the assessment. The pilot results also highlighted the importance of revising problematic items, particularly in the familiar words, listening comprehension questions, reading comprehension questions, and dictation subtasks. Specific items that require revision are listed in the report. Additionally, the results suggest that the listening comprehension passages require significant revisions to improve their reliability and validity. The EGRA strengthening and pilot activities were conducted to ensure that the instrument accurately reflects the learning expectations for early grade students in Cambodia. The results of the pilot provide valuable insights into the strengths and weaknesses of the EGRA instrument, and the recommendations outlined in this report will inform the development of a revised EGRA instrument that meets the needs of the education sector in Cambodia.