M-CAFE: MOOC Student Feedback with Collaborative Filtering

The platform allows students to numerically assess their own performance, provide suggestions for course improvement, and rate others' suggestions on a weekly basis. M-CAFE employs statistical methods and Collaborative Filtering (CF) to identify valuable suggestions and provide weekly reports to instructors. Student engagement and feedback are crucial for effective teaching, particularly in MOOCs where massive class sizes hinder instructors from obtaining valuable information. M-CAFE aims to address this issue by collecting and analyzing data from students. The platform was evaluated with two EdX MOOCs, CS 169.1x and CS 169.2x, to test the following hypotheses: 1. CF can sort textual suggestions near expert human performance. 2. The quantitative feedback trends agree with course events. A total of 560 students provided 83 suggestions for CS 169.1x and 132 suggestions for CS 169.2x. Peer-to-peer ratings on textual suggestions resulted in 1,691 and 3,564 ratings for the two courses, respectively. The quantitative analysis of quantitative analysis of topics (QATs) between weeks revealed unusual behavior in CS 169.2x, primarily due to increasing difficulty in homework. The unusual behavior was attributed to the top-rated suggestions, which included more examples of well-written Rspec and Cucumber tests or some recommended answers from tutors. The textual suggestions aimed to answer the question of how the course can be enhanced to make it more valuable for students. The suggestions were sorted using Wilson score based on peer-to-peer CF. By evaluating the top-rated comments on novelty, topics covered, and quality, it was demonstrated that M-CAFE is capable of identifying valuable suggestions, addressing the scale issue of qualitative feedback. Examples of CF-identified top-rated and lowest-rated comments in CS 169.2x included more examples of well-written Rspec and Cucumber tests or some recommended answers from tutors, and changing the grading policy to let all HW be submitted late as in Part 1 of the course, respectively. The current implementation of M-CAFE is being used in a regular class, IEOR 170 at UC-Berkeley, to compare M-CAFE performance in various settings. The platform has demonstrated to be extremely helpful and engaging to use, with students thinking it is an engaging feedback tool. Future versions of M-CAFE may integrate into MOOC platforms to ensure feedback can be quickly analyzed and utilized by instructors. The M-CAFE platform allows students to track their rating history and compare to the class average, facilitating student engagement and feedback. The platform also provides weekly reports to instructors, enabling instructors to analyze and utilize feedback from students. The future development of M-CAFE aims to integrate into MOOC platforms, customize the platform to individual courses, and add new features to facilitate student engagement and feedback. The M-CAFE platform has been sponsored in part by the CITRIS Data and Democracy Initiative at UC Berkeley and Fushitsu Corporation. The platform has demonstrated its effectiveness in identifying valuable suggestions and providing weekly reports to instructors. The future development of M-CAFE aims to integrate into MOOC platforms, customize the platform to individual courses, and add new features to facilitate student engagement and feedback. The M-CAFE platform has been evaluated with two EdX MOOCs, CS 169.1x and CS 169.2x, to test the following hypotheses: Examples of CF-identified top-rated and lowest-rated comments in CS