Jiangang Hao, Lei Liu, Jessica Andrews, and Diego Zapata-Rivera, Educational Testing Service
Alina von Davier, ACTNext, ACT Inc.
Art Graesser, Department of Psychology and the Institute of Intelligent Systems, University of Memphis

Contact address
Jiangang Hao
ETS Rosedale Rd., MS 02-T, Princeton, NJ 08541
Tel: 609-252-8472, Email:

Workshop description

Most of the existing studies on collaborative learning and assessments are designed to reveal important aspects or patterns of collaboration (Cohen et al., 1999; DeChurch and Mesmer-Magnus, 2010; O’Neil, 2014) based on small samples of participants (von Davier & Halpin, 2013). The items/tasks used in these studies are generally not standardized. The convenience sample and the non-standardized items/tasks often lead to questions of possible bias and reproducibility of the findings (Hao, Liu, von Davier & Kyllonen, in press). The goal of this workshop is to promote consensus on “standardized” practices for large-scale studies of computer-supported collaboration.

Large-scale collaborative activities require non-trivial technological infrastructure. Educational Testing Service (ETS) has made a major investment in developing a web-based platform to support collaborative learning and assessment (ETS platform for collaborative assessment and learning, EPCAL, Hao, Liu, von Davier, & Lederer, 2016). Different from most tools (such as Skype) that are designed to enable the collaboration, the EPCAL was designed to both enable the collaboration itself and facilitate the study of collaboration with additional team and task management system. This platform allows multimodal communication, template-based tasks uploading, and customizable real-time facilitation/intervention. Most importantly, it is integrated with a powerful data analytics support from ETS’s glassPy data analytics solution (Hao, Smith, Mislevy, von Davier, & Bauer, 2016). All these features make it an ideal environment to develop computer-supported collaborative research and assessment prototypes at large-scale.

In this workshop, we introduce the EPCAL platform to the participants via a set of hands-on collaborative learning/assessment activities. Participants will team up in dyads or triads on the EPCAL platform to complete these tasks collaboratively. We will facilitate discussions on common challenges and solutions regarding designing collaborative learning and assessment environments and how the EPCAL platform can be used to support various needs of such designs. The planned activities includes the following:

  • An overview of CSCL and the challenges
  • Introduction of the EPCAL platform with live demo
  • Selecting tasks/items from participants and deploy them to the EPCAL platform.
  • Participants will team up into dyads or triads to complete tasks hosted on the EPCAL.
  • Demo of real-time intelligent facilitation
  • Demo of collaboration process data analytics and visualization.
  • Collaborative discussions and feedback via the EPCAL platform.

We encourage participants to bring in their own tasks/activities for this workshop, please visit our workshop website for more details about our currently supported task types. We will discuss how participants can use the EPCAL platform for their future research studies on collaborative learning and assessment and how EPCAL platform may help to scale up their research work. In the discussion and feedback part, we welcome suggestions and comments for further improvements of the EPCAL platform.

The findings based on this workshop will be wrapped into a white paper. Participants are welcome to contribute to the white paper to be listed as co-authors. This white paper will focus on practical implementation of computer-supported collaboration at large scale.

Workshop website

The official workshop website is located at: You can find more details about the workshop and how to register on the webpage.

Participation procedure

Participants need to apply for this workshop by filling out a short form on the workshop website ( This application is for the purpose of counting the number of participants so as to arrange the corresponding logistics. We will send our confirmation of acceptance in early May. The participants need to officially register the workshop from the conference website.

Important dates for the workshop

April 28: Deadline for applications to workshop
Early May: Confirmation of acceptance for participation