Video Data Collection and Video Analyses in CSCL Research
Carmen Zahn1, Alessia Ruf1, and Ricki Goldman2
1 University of Applied Sciences and Arts, Northwestern Switzerland, School of Applied Psychology, Olten, Switzerland, carmen.zahn@fhnw.ch, alessia.ruf@fhnw.ch
2 New York University, NY, USA, ricki@nyu.edu
Abstract: The purpose of this chapter is to examine significant advances in the collection and analysis of video data in Computer Supported Collaborative Learning (CSCL) research. We demonstrate how video-based studies create robust and dynamic research processes. The chapter starts with an overview of how video analysis
developed within CSCL by way of its pioneering roots. Linked throughout the chapter are the theoretical, methodological, and technological advances that keep advancing CSCL research. Specific empirical and experimental research examples will illustrate current and future advances in data collection, transformation, coding, and analysis.
Research benefits and challenges that include the current state of understanding from observations of single, multiple, or 360° camera recordings will also be featured. In addition, eye-tracking and virtual reality environments for collecting and analyzing video data are discussed as they become new foci for future CSCL research.
Definitions & Scope
The particularity of rich video data compared to other data gathering methods in the Learning Sciences is that video data make both verbal and nonverbal social interactions in learning situations enduringly visible and audible to researchers. In this regard, video data differ from outcome data (e.g., quantitative data gathered in learning experiments systematically examining treatments and their effects), because they open the “black box” of collaborative learning processes. The scope of this chapter is to illuminate the scholarly understanding of existing and future methods for video data collection and data analysis in CSCL research in a practical fashion. The chapter maps past, present, and future innovative advances with specific examples selected to demonstrate the methods of video data collection and data analysis that Learning Science and CSCL researchers in a range of fields (e.g., Zheng et al., 2014) have been using for a better understanding of complex collaborative learning processes .
CSCL video methods span the entire spectrum of the social sciences (Brauner et al., 2018) which includes qualitative research methods such as case-based field work, and video ethnographic accounts, as well as quantitative methods such as experimental, and data-driven statistical research which includes learning analytics accounts. The
majority of CSCL research articles in the International Journal of Computer Supported Collaborative Learning (IJCSCL) as well as other related journals and volumes tend to consist of mixed methods studies, using both case-based and quasi-experimental research methods (e.g. Sinha et al., 2015; Zahn, 2017). In this chapter we will provide rich examples of how researchers can use video data for deep qualitative case studies or with advanced and automated methods for complex visual analyses (see below).
Over time, we propose, they may also be used along with Learning Analytics.
We open the chapter with an historical overview of pioneering analogue and digital video researchers in the Learnings Sciences and CSCL. We also delve into current research in CSCL video research to explore the benefits and challenges that exist now and will likely exist in the coming years. Questions about data collection, data
transformation, data analysis, and interpretation will be followed by three examples of contemporary research studies. We will also present new approaches to using video data that allow for deeper post-hoc observations of recorded learning interactions and digging into the details of knowledge co-construction and knowledge-building in and beyond CSCL research. For example, certain collaborative theoretical approaches, such as complex qualitative interaction analyses (Rack et al., 2019), focus on coordination and collaboration group processes. This interactional approach is especially enhanced by collecting and analyzing video data which can, if needed, be linked to ethnographic video accounts.
The closing sections of this chapter address the current understanding of video data as observations from single or multiple cameras or 360° camera recordings. It will also look at the emergence of video data as ways of “looking through people’s eyes” when eye- tracking or the use of virtual reality tools for collecting and analyzing video data are used (Greenwald, et al., 2017; Sharma et al., 2017). Such tools represent promising areas for future developments. A deeper understanding of how a range of theories and collaborative methods and tools influence the research process can be found in
Goldman et al., (Part 1 and 4, 2007); Derry et al. (2010) as well as in Goldman et al.
(2014).
History & Development: Pioneering Video Research
The 20th century heralded in a range of new visual media forms such as social
documentary, fictional photography, and ethnographic film-making. To study this topic more deeply, refer to the AMC filmsite called The History of Film. See:
https://www.filmsite.org/pre20sintro2.html. The affordances of both photography and film were soon adopted by sociologists, anthropologists and ethnographers around the world as tools for studying the lives of people at home, school, work or play, in places both near and far. For example, anthropologist Margaret Mead and cybernetician Gregory Bateson used the film camera as a tool for social and cultural documentation, producing a film called Bathing Babies in Three Cultures in 1951 based on Mead’s research comparing the bathing practices of mothers in three countries - New Guinea, Bali, and the USA. Mead, ever the futurist, imagined a time when there would be a 360º cameras (Mead, 1973). She thought it would take 10 years. It took 40!
Foundational Analogue and Digital Video Studies in LS and CSCL
Erickson (2011) looking back on his own early video observations of learning processes in groups emphasizes the central advantage which made him rely on audiovisual
records to study learning in small groups: “…I could see who the speakers were addressing as they spoke – a particular individual, a subset of the group, or the whole group…. A multimodal and multiparty analysis of the locally situated ecological
processes of interaction and meaning making became possible….” (p. 181). The camera he used weighed about 25 pounds and recording was done on reels that were about 16 inches in diameter.
One of earliest breakthrough collaborative classroom studies of interpreting digital video data was conducted by Goldman-Segall (1998) at the MIT Media Lab. For over 3 years, her digital video ethnography at a Boston magnet school included videotaping computer activities of, and conversations with, grade 5 and 6 youth and their teachers. During the decade, Goldman-Segall developed the Points of Viewing Theory (1988) and the Perspectivity Methodology (Goldman, 2007b) based on Clifford Geertz’s (1973) notion of layering data to build thick description. Goldman-Segall along with Dong (2007) advanced the ethnographic use of thick description to become thick interpretations that were built by collaborative by layering diverse views of researchers, teachers, and students. For more than two decades she designed digital video analysis environments with each new research study. The first environment was a simple HyperCard tool that enabled Goldman to establish categories gleaned from thematically-arranged video excerpts that had been transferred onto videodiscs. By using her new tool called Learning Constellations collaborating teachers and researchers could annotate, rate, analyze, and interpret the video (1998). Following LC was the tool, WebConstellations in 1997, and Orion, an online digital video analysis tool for changing our perspectives as an interpretive community (Goldman, 2007a). Each of these collaborative studies and the methods and tools are described in articles found in the references.
Modern technologies also allowed researchers to be more and more flexible in studying more complex learning situations comprehensively. For instance, Cobb and colleagues (e.g., Cobb & Whitenack, 1996) studied children’s mathematical development in long- term social contexts in a classroom study. Two cameras captured pairs of children collaborating on mathematics problem solving over a course of 27 lessons, the authors articulate a three-stage method that begins with interpretive episode-by-episode
analyses and meta-analyses resulting in integrated chronologies of children’s social and mathematical developments.
Another comprehensive study was the collected videotaped records for classroom instructions from classrooms around the world called the Third International
Mathematics and Science Study (TIMSS; Stigler et al., 1999). This video-based comparative study aimed at drawing comparisons between national samples. It set a standard for international sampling and video-based methods (Seidel et al., 2005): 231 eighth-grade mathematics lessons from Germany, Japan, and the United States were observed. In each classroom, one lesson was videotaped. The tapes then were
encoded, transcribed, and analyzed based on a number of criteria. Analysis focused on the content and organization of the mathematics lessons and on the teaching practices
that used a software especially developed for this study. According to Stigler et al., advantages of videos compared to real-time observations make it possible for observers to work collaboratively on the video data. A further advantage described is the
facilitation of communication of the research results. Similar advantages were achieved with new approaches when digital video tools entered the scene.
A comprehensive video workflow model for using video data in learning science research so that it can be shared, was presented by Pea and Hoffert (2007). Their process model goes from the strategic planning of video research to a pre-production phase to the phases of video capturing, coding, storing, and chunking. Analysis then turns into collections of video segments, further statistical analyses or case descriptions.
The model moves from creating video as a means of observation and data collection towards de-composing video for analysis and then towards re-composing video for shared interpretation, collaboration, and discussion in a group or larger community of researchers. Pea and Hoffert thereby suggest that staying as close as possible to the video data during the research process, instead of translating results back and forth, affords an almost absolute closeness to the data during the whole process - also in sharing or presenting and discussing results. The authors introduce “WebDIVER”, which is a streaming media interface for “web-based diving” into the video.
Koschmann et al. (2007) used the ethnomethodology of mini-chunks of video data to closely examine how learners form and act in collaborative communities. Their narrative methods used video to compose analytic narratives/stories from their footage.
Powell et al. (2003) developed a seven-step method through their longitudinal study of children’s mathematical development within constructivist learning environments. The method starts with the researcher attentively viewing micro-video and then proceeds through stages of identifying critical events, transcribing, coding, until it ends with composing analytic narratives.
We will now discuss lessons that have been learned and how to integrate those lessons into future research practices.
State of the Art
Video analysis is now a common practice in learning science and CSCL research that spans across methodological approaches, be they experimental, quasi-experimental, field research, or case-studies (see Derry et al., 2010). Video data are used to capture social or individual interaction, present moments of learning, and, in qualitative case studies, produce “collaborative learning accounts” (Barron, 2003). In this section, we first take a generic methodological perspective that tackles the general challenges and practices of applying video analysis in the learning sciences. Then we highlight specific problematics, and solutions when video data are used with qualitative, quantitative or mixed-methods research in CSCL settings, provide examples for CSCL video
collections and analysis. Ethnographic, narrative, problem-based and design-based methods are also included.
Benefits and challenges of using video methods in learning science research
From a methodological viewpoint, researchers agree that video-based research provides highly valuable data on learning processes in collaborative settings. For
instance, they provide for detailed process data that can be analysed in an event-based, but also in a time sequence-based approach (for analysis of discrete event sequences, see Chiu & Reimann this volume). At the same time such research is highly selective and researchers’ decisions determine what is being recorded and analysed.
Researchers decisions precede the production of video data, adding their points of viewing on them at all stages of the research (Goldman-Segall, 1998). On the one hand, video technologies can be beneficial in that they represent powerful ways of collecting video data with easy to use, relatively lightweight and affordable cameras. They also constitute well-designed web-platforms for storage and for sharing video data with other researchers, and they are effective tools for deeper analysis and video editing. Despite these notable advantages, as Derry et al. (2010) specify, there are challenges posed to researchers who collect and use video records to conduct research in complex learning environments. These challenges include: developing or finding appropriate analytical frameworks and practices for given research goals; identifying available technologies and new tools for reporting, and sharing videos; and, protecting the data and rights of participants, i.e., ethics and privacy issues. Blikstadt-Balas (2017) adds further key challenges: contextualization, (getting close enough to a situation to detect details, but always keeping an extra eye on the context); magnification, (magnifying small details that might be irrelevant for learners in the situation, even if it may be critically important to researchers); representation (presenting data in a way that others can understand and follow scientific interpretations).
With respect to the tension between the aforementioned benefits and challenges, Derry et al. (2010) suggest careful consideration of the different phases in the practice of using video analysis and interpretation of results. In each of these phases, researchers must be aware of the consequences of their selections, decisions, and the procedures they apply.
Specification for CSCL- research – Selected Research examples
In CSCL we consider specific issues related to the use of video data in computer- supported and collaborative learning. An additional challenge for CSCL settings is that researchers have to integrate or synchronize data streams on social interactions or conversations (recorded in a physical space) with further data (e.g., screen recording or logs of human-computer-interactions). How can this be accomplished in practice? The following three examples illustrate possible solutions: First, a case study of a qualitative and in-depth analysis of the collaboration process based on online-verbal
communication, where video was used as additive. Second, an exploratory study
following N=5 groups over a time span of a six-week course where video analysis based on coding and counting was central and both conducted and reported in a very
distinguished way. Third, an example from experimental research with a sample of
N=24 pairs of learners where video analysis was used in a complex and multi-leveled mixed-methods approach.
In the first example, Vogler, Schallert, et al. (2017) report a case study on the emerging complexity of online interactions and the way participants contribute through meaning making in a classroom discussion that took place in a CSCL environment. The research question was how meaning emerges from the collective interactions of individuals. In particular, the researchers investigated how the small groups introduced, sustained, and eventually closed a discussion topic. Therefore computer-mediated discussions of small student groups in class were analyzed. Data were collected by means of screen
recording (Camtasia software) for capturing the participants’ activities on the computer (e.g., any changes that occurred on the screen display, typing, deleting, or opening of online resources). Further on, the researchers captured by means of four video
cameras the activities and interactions that took place in the physical classroom – i.e., the small groups of two to three participants were recorded (e.g., eye gaze away from the screen, body movements, and accessing offline materials). In addition, trained observers took ethnographic notes. From the collected online-conversations, the
researchers created transcripts, coherence maps (for an example, see Fig. 1) and then spreadsheets showing how individual comments were connected and how threads and topics evolved (Vogler, Schallert, et al. 2017). The authors report on micro-analyses of those learners` discourses and present a detailed analysis of the life cycles of two selected discussion threads. The video recordings from the four classroom cameras were used as additional data together with the researcher’s observations. The data streams were synchronized by means of a tedious process that had to be done manually prior to analysis. It would have been interesting to couple different data sources (screen recordings, written discussion threads and video recordings of non- verbal behaviors) using complex and elaborate visual analysis methods. A point to which we will return below.
Fig. 1: Example of a coherence graph kindly with friendly permission by Jane Vogler In the second example, Näykki et al. (2017) examined, in an exploratory study, the role of CSCL-scripts for regulating discussions during a during a six week-long
environmental science course in teacher education. The scripts (i.e., prompts presented on tablet computers) aimed at supporting the planning and reflection of the collaborative process. The authors compared processes of scripted and non-scripted collaborative learning asking how socio-cognitive and socio-emotional monitoring would emerge in groups depending on the (more or less) active use of such scripts. They also
investigated how monitoring activities would transfer to subsequent task work. The study took place in a classroom-like research space and video data were collected by means of a 360-degree recording method (for details, see: https://www.oulu.fi/leaf- eng/node/41543). The authors extracted 30 hours of video data (discussions,
movements, and gestures) from five student groups that were repeatedly captured five times. A multi-step analysis method was applied for analysis: the video data were first segmented into 30-second events. Each 30-second segment was annotated by a researcher with a description of what had occurred within the segment resulting in a content log of each video (e.g. group finishes task; one person shows their created mind map to others, group discusses task completion, suggestions on further proceeding).
The content log of each video was complemented with a comprehensive memo of the most salient observations. In a second step, each 30-second segment was observed to see if group members showed socio-cognitive and socio-emotional monitoring (i.e., the behaviors associated with the understanding and progress of the study-like task, content understanding, socio-emotional support). The subsequent development of categories and coding procedure is described thoroughly in Näykki et al. (2017). 25% of the video data were also coded by an independent coder. Upon this data, frequency analysis was applied for further statistical hypothesis testing. Time-based video segmentation was also applied by Sinha et al., (2015) studying collaborative
engagement in CSCL groups, but here the video segments were subjected to observer ratings of the quality of collaborative engagement in small groups (high, moderate or low) and used for qualitative case studies.
In the third example, N = 24 pairs of students were investigated when learning with advanced digital tools in history lessons (Zahn et al., 2010). Two conditions supporting collaborative learning were compared: one where students used an advanced web- based video tool (WebDIVER, see Pea & Hoffert, 2007) and one where students used a simple video player and text tool (controls). The advanced tool allowed cutting out of details from video sequences and extracting those “pieces of video” in order to
comment on the details. Students’ interactions with technology were captured by means of screen recording (Camtasia Studio by TechSmith) and dyadic social interactions were recorded by means of a webcam. In order to analyze these data, a mixed-methods strategy was applied in combining both types of data in a two-step coding procedure (for subsequent quantitative analyses) and integrated activity transcripts (for subsequent qualitative case studies). Trained observers first watched the video recordings of social interaction to identify emergent behavior categories and then applied a process of coding and counting. Eight categories of verbal interactions were found in this process (e.g., content-related talk, video-related talk, technical issues talk, help seeking, etc.).
The relative amounts of time spent for talking in the categories, related to total talking time, were then calculated and compared between conditions.
Transcripts of learning episodes were produced for deep analyses of selected cases and specific categories (e.g., content-related talk) from the different conditions. The transcripts synchronized the students’ conversations and interactions with digital tools (e.g., typing, submitting comments, playing video, watching, stopping video, rewinding, making marks with an advanced video function, etc.). The transcripts were analyzed according to Barron (2003) as “localized accounts” of “successful learning”.
Based on this qualitative approach it would be interesting in further research to return to a quantitative strategy by counting collaboration patterns in dyads from both conditions and compare their prevalence statistically thereby testing for significance. Yet, limited resources often force research to disclaim such mixed method approaches. Future perspectives, however, include automated analyses that could render this option feasible.
In sum, from these examples it can be noted how a number of decisions were made in the phases of video data collection and analyses, starting from the number and types of cameras used as well as their placement in the investigated scene; to the number of groups and group sizes under scrutiny; the duration and frequency of video data collection; the decision of using transcripts for qualitative in-depth analysis vs.
developing categories to be coded and counted or both; the using of extra-visualizations or verbal comments for data exploration; and, to the selecting of results to be presented in a scholarly publication.
The Future
Video analysis has evolved rapidly alongside recent technological progress (e.g., mobile eye-tracking, social computing, virtual reality). In this section, we will look ahead and include developments such as tracking and automatic data analysis methods from social computing technologies.
Eye-Tracking in CSCL research
Eye-tracking as a method to investigate learning behaviors has been widely used in individual learning settings in the last few years (for an overview, see Alemdag &
Cagiltay, 2018; Lai et al., 2013). Mobile eye-tracking, for example, was applied to research on informal learning in museums (Mayr et al., 2009; Wessel et al., 2007) where researchers can reflect on eye-tracking videos afterwards together with visitors in order to gain insights into the motivational factors and possible effects of exhibition design on learning during a museum visit (vom Lehn & Heath, 2007).
Although using eye-tracking in CSCL is not unknown (e.g., Stahl et al., 2013), it still seems rather uncommon. Since 2013, only few studies were published that used eye- tracking as a method in CSCL research. Among these are the studies by Schneider et al. (2016), Schneider and Pea (2013, 2014), Sharma et al. (2017) and Stahl et al.
(2013) summarizing the advantages and possibilities of eye-tracking as a method to support and research collaboration. Schneider and Pea (2013) investigated
collaborative problem-solving situations, were dyads saw the eye gazes of their partner on a screen. The authors found that this mediated joint visual attention helped dyads achieve a higher quality of collaboration and increased learning gains. Results indicate that joint visual attention in collaborations is of great importance since it is crucial that collaborative learners, solving a problem together, take the problem as the same problem and recognize an object as the same object (Stahl et al., 2013; Zemel &
Koschmann, 2013). In a follow-up study Schneider and Pea (2014) investigated
collaborative learning processes in dyads remotely working in different rooms. Similar to their previous study (Schneider & Pea, 2013), participants were able to see the gaze of their learning partner on the screen. Using eye-tracking data, Schneider and Pea (2014) could roughly predict collaboration quality with an accuracy between 85% and 100%.
Hence, joint attention (which involves gaze) is an important nonverbal predictor and indicator for successful collaboration. In addition, Schneider et al. (2016) investigated the way users memorize, analyze, collaborate, and learn new concepts on a Tangible User Interface (TUI) in a 2D vs. 3D interactive stimulation of a warehouse. Eye-tracking goggles were used as method to further investigate collaboration processes in co- located settings. Results suggest that 3D interfaces foster joint visual attention which significantly predicted task performance and learning gains. The little existing research about using gaze in CSCL has demonstrated that eye-tracking data contributes highly relevant and important insights into collaborative processes (see also Schneider,
Worsley, & Martinez-Maldonado, this volume). Sharma et al. (2017) elaborate that “eye- tracking provides an automatic way of analyzing and assessing collaboration, which could gain deeper and richer understandings of collaborative cognition. With the increasing number of eye-tracking studies, in collaborative settings, there is a need to create a shared body of knowledge about relations found between gaze-based
variables and cognitive constructs” (pp. 727).
With eye-tracking devices, especially mobile eye-trackers, becoming cheaper and widely available, we expect further increases of eye-tracking studies in CSCL research.
For this reason, theoretical frameworks for eye-tracking research in CSCL in both co- located and online-remote settings are needed. Moreover, creating and sharing digital mini-movies for “looking through people’s eyes” during collaborative learning would be an interesting open data option for CSCL researchers. In addition, developments of automatic analyses of eye-tracking and other data in collaborative learning processes in the future will lead CSCL research to facilitated behavior pattern-based research. We elaborate on this latter point in the next section on related tracking methods from small group research. In fact, corresponding technology, such as the Tobii Pro Glasses, is already in development. Such new mobile eye-trackers not only provide multiple cameras (such as infrared cameras recording eye-movements and scene carmeras, recording participants view) but also microphones and other tracking technologies.
Furthermore, Li et al. (2019), for instance, proposed a smart eye-tracking system for VR device that is able to detect eye movements in real time in a VR stimulation.
Related tracking & automatic analyses methods
Studies in small group research have developed related methods such as the tracking of head and body activities during social interactions. These methods include the integration with other behavior data such as recordings of verbal conversations or nonverbal cues in communication (e.g., speaking turns, interruptions) as well as
questionnaire data. Studies on the automatic extraction of cues from AV-recorded face- to-face interactions in small groups use either rule-based methods or learning
algorithms. Such algorithms can produce larger amounts of annotated data in accurate ways in less time - compared with annotations added by human observers/researchers.
For instance, Sanchez-Cortes et al. (2012), investigating nonverbal behaviors during collaborative problem solving for early detection of emergent leading behaviors in small groups, used algorithms for the tracking of head activities and integration with other data: In a lab setting consisting of two setups: a static setup with six cameras (four close-ups, two side-views, and one center-view of the interaction under scrutiny), and a portable setup with two web cameras (Logitech® Webcam Pro 9000) including portable audio and video sensors, they recorded social face-to-face interaction of groups in problem solving (survival task) and several nonverbal features were automatically extracted from recordings. The results (from approximately 10 h of audio/video recordings) were integrated with variables extracted from questionnaires filled in by each group member immediately after the recordings. Results of their study indicate that an early identification of emergent leaders was possible with an accuracy of up to 85%. Thus, analyses of complex data from intricate scenarios, such as face-to-face group problem solving scenarios or collaborative learning in the natural field, become feasible.
Another example for an approach to integrate and visualize larger data sets and data scales, including multimodal data, is the Interaction Geography Slicer (IGS) (Steier et al., 2019). The tool was used within a three-year project in collaboration with a
nationally renowned museum in the United States. One goal of the project was to understand how visitors move across galleries within a complete museum visit (Shapiro, 2019). Therefore, visitor groups were recorded by video and audio records and their movements, interactions and social media use was collected. The IGS forces the challenge of making sense of diverse unstructured data by dynamically integrating and visualizing these data and proves new visual analytic techniques. It supports
representing and interpreting of collaborative interaction as people move across physical environments (Shapiro & Hall, 2018).
In the future of CSCL research, the automatic tracking and analysis of video recorded data will yield three noteworthy benefits for the investigation of complex data sets from both face-to-face and online settings: First, it will allow us to investigate more realistic scenarios in more detail thereby allowing us to adequately take into account learners`
nonverbal behaviors in the physical space during computer-supported collaborative learning. Second, with data visualization tools it will become possible to manage the complex data streams from body, head, or eye tracking devices even when we
investigate larger collaborative groups (more than two people) in the field (Rack et al., 2019). Moreover, integrative visualization tools, such as the IGS, will bring new ways to study collaborative learning context from a learning perspective. Third, video-based research can be implemented into new CSCL-models and simulations in future learning environments. For instance, both knowing that joint visual attention has positive impacts on collaboration quality and also recognizing that gaze is an important indicator for it, we can consider implementing gaze data in future virtual 3D collaborative learning environments in order to allow collaborative learners to follow the eye gaze of their partners.
More generally, and on the research side, it might be considered useful to generate collections of mini video clips based on video-recorded quality indicators in order to discuss them on open access platforms with other researchers for joint analyses and knowledge building in CSCL research.
By such developments, CSCL-researchers would probably be able to meet two of the key challenges of using video when investigating social interactions in education as described by Blikstad-Balas (2017): “getting close enough to the details without losing context” and “representing data so that an audience can assess whether inferences drawn from the data are plausible” (p. 511).
Virtual reality (VR) and augmented reality (AR) in CSCL Research
In their content analysis Zheng et al. (2014) found a significant upswing of virtual reality (VR) in CSCL between 2003-2007 and 2008-2012. This is in line with the fact, that VR is becoming more popular and the number of active users worldwide is forecasted to grow (Statista, 2018). A reason for this trend is that VR has become more accessible and cheaper due to innovations in design and manufacturing. Due to this development, VR is affordable for institutions and could bring new possibilities both into the classroom and into CSCL research. Likewise, augmented reality (AR) where virtual objects are
“added” to the physical world (Azuma et al., 2001) provides multiple new options for educational settings and is, in contrast to VR, already widely used in schools (for an overview, see Akçayır & Akçayır, 2017). Nevertheless, research on these topics is still rare—especially methodological research.
Focusing on the potential of virtual reality in education, Greenwald et al. (2017) highlight the importance of asking the question of what problems can be solved with VR in order to not only offer adequate possibilities - “adds-ons” – compared to traditional
approaches in educational settings, but new and required possibilities. Thereby, two main advantages of virtual reality are mentioned: interaction with other humans and adaptable environments. In the category interaction with other humans, it can be
distinguished between how (e.g., take on another shape; avatar) and who one interacts with (e.g., remote people). Environments in the virtual reality, for example, offer the possibility of going to difficult places to reach (e.g., the moon) or to interact in and with an inherently virtual environment (e.g., move through veins of the human body). An initial approach of how environments of VR systems can influence learning comes from a team of the Bauhaus University Weimar (Beck et al., 2013; Kulik et al., 2011;
Salzmann et al., 2009). In their research, they focus on the learning of complex and ambiguous information. In this context two factors seem to be of great importance:
exchange with peers – whereby a certain level of individual autonomy should be ensured (Sawyer, 2008, pp. 64-66) - and learning by doing. Therefore, virtual reality environments should allow interactivity, for the one hand, and, on the other hand, enable communication between multiple learners in addition to having fluent transitions between individual and collaborative activities. On this basis, Kulik et al., (2011) and Salzmann et al. (2009) found that learners interact with virtual objects just as they do with real objects and that the understanding of all learners can be increased in
collaborative visual search. Furthermore, Beck et al. (2013) found that body language with 3D avatars in such systems can be supported, whereas the perceived co-presence of such avatars is limited. Last but not least, Greenwald et al. (2017) describe new opportunities for interacting and learning in children with autism, in order to offer new environments where interactions with others might be easier. In summary, it becomes
visible, that VR offers new possibilities for collaboration processes that in turn should be observed with VR-based methods. In fact, VR offers new opportunities for analysis methods of collaborative learning environments. Especially with 360-degree cameras and spatial microphones new holistic and environmentally sensitive records of
collaborative learning interactions are offered (Steier et al., 2019). Such records enable a holistic immersion in a collaborative learning situation even after the capture of the event. Since there is a lack of existing tools supporting transcribing and analyzing of learning interactions in 360-degree videos, McIlvenny (2018) and McIlvenny and Davidsen (2017) developed a tool that allows researchers – inter alia - to annotate and deploy interactive transcript objects directly into the 360-degree video in virtual reality (AVA360VR: Annotate Visualize Analyse 360° video in VR). With such new methods, the investigation of collaborative interactions in all its complexity gets more holistic, without reducing it by transforming these embodied interactions into transcripts (Davidsen & Ryberg, 2017). With regard to augmented reality, learning environments can be created that facilitate the development of processing skills through independent collaborative exercises by combining digital and physical objects (Dunleavy et al., 2009). In fact, there is evidence that digital augmentations can help conceptual development in informal collaborative settings of science knowledge during science experiences (Yoon et al., 2012).
Conclusion
In conclusion, from the works we reported above, we see new future perspectives for eye-tracking CSCL research devoting by new eye-tracking technologies that not only allow tracking of eye-movements but also recordings of participants’ views and audio recording. Moreover, new tools allowing to dynamically integrate and visualize large, multimodal data sets will bring new and more holistic insights in collaborative
interactions and simultaneous facilitate analysis and interpretation of these data.
Furthermore, we can derive different future perspectives for AR- or VR-based CSCL research: Firstly, AR and VR changes the research object itself, because learning in augmented or inherently virtual environments not only offers new possibilities, but also new forms of collaboration by interacting partially or fully in virtual systems (e.g., with avatars). These new forms are important research topics to be observed by VR-based methods. For example, when interactions with others become easier in VR
environments for children with autism, these interactions and respectively, their collaborative learning, can be investigated empirically. Second, the new forms of
collaboration processes can be investigated through new forms of interactivity in virtual environments (e.g., adaptive environments) - which offers important options in
experimental research. In a VR simulation, collaboration and behaviors (such as head movements or gaze) can be easily tracked. Consequently, we have new methods for rich observation and reliable measurement of social interactions. For instance,
collaborative processes could be recorded as 360-degree videos and transferred in VR simulations instead of simple videos or video could be transformed into VR
environments in order to investigate specific behaviors and interactions during
collaborative learning. Appropriate tools will provide new opportunities to transcribe and
analyze collaborative settings, for example by possibilities to annotate directly into 360- degree videos. Yet, such methods still need to be further developed - and their potential role of producing methodological artifacts must be critically reflected upon. If we allow such tools to serve as methods in collaborative learning, we might not always be
investigating what we think we are investigating. Future methodological research on the effects of VR or AR on collaborative learning is needed.
References
Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature.
Educational Research Review, 20, 1–11.
Alemdag, E., & Cagiltay, K. (2018). A systematic review of eye tracking research on multimedia learning. Computers & Education, 125, 413–428.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001).
Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47.
Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 307-359.
Beck, S., Kunert, A., Kulik, A., & Froehlich, B. (2013). Immersive Group-to-Group Telepresence. IEEE Transactions on Visualization and Computer Graphics, 19(4), 616–625.
Blikstad-Balas, M. (2017). Key challenges of using video when investigating social practices in education: Contextualization, magnification, and representation.
International Journal of Research & Method in Education 40(5), 511-523.
Brauner, E., Boos, M., & Kolbe, M. (Eds.) (2018). The Cambridge handbook of group interaction analysis. Cambridge, U.K. Cambridge University Press.
Chiu & Reimann [this volume]
Cobb, P., & Whitenack, J. W. (1996). A method for conducting longitudinal analyses of classroom videorecordings and transcripts. Educational Studies in Mathematics, 30(3), 213-228.
Davidsen, J., & Ryberg, T. (2017). “This is the size of one meter”: Children’s bodily- material collaboration. International Journal of Computer-Supported
Collaborative Learning, 12(1), 65–90.
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., Hall, R., Koschmann, T., Lemke, J. L., Sherin, M. G., & Sherin, B. L. (2010). Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics. Journal of the Learning Sciences, 19(1), 3-53.
Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22.
Erickson, F. (2011). Uses of video in social research: a brief history. International Journal of Social Research Methodology, 14(3), 179-189.
Geertz, C. (1973). The Interpretation of Cultures. Basic Books.
Goldman, R. (2007a). Orion™, an online digital video analysis tool: Changing our perspectives as an interpretive community. In R. Goldman, R. Pea, B. Barron, &
S. Derry (Eds), Video research in the learning sciences (pp. 507–520).
Erlbaum/Routledge.
Goldman, R. (2007b). Video Representations and the Perspectivity Framework:
Epistemology Ethnography, Evaluation and Ethics. In R. Goldman, R. Pea, B.
Barron, & S. Derry (Eds), Video research in the learning sciences (pp. 3-38).
Erlbaum/Routledge.
Goldman, R., & Dong, C. (2007). Using Orion™ and the Perspectivity Framework for video analysis to interpret events: The CSCL’05 Video Exemplar. CSCL‘07: The Computer Supported Collaborative Learning 2007 Conference. Rutgers, pp. 16–
25.
Goldman, R., Zahn, C. & Derry, S. (2014). Frontiers of Digital Video Research in the Learning Sciences: Mapping the Terrain. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed.). Cambridge University Press.
Goldman-Segall, R. (1998). Points of Viewing Children’s Thinking: A Digital
Ethnographer’s Journey. Erlbaum/Routledge. – with accompanying online video vignettes and commentary tools.
Greenwald, S., Kulik, A., Kunert, A., Beck, S., Frohlich, B., Cobb, S., Parsons, S., Newbutt, N., Gouveia, C., Cook, C., Snyder, A., Payne, S., Holland, J., Buessing, S., Fields, G., Corning, W., Lee, V., Xia, L., & Maes, P. (2017). Technology and applications for collaborative learning in virtual reality. In B. Smith, M. Borge, E.
Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL, 12th International Conference on Computer Supported Collaborative Learning (CSCL) 2017 (Vol. 2, pp. 719-726). International Society of the Learning Sciences.
Koschmann, T., Stahl, G., & Zemel, A. (2007). The video analyst’s manifesto (or the implications of Garfinkel’s policies for studying instructional practice in design- based research). In R. Goldman, R. Pea, B. Barron, & S. Derry (Eds.), Video research in the Learning Sciences (pp,133-143). LEA/Routledge.
Kulik, A., Kunert, A., Beck, S., Reichel, R., Blach, R., Zink, A., & Froehlich, B. (2011).
C1x6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. ACM Transactions on Graphics 30(6), 1-12 ,.
Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y, Lee, M.-H., Chiou, G.-L., Liang, J.-C., & Tsai, C.-C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90–115.
Li, B., Zhang, Y., Zheng, X., Huang, X., Zhang, S., & He, J. (2019). A Smart Eye Tracking System for Virtual Reality. In 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC) (Vol. 1, pp. 1–3).
Mayr, E., Knipfer, K., & Wessel, D. (2009). In-sights into mobile learning: An exploration of mobile eye tracking methodology for learning in museums. Researching mobile learning: Frameworks, methods, and research designs, 189-204.
McIlvenny, P. (2018). Inhabiting spatial video and audio data: Towards a scenographic turn in the analysis of social interaction. Social Interaction. Video-Based Studies of Human Sociality, 2(1). https://doi.org/10.7146/si.v2i1.110409
McIlvenny, P. B., & Davidsen, J. (2017). A Big Video Manifesto: Re-sensing Video and Audio. Nordicom Information, 39(2), 15–21.
Mead, (1973)
Näykki, P., Isohätälä, J., Järvelä, S., Pöysä-Tarhonen, J., & Häkkinen, P. (2017).
Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script–an exploratory study. International Journal of Computer-Supported Collaborative Learning, 12(3), 251-279.
Pea, R. D. & Hoffert, E. (2007). Video workflow in the learning sciences: Prospects of emerging technologies for augmenting work practices. In R. Goldman, R. Pea, B.
Barron, & S. J. Derry (Eds.) Video research in the learning sciences. (pp. 427- 460). LEA/Routledge.
Powell, A. B., Francisco, J. M., & Mager, C. A. (2003). An analytical model for studying the development of learners’ mathematical ideas and reasoning using videotape data. Journal of Mathematical Behavior, 22, 405–435.
Rack, O., Zahn, C. & Bleisch, S. (2019). Do you see us? – Applied Visual Analytics for the Investigation of Group Coordination. Gruppe. Interaktion. Organisation.
Zeitschrift für Angewandte Organisationspsychologie (GIO) 50, 53-60. [Group.
Interaction. Organisation. Journal of Applied Organisational Psychology].
Salzmann, H., Moehring, M., & Froehlich, B. (2009). Virtual vs. Real-World Pointing in Two-User Scenarios. In 2009 IEEE Virtual Reality Conference (pp. 127–130).
Sanchez-Cortes, D., Aran, O., Mast, M. S., & Gatica-Perez, D. (2012). A nonverbal behavior approach to identify emergent leaders in small groups. IEEE Transactions on Multimedia, 14(3), 816-832.
Sawyer, K. (2008). Group genius: The creative power of collaboration. Basic Books.
Schneider, B., & Pea, R. (2013). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. International Journal of Computer-Supported Collaborative Learning, 8(4), 375–397.
Schneider, B., & Pea, R. (2014). Toward collaboration sensing. International Journal of Computer-Supported Collaborative Learning, 9(4), 371–395.
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2016).
Using Mobile Eye-Trackers to Unpack the Perceptual Benefits of a Tangible User Interface for Collaborative Learning. ACM Transaction on Computer-Human Interaction, 23(6), 39:1-39:23.
Schneider, Worsley, & Martinez-Maldonado [in this volume]
Seidel, T., Prenzel, M., & Kobarg, M. (Eds.) (2005). How to run a video study. Technical report of the IPN video study. Waxmann.
Shapiro, B. R. (2019). Integrative Visualization: Exploring Data Collected in Collaborative Learning Contexts. In Proceedings of the 13th International
Conference on Computer Supported Collaborative Learning (CSCL) (Vol. 1, pp.
184–191). International Society of the Learning Sciences.
Shapiro, B. R., & Hall, R. (2018). Personal Curation in a Museum. In Proceedings of the ACM in Human-Computer Interaction, CSCW (Vol. 2, Article 158). ACM.
Sharma, K., Jermann, P., Dillenbourg, P., Prieto, L. P., D’Angelo, S., Gergle, D., Schneider, B., Rau, M., Pardos, Z., & Rummel, N. (2017). CSCL and Eye- Tracking: Experiences, Opportunities and Challenges. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL, 12th International Conference on Computer Supported Collaborative Learning (CSCL) 2017 (Vol. 2). International Society of the Learning Sciences.
Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015).
Collaborative group engagement in a computer-supported inquiry learning environment. International Journal of Computer-Supported Collaborative Learning, 10(3), 273-307.
Stahl, G., Law, N., & Hesse, F. (2013). Reigniting CSCL flash themes. International Journal of Computer-Supported Collaborative Learning, 8(4), 369–374.
Statista. (2018). Number of active virtual reality users worldwide from 2014 to 2018 (in millions). Retrieved November 27, 2018, from
https://www.statista.com/statistics/426469/active-virtual-reality-users-worldwide/
Steier, R., Shapiro, B. R., Christidou, D., Pierroux, P., Davidsen, J., & Hall, R. (2019).
Tools and Methods for ‘4E Analysis’: New Lenses for Analyzing Interaction in CSCL. In Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL) (Vol. 2, pp. 759–766). International Society of the Learning Sciences.
Stigler, J. W., Gonzales, P., Kwanaka, T., Knoll, S., & Serrano, A. (1999). The TIMSS Videotape Classroom Study: Methods and Findings from an Exploratory
Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States. A Research and Development Report (National Center for Education Statistics Report No. NCES 99-0974). U. S. Government Printing Office.
Vogler, J. S., Schallert, D. L., Jordan, M. E., Song, K., Sander, A. J. Z., Chiang, Y. Y. T., Lee, J.-E., Park, J. H., & Yu, L.T. (2017). Life history of a topic in an online discussion: a complex systems theory perspective on how one message attracts class members to create meaning collaboratively. International Journal of
Computer-Supported Collaborative Learning, 12, 173–194.
https://doi.org/10.1007/s11412-017-9255-9
vom Lehn, D. & Heath, C. (2007). Social interaction in museums and galleries: a note on video-based field studies. In R. Goldman, R. Pea, B. Barron, & S. Derry (Eds.), Video Research in the Learning Sciences (pp. 287-301). LEA/Routledge.
Wessel, D., Mayr, E., & Knipfer, K. (2007). Re-viewing the museum visitors view. In G.
N. Vavoula, A. Kukulska-Hulme, & N. Pachler (Eds.), Research Methods in Informal and Mobile Learning (pp. 17-23). WLE Centre.
Yoon, S. A., Elinich, K., Wang, J., Steinmeier, C., & Tucker, S. (2012). Using
augmented reality and knowledge-building scaffolds to improve learning in a science museum. International Journal of Computer-Supported Collaborative Learning, 7(4), 519–541.
Zahn, C. (2017). Digital design and learning: Cognitive-constructivist perspectives. In: S.
Schwan & U. Cress (Eds.), The Psychology of Digital Learning. Constructing, Exchanging, and Acquiring Knowledge with Digital Media. Springer.
Zahn, C., Pea, R., Hesse, F. W., & Rosen, J. (2010). Comparing simple and advanced video tools as supports for complex collaborative design processes. Journal of the Learning Sciences, 19(3), 403-440.
Zemel, A., & Koschmann, T. (2013). Recalibrating reference within a dual-space interaction environment. International Journal of Computer-Supported Collaborative Learning, 8, 65–87.
Zheng, L., Huang, R., & Yu, J. (2014). Identifying computer-supported collaborative learning (CSCL) re-search in selected journals published from 2003 to 2012: A content analysis of research topics and issues. Journal of Educational
Technology & Society, 17(4), 335.
Additional Reading
Blikstad-Balas, M. (2017). Key challenges of using video when investigating social practices in education: Contextualization, magnification, and representation.
International Journal of Research & Method in Education, 40(5), 511-523.
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., Hall, R., Koschmann, T., Lemke, J. L., Sherin, M. G., & Sherin, B. L. (2010). Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics. Journal of the Learning Sciences, 19 (1), 3-53.
Goldman, R., Zahn, C. & Derry, S. (2014). Frontiers of Digital Video Research in the Learning Sciences: Mapping the Terrain. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed.). Cambridge University Press.
Näykki, P., Isohätälä, J., Järvelä, S., Pöysä-Tarhonen, J., & Häkkinen, P. (2017).
Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script–an exploratory study. International Journal of Computer-Supported Collaborative Learning, 12(3), 251-279.
Schneider, B., & Pea, R. (2013). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. International Journal of Computer-Supported Collaborative Learning, 8(4), 375–397.
