Multimodal Data Transcription and Analysis: Methodologies and Tools to Advance Your Work

Abstract

There has been a growing interest in examining gestures and multimodality in second language acquisition and teaching (Stam, 2013; Stam & Buescher, 2018). However, there have been insufficient opportunities for learning about various software programs and different methods for coding. This workshop addresses this gap by providing participants with interactive presentations and workshop time. The session includes two hours of presentations by experienced gesture and multimodal researchers and one hour of hands-on work so that participants can gain experience transcribing, coding and analyzing data and receive feedback from the presenters.

This hands-on workshop introduces participants to varied methods of coding multimodal (in particular video) data from a range of different perspectives, including McNeillian, which views speech and gesture as a single-integrated system (psycholinguistic), Ethnomethodological (Conversation Analytic), and discourse (linguistic). Participants will gain an understanding of some of the most useful software programs that are available as well as methods for coding. Emphasis will be on understanding the theoretical assumptions behind different transcription choices as well as on how to do this transcription and analytical work for varied purposes. Participants will gain skills in analyzing multimodal data using various systems and have the chance to engage in this work with their own data as well. Participants should bring their own laptops to the session loaded with the templates and data that will be provided beforehand. In the event that the workshop has to be conducted online, adjustments will be made to the hands-on portion of the workshop.

Gale Stam will introduce and manage the workshop and overview coding of data from a McNeillian (psycholinguistic) perspective of speech and gesture being a single-integrated system, focusing on thinking for speaking and teachers’ gestures. Søren Wind Eskildsen and Johannes Wagner will discuss coding of embodied phenomena in interactional data from the perspective of multimodal Conversation Analysis. Marianne Gullberg will describe coding related to the study of sustained discourse (as opposed to individual clauses or utterances) from a linguistic perspective, including how to address the challenges of coding learner discourse data (child and adult).

This workshop is dedicated to the memory of Kimi Nakatsukasa.

Biographies

Søren Wind Eskildsen is Associate Professor in Second Language (L2) Learning at the University of Southern Denmark, Sønderborg, Denmark. His primary research interest concerns usage-based processes and practices in second language learning over time as seen through the lenses of usage-based models of language and conversation analysis. A focal point in this work concerns the role of gestures and other embodied conduct in L2 learning, in-situ and over time. He works with both in- and out-of-class data, predominantly from English and Danish and Danish L2 settings.

Marianne Gullberg is Professor of Psycholinguistics at Lund University, Sweden, and Chair/Director of Lund University Humanities Lab. She examines acquisition and real time language use in adult L2 users and bi-/multilingual speakers, targeting semantics, discourse, and gesture production and comprehension. She headed the group The Dynamics of Multilingual Processing at the Max Planck Institute for Psycholinguistics (2000-2009), and co-founded the Nijmegen Gesture Centre in 2003. Her publications include the edited volumes Gestures and Second Language Acquisition (IRAL, 2006), Gestures in language development (2010 with K. de Bot), and From gesture in conversation to visible action as utterance (2014 with M. Seyfeddinipur). She was area editor of Second Language Acquisition in the Encyclopedia of Applied Linguistics (2012 with J. Williams), Associate Editor of the journal Language Learning (2009-2014) and is currently editor of the journal Language, Interaction, Acquisition.

Gale Stam is Professor of Psychology at National Louis University, Chicago. Her research interests include language and culture, language and cognition, gesture, and first language and second language acquisition, and teachers' gestures. She has published articles on changes in thinking for speaking, the importance of looking at gesture in second language (L2) acquisition, gesture and lexical retrieval in a L2, and language teachers’ gestures. She is a member of the editorial board of the journal Language and Sociocultural Theory (2013-present) and was a member of the editorial board for Gesture (2010-2018). She has also co-edited two volumes on speech and gesture – Gesture: Second Language Acquisition and Classroom Research (Routledge, 2008) and Integrating Gestures: The Interdisciplinary Nature of Gesture (John Benjamins, 2011). She is co-convener with (the late) Kimi Nakatsukasa of AILA ReN (Research Network): Gesture, Multimodality, and SLA (2018-2021).

Johannes Wagner is Professor of Communication in the Department of Design and Communication at the University of Southern Denmark. In recent years he has been working on a comprehensive understanding of human social praxis as the nexus of verbal interaction, embodied practices and tangible objects in the environment. He co-edited Longitudinal Studies on the Organization of Social Interaction (Palgrave Macmillan, 2018) with Simona Pekarek Doehler and Esther Gonzalez Martinez and Objects, Bodies and Work Practices (Multilingual Matters, 2019) with Dennis Day.