Multimodal data transcription and analysis: methodologies and tools to advance your work

Abstract

There has been a growing interest in examining gestures and multimodality in second language acquisition and teaching (Stam, 2013; Stam & Buescher, 2018). However, there have been insufficient opportunities for learning about various software programs and different methods for coding. This workshop addresses this gap by providing participants with interactive presentations and workshop time. The session includes two hours of presentations by experienced gesture and multimodal researchers and one hour of hands-on work so that participants can gain experience transcribing, coding and analyzing data and receive feedback from the presenters.

This hands-on workshop introduces participants to varied methods of coding multimodal (in particular video) data from a range of different perspectives, including McNeillian, which views speech and gesture as a single-integrated system (psycholinguistic), Ethnomethodological (Conversation Analytic), and discourse (linguistic). Participants will gain an understanding of some of the most useful software programs that are available as well as methods for coding. Emphasis will be on understanding the theoretical assumptions behind different transcription choices as well as on how to do this transcription and analytical work for varied purposes. Participants will gain skills in analyzing multimodal data using various systems and have the chance to engage in this work with their own data as well. Participants should bring their own laptops, and are encouraged to bring multimodal data excerpts as well.

Gale Stam  will introduce and manage the workshop and overview coding of data from a McNeillian (psycholinguistic) perspective of speech and gesture being a single-integrated system, focusing on thinking for speaking and teachers’ gestures. Søren Wind Eskildsen and Johannes Wagner will discuss coding of embodied phenomena in interactional data from the perspective of multimodal Conversation Analysis. Marianne Gullberg will describe coding related to the study of sustained discourse (as opposed to individual clauses or utterances) in a linguistic perspective, including how to address the challenges of coding learner discourse data (child and adult).

This workshop is dedicated to the memory of Kimi Nakatsukasa.

Biographies

Søren Wind Eskildsen is Associate Professor in Second Language (L2) Learning at the University of Southern Denmark, Sønderborg, Denmark. His primary research interest concerns usage-based processes and practices in second language learning over time as seen through the lenses of usage-based models of language and conversation analysis. A focal point in this work concerns the role of gestures and other embodied conduct in L2 learning, in-situ and over time. He works with both in- and out-of-class data, predominantly from English and Danish and Danish L2 settings.
Marianne Gullberg is Professor of Psycholinguistics at Lund University, Sweden, and Chair/Director of Lund University Humanities Lab. She examines acquisition and real time language use in adult L2 users and bi-/multilingual speakers, targeting semantics, discourse, and gesture production and comprehension. She headed the group The Dynamics of Multilingual Processing at the Max Planck Institute for Psycholinguistics (2000-2009), and co-founded the Nijmegen Gesture Centre in 2003. Her publications include the edited volumes Gestures and Second Language Acquisition (IRAL, 2006), Gestures in language development (2010 with K. de Bot), and From gesture in conversation to visible action as utterance (2014 with. M. Seyfeddinipur). She was area editor of Second Language Acquisition in the Encyclopedia of Applied Linguistics (2012 with J. Williams), Associate Editor of the journal Language Learning 2009-2014 and is currently editor of the journal Language, Interaction, Acquisition.
Kimi Nakatsukasa was Assistant Professor in Applied Linguistics and Second Language Studies at Texas Tech University. Her research interests included gestures and second language learning and teaching, classroom interaction, dynamic approach to learner psychology, and attention and second language development. Her publications appeared in the journals Studies in Second Language Acquisition, Language Teaching Research, and Modern Language Journal and her book chapters were published by the Cambridge University Press, Routledge, and John Benjamins. She was co-convener with Gale Stam of AILA ReN (Research Network): Gesture, Multimodality, and SLA (https://www.kiminakatsukasa.net/ren-gesture-multimodality-and-sla) 2018-2021.
Gale Stam is Professor of Psychology at National Louis University, Chicago. Her research interests include language and culture, language and cognition, gesture, and first language and second language acquisition, and teachers' gestures. She has published articles on changes in thinking for speaking, the importance of looking at gesture in second language (L2) acquisition, gesture and lexical retrieval in a L2, and language teachers’ gestures. She is a member of editorial board of the journal Language and Sociocultural Theory (2013-present) and was a member of the editorial board for Gesture (2010-2018). She has also co-edited two volumes on speech and gesture – Gesture: Second Language Acquisition and Classroom Research (Routledge, 2008) and Integrating Gestures: The Interdisciplinary Nature of Gesture (John Benjamins, 2011). She is co-convener with Kimi Nakatsukasa of AILA ReN (Research Network): Gesture, Multimodality, and SLA (2018-2021).
Johannes Wagner is Professor of Communication in the Department of Design and Communication at the University of Southern Denmark. In recent years he has been working on a comprehensive understanding of human social praxis as the nexus of verbal interaction, embodied practices and tangible objects in the environment. He co-edited Longitudinal Studies on the Organization of Social Interaction (Palgrave Macmillan, 2018) with Simona Pekarek Doehler and Esther Gonzalez Martinez and Objects, Bodies and Work Practices (Multilingual Matters, 2019) with Dennis Day.