Advances in the study of semiotic repertoires: mobilities, modalities, spatialities

Abstract

Recent years have seen a revitalisation in the use of Gumperz and Hymes’ (1972) notion of the verbal repertoire. This is largely due to shifts in conceptual understandings of language use, and moves away from bilingualism and code-switching to concepts such as (trans)languaging. In line with this shift, researchers have used various overlapping terms such as “linguistic repertoire”, “communicative repertoire”, “spatial repertoire” and “semiotic repertoire”. This colloquium engages with the latter: the concept of the semiotic repertoire, defined as the totality of resources or tools that people use when they communicate (such as speech, image, text, gesture, sign, gaze, facial expression, posture, objects and so on). Importantly, Kusters, Spotti, Swanwick and Tapio (2017) argued that the lens of semiotic repertoires enables synergies to be identified between sign language research, gesture studies, spoken language linguistics and provides a holistic focus on action that is both multilingual and multimodal. This symposium examines how the semiotic repertoire is distributed, experienced and embodied among diverse population groups. Paying attention to ideological and practical (assumed) hierarchies of resources (which are contextual and situated), we theorize on what the study of the distribution of resources, spatiality, moral orientation and emotions means for the conceptualization of the semiotic repertoire.

Analyzing spatial repertoires from a New Materialist perspective
Suresh Canagarajah, Pennsylvania State University

Though applied linguists have expanded their orientations of analyzing language interactions to spatial repertoires, their analysis is largely influenced by the traditional structuralist assumptions of methodological individualism, human agency, and cognitive representationalism. Typical analysis is influenced by the perspective that individuals agentively use the spatial repertoires to convey their predefined meanings. Scholars from New Materialist orientations (Barad, 2008; Coole & Frost, 2010) have recently persuaded us to consider, alternatively, the role of distributed practice, the agency of matter, and nonrepresentational thinking. From this perspective, we have to analyze how meanings emerge through the collaborative work of participants in relation to the agentive contribution of semiotic and material resources. 

Drawing from my research on the communicative practice of international STEM scholars, I illustrate how an orientation to spatial repertoires can move beyond the structuralist assumptions. I narrate the case study of Gunter, a Turkish doctoral candidate in entomology. In a co-authored article that reviews 79 publications on the causes for the overwintering of honeybees, Gunter attempts to represent the state of the art in a figure. He is dissatisfied with his attempts because he doesn’t have the skills and technical resources to represent what he has in mind. Thereafter, he collaborates with a technician in his lab. As they engage in drawing, Gunter finds that the activity clarifies his ideas and generates a new hypothesis that proves to be significant for his future research. He stated in a stimulated recall interview that the visual artefacts and the drawing activity produced a representation that he didn’t possess from published literature or his own prior analysis. 

Semiotic repertoires: Learning from the semiotic ecology of an online, video-only-mediated, classroom
Johan Hjulstad, Oslo Metropolitan University

The presentation builds on a microethnographic study of teachers’ and students’ signed language interactions in a video-mediated distance education classroom in Norway. In the distance education setting, advanced video-conferencing technologies were set up to create a virtual classroom for remote sign language learning students who were participating from different locations. The study explores how teachers and students employ various semiotic resources such as signs, gesture, gaze, head movements, facial expressions, body postures, object manipulations and the technology itself, when interacting together – and do so in a highly coordinated manner.

The investigation reveals that the participants quickly adapt to this video-only mediated environment and do so by mutually orienting and attending to various aspects of material and spatial resources in their collaborative meaning-making. The use of gaze and the alignment of bodies in relation to the mediated visual space plays a significant role for establishing a common ground. Teachers’ work of coordinating attention comprises a critical skill in adapting to this environment. Further, we find that in order to make sense in visual communication, it is not enough to see and be seen. Participants’ practices of validating that what is seen is also seen by others suggests that in order to communicate (at all) it is necessary to gain access to what or whom the interlocutor is looking at. These findings contribute to an emerging theory of communication that locates meaning-making (sense-making) primarily with the interpreter, in that meaning should not be based on what the producer does, but on what resources the interpreter can recognize (have access to) from the information the producer reveals. This is consequential for how we might understand semiotic repertoires.

Communicative repertoires between situated interaction, normative Discourse, and lived experience: Exploring processes of ‘language shaming’
Brigitta Busch, University of Vienna

With the recent revival of Gumperz’ (1964) notion of the verbal repertoire, now commonly referred to as linguistic/communicative/semiotic repertoire, many scholars have tended to locate the repertoire with individual speakers and their life trajectories. In turn, others see the repertoire as emerging from the particular social or spatial arrangements in which interactions take place. What is often underestimated is the importance of the bodily and emotionally lived experience of communicative interaction. These can be critical in preventing communicative resources from being deployed even though they are individually available and appropriate to the situation, or conversely, in mobilizing unexpected resources to achieve understanding.

Conceiving the repertoire as holding an intermediate and mediating position between interactions situated in time and space, (sometimes competing) discourses on linguistic appropriateness, and subjects’ emotionally and bodily lived experiences of language, this paper examines the interplay of these instances by analyzing narratives in which feelings of shame linked to particular ways of speaking/interacting are articulated. Understanding communication primarily as an inter-subjective and inter-corporal gesture towards the other (Merleau-Ponty, 1962), I discuss how in moments of shame the acting and perceiving subject-body turns into an object-body that views itself through the eyes of others. What then comes into play are norm setting and regimenting discourses on language and language use, power asymmetries, and processes of (self-)positioning up to exclusion.

Experiences of language shaming can leave long-lasting imprints on the repertoire when they are “inscribed into the body” (Butler 1997) because of their conspicuous affective character or frequent reiteration. Such embodied experiences can result in linguistic insecurity and in patterns of communicative practices such as language avoidance, silence, hypercorrectness, or speaking back. Taking into account the impact of bodily and emotionally lived experience of language, I suggest the importance of understanding communicative repertoire in similar ways to how Bakhtin (1981) understood language, that is, as existing “on the borderline between oneself and the other”.

Moral orientation and the mobile semiotic repertoire
Erin Moriarty Harrelson and Annelies Kusters, Heriot-Watt University

How is the use and expansion of the semiotic repertoire when engaging in international mobility tied up with moral orientations of interactants? Our approach to this question is based on linguistic ethnography with deaf people who engage in short-term forms of international mobilities such as conference attendance, sports competitions, and tourism. Deaf people engaging in international mobilities make use of semiotic resources that can include International Sign; the learning and use of different sign languages; gesturing; mouthing, writing and fingerspelling in different languages and scripts; speech; and drawing. International Sign (IS) is used between signers from different (sign) linguistic backgrounds, and it typically combines features of different sign languages, mouthings derived from English or other languages, and intensive use of iconic structures.

As evident in this presentation, mobile deaf people often quickly expand their linguistic repertoires by engaging in rapid, immersive and informal (sign) language learning and direct communication in IS because they are morally oriented towards each other (Green 2014). Other examples of morally-driven communicative practices include language brokering and the pooling and sharing of linguistic resources between two or more people in order to achieve understanding.  Project data demonstrate how mobile deaf people’s use of diverse communicative resources as they engage with interlocutors are shaped by ideologies that include moral ideas about what linguistic resources are most appropriate in specific contexts and/or with/by whom. Examples include ideas about the power and dominance of certain sign languages, especially American Sign Language (ASL). We show how the learning of foreign sign languages, resistance towards the use of ASL and English, and sometimes even IS, as morally correct behaviors are entwined with fears about linguistic contamination and hegemonic languages/modalities. These interactional practices thus foreground the accumulation and deployment of linguistic resources as a moral practice, sometimes resulting in situations where the most effective communicative resources are not used, in the interests of morality.