Historically, research on semiotics and multimodality has been dominated by approaches that focus on artifacts; that infer practices of production, reception, and use from those artifacts; and that aim to describe rule-governed systems of signs (on models of abstract grammars for languages) (e.g., Barthes, 1967; Kress, 2010; Kress & van Leeuwen, 2001). On the other hand, situated and observational studies of semiotic/multimodal activity have often focused on particular dimensions of interaction (e.g., talk, gesture, writing), typically limited to a focus on only one or two semiotic resources (e.g., Goodwin, 2003; LeBaron & Streeck, 2000; Gullberg, 2009; Waring, 2012). Recently, more expansive engagements with semiotic activities as well as artifacts have increasingly been embraced (e.g., Mondada, 2019; Thorne, Hellerman, & Jakonen, 2021; Smith, Pacheco, & Khorosheva, 2020). In this paper, we describe an integrative transdisciplinary framework for such engagements, SHES (situated, historic, embodied semiosis). Uniting multiple lines of theory/research, SHES offers an integrated account of communication, activity, and becoming (a more expansive framework than learning; e.g., Erstad et al., 2016). SHES emphasizes complexity and emergence as fundamental, both materially and biologically (Barad, 2007; Gilbert, 2019; Salthe, 1993).
SHES takes up Peirce's late semiotic theories (Jappy 2017; Peirce 1998), which switched from signs (products) to semiosis (the process), replaced a representational account with non-representational mediation, and flipped the relations of tokens to types in ways that align with dialogic notions of utterance and genre (Voloshinov, 1973; Bakhtin, 1986). SHES is likewise grounded in recent biological theories that highlight complexity and symbiosis and biosemiotic frameworks that challenge human-centered and Platonic semiotic ideologies (e.g., Gilbert, 2019; Kull, 2010; Salthe, 1993). SHES draws together lines of research that have situated language and other semiotic resources in communicative events (e.g., Goffman, 1981; Hanks, 1990; Irvine, 1996; Ochs, Gonzales, & Jacoby, 1994; Streeck, Goodwin, & LeBaron, 2011); have conceptualized communication and becoming as matters of dialogic histories (e.g., Linell, 2009; Wertsch, 1991; Volshinov, 1973), and have understood activity as always embodied and materially distributed across human and non-human (biological and physical) elements of functional systems (e.g., Cole & Engeström, 1993; Hutchins, 1995; Zittoun et al., 2013). SHES also highlights the central role of embodied metaphoricity in thinking and communication (e.g., Lakoff, 1987; Mittelberg, 2019; Müller, 2008). Locating the core phenomenon in activity in the world, these lines of research have increasingly shifted attention to semiosis, as nicely expressed in Agha's (2007) definition of language-in-use as "events of semiosis in which language occurs," events where language represents no more than "a fragment of a multi-channel sign configuration" (p. 6).
After sketching the SHES framework, we consider what it can contribute to applied linguistics research and practice in classrooms (focusing on academic literacies), clinics (focusing on aphasia), and social worlds (focusing on community and workplace communication). Theoretically, we highlight the implications of complexity and emergence in biological, material, and interactional theories and research. Ultimately, we argue SHES provides a powerful set of tools for understanding human and non-human action, communication, and becoming.
