Review: Multimodality

Nuts & Bolts

Condensed review focusing on the most salient arguments of the book, and how they are managed within Multimodality.

read more

Negative

What are the problems with Multimodality? This focused section provides analysis of the issues arising in the book.

read more

Positive

Interested in the positive aspects of Multimodality? This section provides the most useful arguments of the book.

read more

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Nuts & Bolts

 

Multimodality: A social semiotic approach to contemporary communication is my second experience with Gunther Kress. A heavy theoretical text, filled with terminology, it offers several ways to look at the different modes of communication; Kress offers lenses through which we can look at both communication itself and visual representation within communication.

Some of Kress's claims are broad-sweeping, whole arguments boiled down to easily digestible kernels such as “there is no meaning without framing” (p. 10, italics in original), “Knowledge and text are entirely linked” (23), “Communication is a quintessentially social activity” (p. 51, italics in original), or my favorite: “All communication is movement” (p. 169). These statements – each well supported within the text – are wonderful markers along the journey Kress is taking, acting as sign posts to help get through the intense theory in the majority of the text.

There is some difficulty following Kress and knowing what he is trying to say. It's not that Kress isn't trying to be clear; if anything, the problem is that he is trying too hard to be clear. He takes so much time explaining and delving into all the tangential meanings that the original point ends up lost or, at best, heavily obfuscated by the surrounded text. While Kress says something as clear as “there is no meaning without framing” (p. 10, italics in original), he says it amongst a full four paragraphs of explanation. The quoted sentence is very clear, and does act as a signpost, but the amount of explanation required for Kress to get to such a simple statement muddies the waters.

The excessive explanation may be necessary; in many ways, Kress is blazing a new trail, going to new ground, and he needs to do a lot of establishing in order to work in this new space.

However, this is Kress's method, and he makes no apologies for it. In fact, early on he says that “Developing ways of thinking about this at once simple and complex phenomenon – that is, setting out a social-semiotic theory of multimodality – is what this book is about” (p. 5, italics in original). In order to develop new ways of thinking, it makes sense that we need a lot of new terms. The terms themselves are not complicated, but the ways Kress uses them tend to be, so we need some explaining; Kress tells us what he means when he says “writing” “image” “colour [sic]” “names” “shows” “style” “mean” “frames” “highlights” and “read.” These definitions may be essential building blocks to the arguments Kress makes throughout the book, which is why he defines all of these before even beginning his argument. This allows him, later in the book, to make much more complex arguments without having to stop and redefine the words.

The definitions are not complex, and Kress offers them up just so that we will be on the same page as he and so we can follow along his argument. There is no quiz at the end; as readers, we're able to look at these definitions as he shows them to us, then let them fade away, confident that, when they become important, Kress will bring them out again, and we will at least recognize them.

His argument is to extend the world of meaning, to show us that semiotics needs to stretch further than we have thus far allowed it to; that is, he is using the work of semiotics, which has thus far been applied primarily to text, and applying it across multiple modes of communication. He tells us that “'Language' isn't a big enough receptacle for all the semiotic stuff we felt sure we could pour into it” (p. 15). We have to accept that words are not enough; there are methods of communication we need to employ that aren't presented with just language. We have to reach beyond language and towards visual spaces, to the other modes of semiotic-conceptual work (p. 15). It is not an easy thing to do; there is much involved in what is essentially a reinterpretation of existing theory coupled with an exploration into new territory. As Kress says:

 

The task is to establish, with as much precision as we might, what these differences are, in specific cases and circumstances. What new kinds of questions emerge and are made possible; how do persistent, older questions get recast, in ways possibly that lead to more plausible answers; and who might benefit in what ways from the different answers (p. 15).

 

If he is successful, there will be new areas for communicators and rhetors to explore, new questions to ask, and, more importantly, new ways to answer old questions.

Kress shows us that “Signs are means of making knowledge material. Signs-as-knowledge are tools in dealing with problems in the sign-maker's life-world” (p. 30). By emphasizing the use of signs (rather than words), he continues to stretch the idea of what can be used to communicate, and what belongs within the field of semiosis, and semiosis is a large field. In fact, according to Kress, “Semiosis, the making of meaning, is ongoing, ceaseless” (p. 93). Meaning is extended with every expressed communication. This very review is an ongoing exercise in semiosis, as Kress would suggest.

This expansion of semiosis comes into play because, as Kress says, “Writing, previously the canonical mode par excellence, is giving way to image” (p. 133, italics in original). This statement, arguably the core of the argument Kress is making, tells us about a shift many of us have seen around us. The rise of images is pervasive; there is no way to live in our culture without seeing it. The simple fact is that images are good rhetorical tools, and so images are used more frequently. Kress tells us that “Rhetorical considerations do still organize all of communication, all semiotic interaction, at all times, even in times of stability and the dominance of convention” (p. 45). Those rhetorical considerations lead us to the use of images more and more frequently, and the shifting of the 'canonical mode par excellence' as Kress suggests.

Kress is not suggesting that the image is, or ever will, completely replace the written word. That is not his aim, and to think it is would be to miss the point. Kress is telling us that image (along with sound, moving pictures, etc) are supplemental and add to writing. This is not to say that there is a superior/subordinate relationship; each mode is an independent form of communication that conveys messages differently. That's what multimodality is: literally, it is the use of multiple modes. Different modes work in different situations, and as rhetoricians, we need to understand those different modes. Kress takes this opportunity for another definition: “Multimodal design refers to the use of different modes – image, writing, colour [sic], layout – to present, to realize, at times to (re-)contextualize social positions and relations, as well as knowledge in specific arrangements for a specific audience” (p. 139, italics in original). By using multiple modes, we engage in multimodal design. As obvious as this sounds, it's an important idea. Once again, this is Kress showing us that the other modes add to text rather than replace it. Text remains the result of the sign-maker's choices. But Kress shows that all modes are the result of the sign maker's choices. He writes that “Texts are always multimodal, so the rhetorical and design decisions lead to the making of ensembles of modes; these are themselves assembled as orchestrations of modes, in which purposes and needs of the maker of sign-frames are brought together with affordances in the best possible manner, from the rhetor's and the designer's perspective” (p. 157, italics in original), showing us that the rhetor and the designer are often – if not always – the same person, and when they are not, they need to work in concert with one another to achieve the desired goals.

All in all, this is a worthwhile book, an interesting project with many ideas and many, many terms that can be used for further examination into the field of social semiosis. While there are many holes in what Kress writes, those holes are not the result of sloppy argument; they are the places that Kress has opened up for further theorizing. Semiotics is, as Kress said, ever expanding, a continued process of meaning making. Multimodality is both a theoretical work and an exercise in what it theorizes about.

 

Reviewed by Joe Weinberg

University of Minnesota, Twin Cities

Reviewed by Joe Weinberg

University of Minnesota, Twin Cities