LEARNING SPACES / DIGITAL EDUCATION / MULTIMODALITY / SOUND
A flying visit to London to attend two sessions concerned with multimodality. Yesterday evening (14 April 2016) I visited the London Knowledge Lab for a meeting of the Visual and Multimodal Research Forum, followed today by Multimodality in Social Media and Digital Environments from the New Media Group of the British Association of Applied Linguistics, this time at Queen Mary University of London. My flight back to Edinburgh tonight is delayed so I’m filling the time by gathering my thoughts over an expensive coffee.
At the Visual and Multimodal Research Forum, Dr Elisabetta Adami, University Academic Fellow in Multimodal Communication at the University of Leeds, presented some ongoing research where she is taking a multimodal approach to investigate the experience of super diversity in Kirkgate Market in Leeds city centre. I had the chance to spend some time chatting with Elisabetta in the post-forum debrief where it emerged that we share a number of research interests. First, her work around understanding experiences of the Kirkgate Market isn’t so far from my own interest in how we can take a multimodal approach to understanding our relationship with the urban environment. Add to that Elisabetta's interest in multimodal assessment - she teaches a course on multimodality - and we had lots to talk about.
And then today was dedicated to Multimodality in Social Media and Digital Environments where I presented the following paper about tutor experiences of multimodal assessment:
Judging by the conversations which took place over lunch, and the Twitter commentary that accompanied my presentation, my discussion of the ways that multimodality affects assessment seemed to strike a chord:
Now in brief summary - my flight has now moved from red to green on the Departures screen - here are three of the ideas that emerged most strongly from the different presentations, workshops and conversations over the last day-and-a-half.
The journey continues.
With thanks to Sophia Diamantopoulou (University College London), Agnieska Lyons and Colleen Cotter (both Queen Mary University of London) for giving me the chance to attend the sessions described above.
To set the scene, we can describe any presentation or public speaking occasion as a multimodal event in the way that it draws on a range of different semiotic resources to communicate meaning. In fact the presentation setting would seem to be a very effective enactment of Carey Jewitt’s description of multimodality when she proposes that:
This is illustrated in the annotated photograph below, capturing one of the project teams pitching their proposed textile range. Each of the stars in the image represents what we might understand to be a mode, all with their own ‘special powers and effects’ (Kress 2005, p 7) when it comes to the communication of meaning. This includes (but is not limited to) gesture, posture, eye contact, language (written and spoken), image and so on. From there we can go on to look at the meaning carrying potential of typeface, colour and layout within the visual realm, before turning an ear to the signifying power of pitch, volume and tone within the oral dimension. That the different modes are represented through stars is in line with the conceptualisation of multimodality as a 'constellation’ of different resources within a single communicational event (see for instance Carpenter (2009), Flewit et al. (2009) and Merchant (2007)).
'About more than language': Textile Design students use a full range of communication resources to convince us to invest in their bespoke flooring.
Looking at the assessment criteria for the presentation exercise, students were encouraged to think about how they could communicate their ideas orally (how they spoke), visually (the content of their PowerPoint slides) and also physically (eye contact and smiling, for instance). Marnie explained to me that the criteria build upon her own professional experience as a textile designer, and recognise the need for her students to "stand up and defend their work in front of clients" in the future. This immediately reminded me of the work of Kimber and Wyatt-Smith (2010) around multimodal assessment, where they emphasise the need for assessment practices to 'authentically' encourage the development of skills that young people will need when they enter the workplace, which includes the ability to think and work creatively.
Of course, multimodality is not simply concerned with documenting the different meaning-making resources that are at play, as my annotated image has started to do. As the latter part of Carey Jewitt's definition highlights, we are also interested in the relationship between these different modes, which in turn shapes how meaning is constructed and conveyed. When a presentation is delivered we are influenced by what is said (the words), but at the same time how it is said (pitch, tone and volume for instance) , the accompanying use of body language, the supporting visual content, what the presenter has chosen to wear and other resources that each carry meaning. As an audience we then interpret our own meaning from the combined effects of all the different resources. Consciously or subconsciously, our interpretation might be shaped by how all the different resources cohere (good!) or collide (probably bad). Therefore within a single communicational act - such as a presentation - the different modes sit in relation to each other, not in isolation. With this in mid, to the earlier constellation diagram we can now add all these different lines of interaction, emphasising how the 'inter-modal synergy' as I'm calling it, generates meaning.
Inter-Modal Synergy or The Celestial Plough meets Tron
To bring this back to the presentations taking place in the Textile Design class, the work I found most convincing was by a team who seemed to have spent time considering how they could convey a message of professionalism through all the different communicational modes stipulated within the assessment criteria. Their oral delivery (spoken 'in character'), the design of their slides (attention to layout, font and white space) and use of physical communication (eye contact with the audience, open body language) seemed to weave together into a single coherent message: "We're serious about this work...now trust us with your money." In contrast, on other occasions the use of spoken language was sometimes out-of-step with the quality of ideas being conveyed within the slides, for instance when a group opened their presentation by asking “Should we introduce ourselves, Marnie?” The spell was broken and I was reminded that I wouldn't, after all, be investing in colour-changing brollies, chic children's clothes or any of the other fabulous ideas that were put forward on the day.
Returning for a final time to Carey Jewitt's articulation of multimodality, it is also a useful way of looking at what was not present within the student presentations. Whilst recognising the limitations of what can be achieved within a timed five-minute presentation, and the need to satisfy the stipulated assessment criteria, none of the groups exploited the space of the room as a resource (which Jewitt has elsewhere pointed to as a modal resource, for instance through the ability of teachers to enact power relations by walking around the classroom). At the same time, the tactile qualities of the different products remained trapped on screen. Even though it wasn't anticipated in the assessment criteria, and was perhaps even deemed surplus to requirements bearing in mind the background of the wider audience, I wonder whether teams might have come over as even more assured if they had stepped away from the computer terminal to hand us examples of the textiles they intend to use. Perhaps, though, this is being saved for the assessed exercise next week when the teams will present their wonderfully colourful, creative and multimodal work 'for real'.
With thanks to Assistant Professor Marnie Collins and her BA Textile Design students.
CARPENTER, R. 2009. Boundary negotiations: electronic environments as interface. Computers and Composition. 26, 138-148.
FLEWIT, R., HAMPEL, R., HAUCK, M. and LANCASTER, L. 2009. What are multimodal data and transcription? In The Routledge Handbook of Multimodal Analysis. Jewit, C. (Ed) (London, Routledge): pp. 40-53.
KIMBER, K. & WYATT-SMITH, C. 2010. Secondary students' online use and creation of knowledge: Refocusing priorities for quality assessment and learning. Australasian Journal of Education Technology, 26, 607-625.
KRESS, G. 2005. Gains and losses: new forms of texts, knowledge and learning. Computers and Composition. 22(1): pp. 5-22.
JEWITT, C. 2009. An introduction to multimodality. In The Routledge Handbook of Multimodal Analysis. Jewit, C. (Ed) (London, Routledge): pp. 14-27.
MERCHANT, G. 2007. Mind the gap(s): discourses and discontinuity in digital literacies, E-
learning, 4 (3), 241-255.
Over the last six weeks I have been undertaking participant observation in the Edinburgh School of Architecture and Landscape Architecture (ESALA) as I have sought to understand how students construct and convey meaning through their design work. This exercise is the subject of an assignment I am completing for an Ethnographic Fieldwork course, whilst at the same time being of direct interest to my Doctoral research where I am investigating multimodal assessment in higher education.
During each visit I spend time observing students as they work on the design of a library building. For the most part my notes have been in ‘text’ form (typed onto my laptop), supported by photographs of models, sketches and other examples of student work. These fieldnotes have worked up to a point, however they haven’t managed to recreate the atmosphere in the design studio: they are muted both figuratively and literally.
On my most recent visit to the design studio last Friday, I decided to complement the gathering of words and images, with sounds. To do this, I walked the length of the studio (and then back again) recording sound through the voice memo function on my phone. I wasn’t interested in recording full conversations, but was instead keen to capture the traces of laughter, music, dialogue and other sounds which contribute to the ambient personality of the studio. The subsequent task of ‘writing up’ my notes later the same day also called for a different approach from normal, as I pulled together a short video combining a representative range of audio and images, juxtaposed with extracts from my typed notes. This is what I came up with:
I think the orchestration of images, sounds and words has enabled me to represent the colour and vibrancy of the architecture studio in a way that I wasn’t previously able to achieve. I think this multimodal approach to data gathering and representation offers something different to the conventional monograph with its heavy privileging of words on page. That said, gathering and interpreting visual and aural data presents its own challenges - both practically and critically - and we should remember that photographs and sound recordings are neither neutral nor an accurate recreation of reality. Nevertheless, in light of Architecture’s interest in a wide range of semiotic materials, I wonder whether an ethnographic approach which pays due attention to the aural and visual, is a more apt way of investigating how knowledge is constructed and conveyed in the design studio.
A creative approach to demonstrating success
Urban flânerie as multimodal autoethnography
Tomorrow morning (Thursday 15 January 2015) I will jointly deliver a conference paper on Urban Flanerie as Multimodal Ethnography, with my colleagues Jeremy Knox and Michael Sean Gallagher. The occasion is the Multimodality: Methodological Explorations conference at the Institute of Education/University College London.
The main thrust of our paper will be that we can better understand urban space by stepping into the shoes of the Flâneur, and then setting out to capture and then convey the sights and sounds we experience as we wander through the city. To make this argument, we will spend today, the day before the Conference, enacting our proposed methodology within the EC1 postcode of London.
What we’re interested in doing is exploring how the collection and then the multimodal communication of data might enable us to ask questions about the character within a particular snapshot of time, as well as our own relationship with urban space.
Beginning at 10am, we will wander the streets of East Central London, taking photographs and recording audio of anything that talks to us about personality of this part of the city. Following in the path of the Flâneur, we don’t have a set route in mind, or the intention to visit particular buildings, parks or pubs. Instead our wanderings and therefore our data collection will be shaped by a left turn here, an interesting alley there, a blocked-off pavement, an enticing cafe, the weather, and so on.
Once we have tired of walking the streets, we will sit down, download and then trying to make sense from our gathered visual and aural data. We will then decide how we might combine all the gathered sights and sounds into a meaningful, multimodal artefact. After that we will reflect on the methodological significance of our exercise. And then tomorrow morning we will present the same conclusions – and the artefact – at the Conference.
The tight deadline presents this as a risky approach and it could end up being a late night (but then, we'd need to fill our evening in central London somehow). We feel though that it's a useful way of testing our methodology in a practical way. Assuming we pull this off, we feel there is the potential for this approach to be used within different learning situations, which is something we hope to discuss tomorrow morning. For now though, it's time to gather our cameras and coats and to step out into the cold of EC1.
Earlier this year I spent a day in the company of Michael Sean Gallagher, Jeremy Knox and Philippa Sheail as we set out to undertake a loosely structured day of multimodal data collection in Edinburgh. Michael, Jeremy and Philippa were all due to contribute to the Networked Learning Conference taking place in the same city in the days that followed, while Michael and I would separately be delivering a session about multimodality and digital learning spaces for an online tutoring course at Edinburgh University. Rather than taking time to make final changes to our respective presentations, we instead agreed to take part in what we vaguely described as a ‘multimodal mapping exercise’.
We weren’t exactly clear what we hoped to achieve, other than to try and collect a range of data that somehow captured a sense of the city during the specific period of time that we experienced it. On reflection, the notion of ‘experiencing’ a city seems overly passive and I wonder whether ‘enacting’ might be a more useful description of our approach. Irrespective, our intention was to wander the city in a way that avoided Tourist Guide Edinburgh in favour of capturing a multimodal snapshot of everyday life. What this meant in practice was that wherever we stopped within view of what might be regarded as a site of beauty or historical interest (and to be fair, Edinburgh's a good looking city), we would search for the untold stories to be found in the street furniture, graffiti and general detritus of our surroundings.
Actually, that's an incomplete way of describing how we tried to get a sense of our surroundings in that it ignores the soundtrack that accompanied and shaped our journey under bridges, through closes, into bars and restaurants, and to other neglected corners of the city. For instance, the sound of an industrial extractor fan within Greyfriar's Churchyard seemed incongruous and therefore significant. I imagine that when feature films use this location, a runner would be dispatched to the adjacent pub to plead with the kitchen staff to switch off the industrial-scale appliance. For our own purposes however, it felt important to capture how the industrial clashed with the picturesque, as if those at rest sleep to the white noise of machines.
On a technical level, we used a range of devices for the purpose of capturing data: iPhones, iPad and a camera. Strong coffee and Tunnocks Caramel Bars also helped our field research in a less direct way. My pen and notebook remained unused, although in hindsight it would have been helpful as a way of recording the specific sites of data collection. That said, Michael usefully tracked our progress using the Trails app. As well as offering a reminder of the specific path we followed, by the time that Michael's iPhone drained of power, the Trails read-out told us that we'd explored more than 14km of Edinburgh's pavements, parks and public houses. With a little more planning we could probably have borrowed some devices that would have produced a better standard of visual and aural material, however I'm relaxed about the mixed quality of what we captured. In fact, perhaps a snapshot shouldn't be too polished: the flaws in the sound clips and photographs in themselves reflect the imperfections of the city.
Reflecting the wider approach to this exercise, by the end of the day we didn't have a clear plan of what we would try and do with the collected data. Thus, over late night food and last orders it was decided that we would each set out to remix the data in our own way. Quick off the mark, Michael blogged about our exercise within days and has since created a multimodal video postcard. It has taken me longer to settle on an apt way of capturing a sense of the city as I've toyed with a few unsatisfactory and uninspiring representational forms. In hindsight, I've been guilty of over-thinking how I might place the modal fragments onto on a digital canvas (and what form this canvas might take), not least as this considered approach was at odds with the haphazard and unrestricted way in which we approached the exercise itself.
What I opted for, as seen and heard here, is an interactive map and an accompanying aural compilation (as well as this text-based commentary). It was important that the audio was experienced concurrently with the map: after all, if we understand multimodality to be concerned with the simultaneous juxtaposition of a range of modes within a single communicational act (and I'm borrowing from Kress and from Jewitt here), then the aural and visual components needed to be configured accordingly. And of course, we didn’t experience Waverley Station with the volume turned down and therefore neither should you.
Looking and listening to the presentation of data here, I think I've created an accurate if incomplete record of how we enacted everyday Edinburgh on Sunday 6 April 2014. The images, sounds and words offer a flavour of the city but can't capture the taste and smell of the IPA or 80 Shilling, or adequately account for other sensory experiences that shaped our engagement with our surroundings. Nevertheless, our focus on detail and detritus tells stories about Edinburgh that, I think, are ignored in the conventional narrative of Scotland's capital.
For instance, the prospective undergraduate student might be less interested in the opening hours of Holyrood Palace than the fact that, at the bottom of Blackfriar's street, equidistant between the University's School of Education and the Cowgate's sites of late night revelry, is a place to access the Internet whilst enjoying a late night kebab.
On Infirmary Street meanwhile, an innocent house number has become a battleground for a generations-old political conflict. As far as we could see, nobody had been defacing the city in the name of the pioneers, writers and thinkers that are more commonly and conventionally celebrated in Edinburgh's story (although we did question how David Hume might feel about the damage done to city's skyline in his name).
Our focus on the everyday is a useful reminder that Edinburgh is understood through parking tickets as well as concert tickets. And for every Michelin Star there are a thousand takeaway restaurants that keep the people active and unhealthy, spreading a trail of discarded plastic over the ancient, cobbled stones of this beautiful city.
James Lamb, 10 May 2014
I am an ESRC-funded Doctoral student in the Centre for Research in Digital Education at The University of Edinburgh.