Taking a few moments here to talk about my ongoing - and evolving - research around assessment practice. Over time the interest of my PhD has broadened from the phenomenon of digital multimodal assessment to also ask questions more generally about the way that assessment practice in the Humanities is affected by the societal and pedagogical shift to the digital. In particular I am interested in investigating how:
In relation to the third of these lines of inquiry, I am particularly drawn towards sociomateriality's attention to the way that meaning emerges from a broader range of influences, opportunities, limitations and pressures beyond human interest and action. I think this is neatly captured by Fenwick, Sawchuk and Edwards when they propose that sociomaterial research looks to take account of:
In this way assessment feels less like a transaction between student and tutor, or a measure of academic performance, and much more like an assemblage of the seen and unseen, the human and machine, and beyond. As such, sociomateriality (supported by critical posthumanism) has had the effect of lifting my conceptual gaze from the ways that knowledge is conveyed and interpreted, to also take into account what previously seemed peripheral (or invisible or irrelevant) to assessment. This in turn has meant extending my ethnographic fieldwork where I have been observing students and tutors from undergraduate courses in Architecture and History. I have continued to investigate what takes place in the lecture theatre, studio, meeting room, corridor and canteen: at the same time though I have taken two further approaches in order to get a better sense of the resources and restrictions that influence the preparation of a piece of a coursework, whilst also investigating how digital literacy practices are enacted beyond what I was able to observe in class and around campus.
For the time being I am referring to this method as ‘digital sociomaterial journaling’, thereby acknowledging how my approach is influenced by Gourlay and Oliver’s recent proposal of longitudinal multimodal journaling (2016). Combining ethnographic approaches with an interest in sociomateriality and New Literacy Studies, Gourlay and Oliver describe research where they gathered journaling data in order to investigate the digital engagement of a group of postgraduate students. Amongst other methods, participants were provided with iPod Touch devices in order to gather data that would ‘document their day-to-day practices with texts and technologies in a range of settings’ (2016: 302), thereby offering insights into their digital literacy practices.
As well as drawing inspiration from Gourlay and Oliver’s work, I have looked to some of my own earlier research where, along with my colleagues Sian Bayne and Michael Gallagher, we used the elicitation of 'digital multimodal postcards’ alongside semi-structured interviews to investigate how online distance students understand and enact their university, and how they construct space for learning (Bayne, Gallagher & Lamb 2013; Gallagher, Lamb & Bayne 2016). Here then is how these different methodologies have shaped my current research.
Inviting students to record their surroundings as they work on an assignment
For a period of approximately one week in the lead up to a recent essay deadline, five students from the American History course were asked to ‘record their surroundings' on every occasion they worked on the assignment. This included taking a photograph, making a one-minute ambient sound recording, and writing a short description of their location and activity at that moment in time. The data were then submitted electronically using a drop box on this website, via e-mail or USB drive. For the purpose of illustration, this is one of the six submissions that Sarah made as she worked on her assignment about the Civil Rights Movement.
Shadowing students as they work on an assignment
Two of the same students who recorded their surroundings also agreed to let me shadow them at different times as they worked on the essay assignment. In Karen’s case this comprised an afternoon in her flat followed by a later period in the main university library. For Harry meanwhile this involved a full day studying in one of the university's smaller libraries, as well as a nearby common room. As Karen and Harry worked on their essays (and drank tea, checked Facebook, listened to music and so on) I made my own sound recordings, took photographs and typed field notes. The following video gathers together representative sights and sounds from my first observation of Karen (although not as yet with the inclusion of entries from my field notes or reference to her Internet history for the corresponding period that she kindly agreed to supply me with).
The approaches described here were designed to shed light on the some on the recent interest of my research (bulleted above). For instance, how does the algorithmic code that is concealed, as Edwards & Michael (2011) suggest, beneath the sophisticated interface of software applications, influence the search results that appear in Google Scholar? How do perceptions and practices around plagiarism detection software influence composition (a concern recognised in research by Introna & Hayes (2011))? How does the use of sophisticated hardware and software pictured in the different images advance the notion of shared authorship between human and machine (see Knox & Bayne 2013)? Meanwhile, through the shadowing exercise in particular I have sought to gain insights into the ‘minute dynamics and connections’ that Fenwick et al. (2011, p.8) believe to be overlooked when we look to understand educational activities.
For the time being I am resisting the temptation to offer any sort of this response to these questions, not least as next month I will interview the same five students from the American History course. This will include discussion around the sights and sounds each student gathered as they worked on their essay assignment. Before that, for the purpose of comparison, tomorrow morning I will begin the same process all over again with five students from an Architectural Design course.
A note on ethics
Pseudoynms have been used in place of participant's real names. Students gave their consent to participate in the research described above, including the sharing of their supplied data. Participants were offered a £20 gift voucher for participating in each part of this research.
As part of my Doctoral research into multimodal assessment in the Humanities I am undertaking an ethnographic study of an undergraduate American History course. I observe lectures, tutorials and other situations where students and tutors gather to construct meaning, as they explore The Making of the United States of America. There are two reasons I wanted to spend time in a History class. First, in common with the majority of Humanities courses, assessment within History programmes tends to privilege language, commonly in the form of the essay. Second, with its interest in visual artefacts as a means of study - drawings, paintings, maps, photographs - History has an eye for the way that images contribute towards understanding. Bringing these two ideas together, I wondered whether History programmes might be open to assessment practice where attention is paid to the visual, multimodal character of student work. Now that I have reached the third week of the History course (and have a couple of hours before the next class), I am recording some early observations about the role that images have played within classroom teaching.
To begin with some context, this is a second year undergraduate course drawing students from a range of degree programmes. Three times a week the lecture theatre is packed with an audience of around 300, augmented by tutorials with groups of around 12 students each. In all of the classes the tutors have used PowerPoint presentations, with images to the fore. Something that really stands out from my field notes is that these images are always more than a backdrop: they appear central to the knowledge that the tutor wishes to convey. For instance:
In these instances the images work alongside the oral delivery, adding colour and context to what is being said. This isn’t to suggest say the lecturer’s oration and wider performance is presented in monochrome: on the contrary it is enthusiastic and eloquent. Simply, the images are vital in helping the tutor to convey meaning.
Click on images to enlarge. Slides reproduced with kind permission of Professor Frank Cogliano.
On other occasions the images are themselves the central focus of study. Cartoon depictions of individuals and events are used to prompt students to reflect on competing perspectives and attitudes of the time. Newspaper adverts and notices, variously drawing attention to slaves-absconding-or-for-sale, are themselves historical artefacts that demand discussion within the tutorial setting.
For the most part the images on screen are accompanied by reasonably small bursts of text (typed words), mostly single sentence captions providing factual information: title, subject, author, date and so on. Three weeks into the course and I have yet to face down a single bullet point. In terms of both prominence and placement, text immediately seems to perform a functional supporting role to the central positioning and critical purpose of the image. This however disregards the presence of text within many of the images: a political proclamation or newspaper notice may be presented in j-peg format however the conveyed meaning is heavily dependent on the use of text. If we narrow our gaze from the slide in its entirety to instead see an image as a communicational act in its own right, we become aware of the intricate assemblage of meaning conveying resources sitting in juxtaposition: text, font, colour, spacing, layout and so on. When this is combined with the tutor’s oral delivery (volume, tone, pitch, pace, silence) and physical performance (eye contact with the audience, gesture, posture, movement across the space behind the lectern) we see that the History classroom is richly multimodal (or ‘densely modal’ to borrow from Norris (2004)) in the way that meaning is conveyed and interpreted.
Picturing Thomas Payne in the richly multimodal History classroom
In contrast to the highly visual and multimodal character of meaning-making in the classroom, assessment for the American History course privileges the use of written language. Coursework comprises two conventional essays while 20% of the final mark is based upon a ‘practical examination’ in the form of a student’s contribution towards tutorial discussion. To be clear, I am not suggesting that the assessment design is flawed in taking what Newfield (2011) and others might describe as a ‘monomodal’ approach: I am simply drawing attention to the way that it differs from what takes place in the lecture theatre and the tutorial room. When I come to interview course tutors at the end of semester I might find there is a very good reason why measurements of understanding and ability rely on language in its different forms. For the time being however, I have three questions to reflect upon in the coming weeks and months:
Before that however I have a lecture on The Origins of the American Revolution. I expect there to be bullets, but no bullet points.
Assessment, feedback and multimodality in Architecture
Multimodality and the presentation assignment
"I'm just glad it's not an essay!": a poster presentation assignment in music
I am an ESRC-funded Doctoral student in the Centre for Research in Digital Education at The University of Edinburgh.