Hi James

Nice job this morning - I found it interesting and stimulating, and you made me think again about certain things (training for example!).

A few thoughts that came to mind:

1. As you know it was possibly a bit long, so I think you maybe need to ask your people to set aside 1.5 hours rather than 1, or reduce the questions a bit
2. I kept wondering when the examples were going to come up - maybe mention these early on and say what you'd like people to do with them - either to use them illustratively throughout, or that you'll ask specific questions about them later
3. I'm not sure how much you are pursuing narrative now in these interviews, but I felt that you could have asked one or two 'life story' questions early on, even just as an ice-breaker. For example, asking people whether they had any exposure to multimodal methods in their own degrees, and/or what it is that has drawn them around to the practice of multimodality in their own courses (I guess you did open with this latter, to an extent).
4. Sometimes it felt slightly as if you wanted me to say things that aligned with an existing agenda - eg the need to re-think assessment, or the need for training - eg this exchange:

[10:11:46 AM] JAMES LAMB: This would seem to be a significant reconceptualisation of how the tutor is traditionally seen.[10:12:02 AM] JAMES LAMB: And you've already talked about the need to re-think assessment...

[10:12:23 AM] Sian Bayne: Yes - except I see this as a 'tendency' rather than a seismic shift

[10:12:41 AM] JAMES LAMB: That's useful - thanks for clarification...

[10:13:38 AM | Edited 10:15:04 AM] JAMES LAMB: Does this tendency then - alongside the need to rethink assessment - call for us to rethink the role and training of tutors in the digital classroom?

Obviously you *do* want people to talk about this, so it's really just a question of nuancing - I don't think I had actually said we needed to re-think assessment, at least not explicitly - so that moment struck me as very briefly jarring.

A minor point though, as overall though I think your interview style was very good indeed - focussed, friendly, and to-the-point without being overly restrictive or directive.

Best of luck with the upcoming ones - I hope we don't all fundamentally disagree and explose the MSc's assessment framework as riven by conflict and inconsistency! ; )

all the best,

Sian

Hello there, James,

Looks great. Impressively precise questions.  I just had a few observations, but the structure is perfect so I wouldn't change too much.

 1.  Does the consent form refer to confidentiality and anonymity? If not, you might state that upfront, that their identity will remain anonymous, etc.

 2.  What type of analysis will be applied to the transcripts? Narrative? If so, I suggest less is more in terms of the number of questions. Ask one, and then prod and probe for more explanation.

3.  In fact, consider jotting down a few probes to spur greater reflection on the most important questions.

4.  Is this question on topic? "It is suggested within the literature that the ability of students to present ideas in new and innovative ways necessitates the need for tutors to be re-trained. How do you feel about this?" By this, I mean does it advance any of your research questions? I am thinking tutor retraining is a separate issue from assessment or maybe I am wrong?

5.  I might consider having a "must-ask" list of questions followed by nice to have questions. I think an hour would be difficult to get through all of this.

6.  Same for this question: "Across the literature it is suggested that students are able to draw on a growing range of technologies and modes to present their ideas. Based upon your experience on the MSc in E-Learning, do you feel that students have exploited the multimodal potential of the range of tools from which they might craft ideas?" Does this relate to assessment? The tools aspect seems secondary to what you are after.

7.  Reflect briefly on whether the questions are too rigid for long answers. I think the greater the structure of the question, the more specific (and shorter) the answer. Do your questions provide the participant an ownership stake in the conversation? If so, they talk more. But I think your questions are great. Just consider versions that are more open-ended and couched in everyday language.

Useful interview feedback03/02/2012

Earlier today I met up with my colleagues/leaders from the New Geographies of Learning project. Having completed the first phase of the project, I was keen to collect some feedback on how my co-rearcher Michael and I had approached the collection of data through text interviews. With a view to feeding the suggestions forward into the research I undertake on the RM course and in my dissertation, this blog seemed like a good place to collect the different points.

Positives

• The tone of the interviews was conversational
  
• Our knowledge of the community was used well to stimulate discussion and seek clarification
  
• We avoided the use of leading questions 
  
• The interviews didn't feel awkward
  
• Other than on one occasion, the interviewees seemingly felt comfortable with the approach
  
• The text format seemed to work in this setting
  
• The interviews collected lots of relevant data

Possible weaknesses or room for improvement

• One some occasions, perhaps we could have been more questioning of points made rather than simply attempting to keep the conversation moving along e.g. being a bit more critical rather than the standard 'that's interesting...'
  
• On one occasion, where the participant and interviewer had studied together, there was perhaps too much focus on the nature of that common study experience

Also of interest

• On more than one occasion, there seemed to be a bit of envy from the participant on why two of their peers had been afforded this opportunity
  

In the discussion that followed, two other points arose. First, that opinion varied between different team members on how the interviews should have been undertaken, perhaps based upon their own background or conceptual stance (although I couldn't attribute it to that with certainty). For instance, would more depth have been achieved had voice been used to collect data? On the other hand, wasn't text the most appropriate approach bearing in mind the nature of the participant group? Furthermore, although voice interviews might collect more information more quickly, the transcription process would be a time consuming exercise. A middle ground was also suggested where the text interviews would have been accompanied by some voice interviews. 

I didn't feel in a position to contribute to this particular discussion, however I found it a really useful demonstration that within research design perhaps there isn't a single right way. The approach calls for pragmatism.

First of all, thanks for agreeing to participate in an interview this morning. I anticipate that this might take around an hour. My approach will be to ask you a small number of questions, however I want to give you plenty of space to tell your story and describe your experiences of multimodal assessment. Can I check that this is fine with you and that you are happy with the nature of my research, included within the participant consent form?

Before we begin, I want to briefly talk about multimodality, simply because this is a contested term. The working definition I am using of multimodality is of a piece of work - an artefact - that simultaneously represents ideas across a range of modes, such as voice, words and image. An example of a multimodal artefact could be a video that uses a voice-over to accompany images or words on screen. For my research, I'm interested in  multimodality and assessment, and in particular the experiences of tutors on the MSc in E-Learning. If you're happy, we'll begin.

Can you begin by telling me a little bit about how you are involved in assessment within the MSc in E-Learning?

When it comes to assessment, do you actively encourage students to represent their ideas multimodally?

Has the concept of multimodality informed or influenced your design of assessment within the MSc in E-learning?

In your experience, to what extent do students on the MSc in E-Learning exploit the potential of multimodality when it comes to assessment?

When a student submits a multimodal artefact for assessment, are there implications for you as a marker (that are specific to the fact that ideas are represented across a range of modes)?

Within the literature it is suggested that a shift towards multimodality prompts a need to rethink assessment criteria that were created with text-based 'essayistic' work in mind. How do you feel about this notion?

One of the characteristics commonly associated with non-traditional multimodal work is that they do not follow a traditional sequential manner - introduction-discussion-conclusion, for instance - and instead that the reader navigates her own way through the presented information? How does this match up with your experience of assessment on the MSc in E-Learning?

When confronted with an assignment where there isn't a clear pre-determined route through the work, can you tell me how you have approached this?

Across the literature it is suggested that students are able to draw on a growing range of technologies and modes to present their ideas. Based upon your experience on the MSc in E-Learning, do you feel that students have exploited the multimodal potential of the range of tools from which they might craft ideas?

In this final part of our interview, I would like to invite you to tell me what you think about two assignments that were submitted for the E-Learning and Digital Cultures course on the MSc in E-Learning, and are publicly available on the course website. I'm interested to hear, as a marker, how you feel about these two assignments as multimodal artefacts?

It is suggested within the literature that the ability of students to present ideas in new and innovative ways necessitates the need for tutors to be re-trained. How do you feel about this?

I think that's all that questions I would like to ask. Before we end however, is there anything that you would like to add about your experience or thoughts on multimodal assessment?

In which case thanks once again for taking time to speak with me today. If you would like to add anything after the interview, please don't hesitate to get in touch. Otherwise, good night and good luck.