‘Phenomenography is an approach for identifying and describing qualitative variation in individuals’ experiences of their reality.’ Marton(1981) quoted in Sandberg (1997)
Sandeep S, Richard F Home
Last amended: 7 April 12.30am RF
Last amended: 5 April 1.43pm RF
Previous amended: 2 April, 2005 18:45 hrs by SS
This document should assess the risks of what could go wrong in the study, in light of the theory relating to reliability and validity. The outcome should be a list of action points that are operationalised in the interview protocol, analysis plan, and write up(?).
Issues | Action | Other studies processes | Interview
The right question
- The right level of analysis (??)
- Management question -> justification (Tim to write), and signoff by the group and Jorgen
Sampling risk
– to ensure process in place to reach theoretical saturation (not possible in five interviews, but process to outline for full study)
Theoretical saturation, a result of theoretical sampling, picking the next data collection on the basis of issues driven from the emerging theory, to pursue variation, to explain discrepancies in the theory. (Glaser and Strauss 1967, Flick 2002 p64-5)
Saturation means no further enriching of the categories discovered. (Ibid p.65)
Suggested process: to think and add (rf)
Interviewer bias
- Bracketing interviewer’s/s presuppositions
- Avoiding ‘why’ at all costs
- Interviewer being oriented (tuned subconsciously) to the phenomenon (Jorgen, 1997).
- Interviewee or background unknown to interviewer (neutrality/ objectivity/ isolation)
Analysis
- Identifying commonalities leading to category identification
- Focussing on different aspects of the variation in experience
- Relationships between categories
Presuppositions presupposing the answer -> bracketing ->
- Writing down our current opinions (avoidance)
- Bracketing (our opinions re) possible outcome space arising from the study
- Working individually on transcript analysis – might lead to different categories n relationships – disagreements to be sorted out in group meetings.
Reliability
- Appropriate selection of interviewees (Should students nearing confirmation be excluded?/ international vs domestic students).
- Use of appropriate questioning (Should there be two interviewers or one? – Barley)
- Maintenance of a non-judgemental, respectful and emphatic attitude towards interviewees
- Detail the process by which bracketing is done – Adding to R&V.
- Following the epistemology of intentionality (five steps) suggested by Jorgen (1997)
-
Influence interviewee
- How much (background information re study) should be provided to interviewees? Or should the purpose of the study be disguised?
- Interviewer and researchers should be unknown to each other (plausible?)
Inconsistency between interviewers/ interview process
- Maintaining consistency (e.g. timings)
- Interview protocol
- Appropriate prompts
- Techniques for winding up (without hurting the prescribed phenomenographic process)
Well.. am tired now…L
- Coding analysis in pairs (independently?)
- Interviewing in pairs (handling of questions?;
- Recording interview to identify any influencing behaviour by the interviewer
- one person to attend all interviews (overview) refer Barley article
- independent coder (Sandberg 2000)
- bracketing presuppositions ie (Sandberg 2000)
- discussing process with interviewees ie communicative validity (Sandberg 2000)
- discussing results with interviewees for feedback that they make sense ie pragmatic validity. Observation would be included in the full study, but not in the pilot, to confirm they are consistent with the results of the analysis (Sandberg 2000)
1. Sandberg(1997) Are phenomenographic results reliable?
Jorgen suggests that interjudge reliability is derived from an ‘objectivistic epistemology within the positivistic research tradition’, citing (Giorgi 1989). Interjudge reliability is the use of a second coder to look at phenomenographic data to see if they find the same categories as the original coder. The categories are provided as part of the process. This process may be unreliable if the data is poor, or coloured by the researcher’s pre-understanding, or superficial, so general categories are found rather than a specific conception. Jorgen claims the basis of phenomenography is a ‘phenomenological epistemology’, that is a subjective conception of reality, thus alternate processes of reliability are required. Consistent with Sandberg(2000), interpretative awareness is suggested as a substitute, that is the subjectivity of the interpretation must be dealt with, identified, controlled and checked. Jorgen suggest five steps to bracket prior knowledge, in what he calls ‘phenomenological reduction’. The steps include:
The ‘epistemology of intentionality underlying phenomenography’ is stressed, to achieve ‘reliability’. I don’t know what this means though. HELP!
2. Hasselgren B, Beach, D (1997) Phenomenography – a “good for nothing brother” of phenomenology? Outline of an analysis
This provides a critique of Sandberg (1997). Ontology and epistemology to this point had only been raised as late as 1993 (citting Uljens). Previously phenomenography is located as a pragmatic and empirical tradition. Problems of validity may occur if the analysis reflects the researchers preconceptions, or a result of the situation of the interview or a ‘particular historical moment’. Thus the need for reflexivity. But the phenomenographic experience of findings are purveyed as a truth (citing Sandberg 1995 – same title as 1997, and Bowden 1995), regardless of a ‘non-dualistic assumption’, which the authors suggest is beyond any ‘genuine evidence’ and is ‘ideologically founded’.
Other points worth noting in this article
The authors go on to distinguish five type of phenomenography – experimental (Marton 1975)– discursive (beyond learning as a topic, Dahlgren 1979 “Why does a bun cost two shillings?”) – naturalistic, where recording without asking questions is the process (Lybeck 1981) – hermeneutic – ‘interpreting statements not originally made for the purpose of phenomenography’ (Lynblad 1995) – phenomenological Theman 1983, looking at political power, and Neuman 1987 who wants to know ‘what is actually going on in the subject’s mind during the interview’.
3. Giorgi, A 2002 The question of validity in qualitative research TO DO
4. Akerlind GS 2002 Principles and practice in phenomenography TO DO
5. Bruce C 1990 Frameworks guiding the analysis: applied to or derived from the data TO DO
6. Cope C >2002 undated Using the analytical framework of a structure of awareness to establish validity and reliability in phenomenographic research TO DO
7.Akerlind GS >2001 undated Academics’ awareness of their own growth and development – five dimensions of variation TO DO
Sandberg (2000)
see above for the start – the interview questions…
Jorgen identifies three criteria to ‘justify’ his interpretations:
- communicative validity – discussing the process with the interviewees, including debating alternative explanations
- pragmatic validity – ‘testing the knowledge produced in action’ @14, ie discussing results with interviewee’s to confirm they appear sensible, observing to ensure appear consistent with the researchers’ interpretation
- interpretive awareness (reliability) by bracketing presuppositions ‘about the research object’
These processes of justification were operationalised, both during the collecting of the data, and later in the analysis stage.
Other processes included replication of results by an independent coder, and later reproduction by another researcher of the categories found in the analysis.
Barley(1996)
- team = sociologist, anthropologist, and five doctoral students
- ‘each researcher took responsibility for one or more technicians occupations’
- copious field notes
- focused interviews (many recorded and transcribed)
- each researcher (inc sociologist/anthropologist??) between two and four days of observations weekly
- team meetings once a week
- each meeting detailed descriptive
Other studies to include: ???? PLEASE ADD IF YOU KNOW OR SUGGEST
Possible interview questions/ prompts:
‘Coursework’ (the phenomenon under investigation) – presented in random order
- What does CW mean to you?
- How did you go about it? (Load distribution)
- Was it beneficial
- Did you feel comfortable doing CW?
- How should the CW be? Should there be any changes?
- What did you achieve or were trying to achieve? Or
- What were your objectives/ expectations?
- Do you know about CW elsewhere (at the doctoral level)?
Sandeep, The interview questions I think will be more brief, like (05.04.05):
What is your understanding of X?
What does X mean for you? (Sandberg 2000)
What is X for you? (ibid)
where X is coursework, or sustainability, or advertising on mobile phones or job satisfaction (other examples of studies in our class)
Then expanded upon with questions such as :
“what do you mean by that? Can you explain that further? Can you give an example?” ibid.p.13