Download Chapter 3: Pilot Studies and more Lecture notes Research Methodology in PDF only on Docsity!
Chapter 3: Pilot Studies
The scope of this research encompasses three different communities and diverse concepts of authenticity. The research began by undertaking several pilot studies; as Yin ( 1989 ) states, such pilots can provide considerable insights into the subject matter, aid in the development of appropriate methods, allow exploration of relevant questions, permit formulation of a clear conceptual framework to analyse the data, and assess the logistical requirements for the eventual fieldwork. The opportunities arose to undertake several pilot studies in Asia during the initial phase (2010 – 2011) of this research. During these pilot studies, a ‘functional naiveté’ (Bordelon, et al., 2011) personality was adopted, i.e., it was conducted in dispositional state of ‘not knowing’, holding no assumptions or any preconceived notion of the subject (Weber, 1949; Lincoln and Guba, 1985). Such a dispositional state enabled free exploration without preconceived ideas on which specific research methods to employ, while at the same time allowing the development of an understanding of how artisans might envisage concepts around authenticity. The table below (Table 2) summarises the details of the different field missions undertaken during the pilot study period including assessment of the various research methods. S/No Field Mission, Period and Informants Objectives Methods of Investigation and Processes
- Hoi An, Vietnam 5 – 10 Oct 2010 Hoi An Artisans - Profile of artisans - Pre & Post training survey - Identification of markers of authenticity - Written questionnaires - Translation and interpretation
- Chengdu, Sichuan China 20 – 24 Feb 2011 Tibetan Artisans - Identification of markers of authenticity - Semi-structured interviews with focus group - Translation and interpretation - Video recording - Usage of exemplars
S/No Field Mission, Period and Informants Objectives Methods of Investigation and Processes Leishan, Guizhou, China 26 Feb – 3 Mar 2011 Miao Chinese Ethnic Minority
- Identification of markers of authenticity - Semi-structured interviews with focus group - Translation and interpretation - Video recording - Usage of exemplars - Political, social and cultural context Longchuan, Yunnan China 4 - 8 Mar 2011 Archang and Budongga Chinese Ethnic Minority
- Identification of markers of authenticity
- Semi-structured interviews with focus group
- Translation and interpretation
- Video recording
- Usage of exemplars
- Bangkok, Thailand 21 – 22 Mar 2011 Khun Tinnart, Jim Thompson
- Identification of markers of authenticity
- Case study
- Open unstructured interview
- Video recording
- Usage of exemplars
- Unobtrusive Observations
- Vientiane, Lao 23 – 25 Mar 2011 Ms Kongthong and Ms Viengkham
- Identification of markers of authenticity
- Case study
- Open unstructured interview
- Video recording
- Usage of exemplars
- Unobtrusive Observations
- Luang Prabang Laos 27 Nov – 5 Dec 201 1 Lao-Tai Communities and Mdm Vandara Amphayphone
- Baseline survey
- Pilot testing of questionnaires and structured interviews
- Structured interviews
- Translation and interpretation
- Unobtrusive Observations
- Political, social and cultural context Table 2 : List of Pilot Studies 3.1 Lessons Indicated from Pilot Studies 3.1.1 Field Research The most appropriate method and the most frequently used instrument for qualitative research is the ‘human researcher’ (Lincoln and Guba, 1985). The field research methodology enables this research to take into account the interviewees’ physical setting and to study the everyday practices; these could be taken into account when seeking to understand and contextualise the interviewees’ thoughts (Reason, 1981).
method (Mann, 1985; Oppenheim, 1992). Both open and closed questions (Oppenheim,
- were used to determine the suitability of the two approaches. The Hoi-An mission study found that the responses from open-ended questions were vague, imprecise or even worse, contradictory. Possible explanations were that the trainees did not understand the questions or perhaps, they were unable to express themselves in writing to convey their thoughts. This might be due to their low level of formal education, an explanation offered by the local Vietnamese UNESCO Project Officer. Also, written questionnaires did not facilitate opportunities for artisans to clarify the meaning of questions, through asking the author directly or discussing the questions with each other during the exercise. (This is particularly significant in Asian societies where people fear ‘losing face’ if they ask questions as it might suggest ignorance on their part.) This may have resulted in frequent misunderstanding of the questions. The scope to elicit information from artisans using questionnaires containing only closed questions was limited. The pilot study showed it was too cumbersome to examine complex issues such as variables, linkages, relationships, etc. For example, examining traditional production methods while comparing with other aspects such as links to contemporary design, would have rendered the formulation of questions excessively complicated. However, closed questions with multiple answers could be used for establishing the general profile of the trainees such as age, gender, religion, ethnicity, involvement in different types of crafts and number of years of practice. This was adopted during the interviews to obtain generic, clear and concise information from interviewees as in Phase Two where quantitative methods were used to obtain definite responses. 3.1.4 Structured Interviews Using methods suggested by Oppenheim (1992), the pilot studies in Laos (S/No. 5, Table 2 ) showed that structured interviews were not particularly effective, possibly because answers were already pre-conceived by the interviewer and interviewees were not free to reply as they wished; rather they had to select a response, but which was limited to one of those offered by the interviewer.
3.1.5 Semi-Structured Interviews From the pilot studies, and recalling that authenticity is experiential in nature, as discussed in Chapter Two, the concept and articulation of authenticity by crafts persons was found not to be always explicit. Rather, findings from the pilot studies showed that such knowledge and understanding of authenticity is mostly tacit. Most of the time, responses were descriptive or narrative in nature, alluding only marginally to concepts of authenticity discussed in this work. A principle employed in this study was to avoid drawing attention to the specific topic of authenticity. Indeed, the approach was to trigger elements in the unconscious minds of the weaver, thereby allowing them to articulate associations with their textiles. Borrowing the methodology from McCracken (1988) and Bowen (2005) interviewees were encouraged to narrate their stories in their own words about their textiles through a semi-structured questionnaire that guided and prompted them to reflect on their work. Semi-structured interviews (Burns, 2000; Wengraf, 2001; Flick, 2002) were found to be the most effective method to elicit information from artisans. Interviewees were guided towards issues which had been decided on beforehand, while they were free to explore relevant and associated ideas, enriching the collected information. The flexibility of this method allowed the possibility of new discoveries that had not been previously considered. Also, it enabled both the interviewees and the interviewer to review and clarify responses during the course of the interview. This method seemed to be very ‘organic’, a collaborative process between interviewer and interviewees that helped to deepen the perspectives of all involved. To help weavers in the discussions, various prompts were used (Oppenheim, 1992) such as citing the examples from the interviewer’s own experience. 3.1.6 Free-Style/Exploratory/Open-ended Interviews The pilot study with Khun Tinnart of the Jim Thompson retail chain (S/No. 3 , Table 2 ) showed that free-style/exploratory/open-ended interview approaches (Oppenheim, 1992 ; Burns, 2000; Flick, 2002) were extremely useful with good potential, especially when eliciting the opinions of interviewees by allowing them to express themselves openly. However, the success of this method depended to a large extent on the personality of the interviewee as well as the dynamics between the interviewee and the interviewer.
3.1.9 Usage of Exemplars Reflecting on the pilot studies (S/No. 2, 3 and 4, Table 2 ), the use of physical objects was indispensible for both the interviewer and informants to express, convey and explain ideas and concepts. Without these objects, it would have been difficult for both parties to understand each other clearly; especially when terms and concepts were unclear and when the interpreters were not technically competent to translate these terms. 3.1.10 Interpretation The pilot studies (and also in the Mosuo and Bhutan case studies) were conducted where English is not the native language of the community. As the focus of the study was to privilege the voice of weavers, it was vital to understand how the concept of authenticity was formulated, used and understood vernacularly. This objective meant it made more sense to enlist the help of non-professional interpreters rather than engaging the services of professionals. As evidenced during the pilot studies, the interactions with non-professional interpreters enabled an understanding of the local and everyday usage of language, particularly in the area of interest to this study. From the experiences in the pilot studies, it was apparent that some preparation was necessary when using non-professional interpreters. For example, interpreters must be briefed prior to the interview sessions to understand key terms and concepts, especially technical terms. Hence, a list of concepts, technical terms and words was drafted and discussed with the interpreter before the start of each field mission. Preparing the interpreter also meant briefing him/her on the protocols and procedures of the interviews. For example, if clarification was needed from either the interviewee or the interviewer, the interpreter should refrain from taking his/her own initiative to explain matters but to address the interviewer’s questions to the persons concerned. 3.1.11 Political, Social and Cultural Context Lessons from these pilot studies showed that the social, gender and cultural norms of the community were woven into the fabric of the interview sessions. It was soon realized through conversations with the interpreter that there existed a status ‘gulf’ between the interviewer and the interviewees during the Lao-Tai fieldwork
(S/No. 5, Table 2 ). Answers given by the informants were often ‘polite’ and ‘expected’, not necessarily reflecting the actual thoughts of the artisans themselves. For example, weavers seldom offered any responses but would readily agree to answers presented by the interviewer. Also, they would not put forward any negative comments (at least comments which could be perceived to be negative). This may be attributed to traditional rural Asian artisans often viewing interviewers as ‘officials’ and treating them as superiors (especially in structurally hierarchical societies such China and Laos). Therefore, it became clear that interviewees would give ‘expected’ answers, hoping that their replies would correspond with what the officials might like to hear. In order to encourage interviewees to respond honestly, it was necessary to establish a rapport so as to make informants feel comfortable and at ease (Oppenheim, 1992). These would include leveling the status between the interviewer and the interviewees and gaining the trust of informants through a less formal setting. These tactics engendered confidence for the weavers, creating a safe environment for them to express themselves freely. The dynamics of artisans in a focus group (Flick, 2002) (S/No. 2, Table 2 ) was important as it affected the credibility of the group’s response. For example, during an interview, one person might dominate the discussion while others shied away. Therefore, facilitation of the discussion was very important to encourage and provide opportunities for every member of the group to contribute to the discussion. It was extremely important to recognise the political undercurrents of the members of the focus group. For example, when interviewing the Miao weavers (S/No. 2, Table 2 ), a non-professional interpreter was selected because of her language skills in Mandarin and the local Miao language. During the course of the interview, it was found that she was actually answering on behalf of the Miao batik artisans because she was a government official and perceived herself as the representative of the group rather than as an interpreter. The significance of gender roles in traditional rural artisan communities needs also to be recognised with due consideration in on how this might affect the accuracy of the research.
Another means of establishing trustworthiness and credibility is through triangulation of different research methodologies (Jick, 1979). According to Rubin and Babbie (2008), this technique required in obtaining and utilising information from a variety of sources such as from observations and information conversations with other people. 3.1.14 Research Ethics Of particular importance in undertaking research ethically, is the understanding of ‘informed’ consent (Miller and Bell, 2012 ). At the fundamental level, the researcher must be clear that participants have understood the nature and purpose of the research. The interviewees’ motivation about being part of the study must be made clear to both parties. Finally, a written consent is necessary in order to document the agreement (Oppenheim, 1992). 3.1.15 Preparation According to Yin (1989), it is important to draft a case study protocol that includes checklists, narratives and a sequence of actions for the field research. These procedures ensure an orderly flow of activities and help to prepare the necessary logistics (Yin, 1989). Missions to the field can be overwhelming, especially for those conducting field work alone as unexpected events and accidents may inadvertently happen. The author’s long experience of undertaking projects in rural communities such as Bhutan and Tibet has demonstrated the value of such preparation. 3.1. 16 Sampling The pilot studies highlighted that in spite of the formations of organizations to address their needs, the weaving industries in these places remain very informal. Therefore, random snow-balling and judgmental sampling methods (Oppenheim, 1992; Crabtree and Miller, 1999) were used extensively in the follow-on research to identify weavers who would participate in this project. However, the first contact with an interviewee were identified and initiated through local organisations. Before undertaking the case studies, a research-planning audit was conducted according to Yin’s recommendations (1989). This identified minimum numbers of interviewees from each category to be interviewed (Table 3 ).
Category of Weavers Quantity Beginners 3 + 1 Intermediate-Skilled Weavers 3 + 1 Highly Skilled or Masters Weavers 3 + 1 Total 12 Table 3: Minimum Weavers Required for Interviews in Each Case Study The justification for three weavers from each category was based on the principle of triangulation (Jick, 1978). An additional person was also considered useful as a fallback. The pilot studies also demonstrated that the quality of the responses from beginners was poor, as they had limited knowledge. Hence, in most cases, interviews with weavers from this category were restricted to one or two persons. The principle of ‘saturation point’ (Bordelon, et al., 2011) or ‘law of diminishing returns’ (Fetterman, 1989) was adopted as the method of determining the maximum number of weavers. ‘Saturation point’ was identified when the responses of interviewees provided no new information (Bogdan and Biklen, 2001). 3.1.17 Transcribing Experience from the pilot studies showed that the transcribing process became more manageable once responses in the questionnaires were classified into themes. This helped address the challenge of transcribing free-style/exploratory/open-ended interviews which was found to be time consuming (Oppenheim, 1992). 3.2 Discussion and Summary A considerable number of lessons were learnt from the pilot studies which informed the development of the main body of research. 3.2.1 Identification of Markers of Authenticity through Deconstruction Reflecting on the paradigms of authenticity (Figure 33: Diagrammatic Representation of Paradigms of Authenticity), the challenge of the research method is to capture all aspects of authenticity including the material, non-material, abstract and concrete. Examining the results of the pilot studies, it is seen that artisans analyse their crafts by deconstructing their products down into various components – the form (shape, size, weight, material, colour, design, etc.), the way it is made, the tools and equipment used,
3.2.2 Methods Determined as Not Suitable for Study The pilot study experience showed that structured interviews were not suitable for investigating weavers’ narrative concerning markers of authenticity. As the focus of this research is to privilege weavers’ view of authenticity, structured interviews were not found to be suitable because interviewees were unable to freely express their thoughts and opinions, without these being prejudged by anticipated answers. The Free Style/Exploration/Open-ended Interviews methodology assumed that both interviewer and the interviewee were well acquainted or the disposition of the interviewee to be open and ‘chatty’. Such a technique would be most successful if both parties used a common language to avoid loss of information in translation. Therefore, as a precaution, and as two of the three case studies (Mosuo and Bhutan) could not be conducted in English, this methodology was not selected as a viable means of research. Finally, Individual Face-to-Face interviews were preferred over Focus Groups to minimize the effects of group dynamics and power differences. 3.2.3 Flexibility in Approach When conducting field research in places where the communications infrastructure is challenging, and where local political, social and cultural factors play important roles when gathering information, it is recognised that research methods must be flexible, the researcher must be creative and at times, work is carried out in a less ordered fashion (Cole, 2005). Furthermore, the topic – markers of authenticity - presented yet another challenge to the investigation as the knowledge is mostly tacit, well imbedded within the minds of weavers and may not be easily vocalised or expressed. Significantly, this knowledge needed to be teased out sensitively, which required a creative approach. Under such circumstances, it is important for investigators to be aware of the local situation, and to seize any opportunity or ‘fieldwork zeros’ (Kutche, 1998) to gather and validate research data. Again, as the situation in the field is often fluid, it is important to change or modify strategies or employ new means of investigation when a previously successful method is no longer yielding results. Finally, the successful deployment of one method in a community need not imply success with another as the political, social
and cultural context, and the capacity of informants will be different. Hence, each study needed to be individually tailored to address its particular and unique circumstances. 3.2. 4 Final Methodology Adopted One important lesson learned from the pilot studies and from the literature (Strauss and Corbin, 1998; Crotty, 1998; Oppenheim, 2005) is that no one method is sufficient to provide the range of answers to the research questions. Therefore, a selection of methodologies (case studies, semi-structured questionnaires, individual face-to-face in- depth interviews, unobtrusive observations, examination of samples, informal conservations, etc.) have been used to address the research question. The foundation of the approach to this research is based on a qualitative method through its ability to yield in-depth descriptive data, thereby developing an understanding of authenticity (Bowen, 2005). However, quantitative data has also been used, complementing the qualitative aspects of this study by legitimising and confirming the narratives of interviewees.
