2.1. The targeted non-linguistic variables

Given that language use is an important mediational tool (Vygotsky, Reference Vygotsky1986), and that language learning is a social practice (Lantolf et al., Reference Lantolf, Thorne, Poehner, van Patten and Williams2015) and a result of ‘a relational engagement among participants of a community’ (Miller and Kubota, Reference Miller, Kubota, Herschensohn and Young-Scholten2013, p. 234); it is essential to examine how learners interact with people and artefacts from their new environment in order to understand language learning and development. In the pilot study, we set out to gather qualitative data on the following non-linguistic variables: (a) learners’ access to and participation in culturally organised activities and communicative events, (b) learners’ attitudes towards their new language and environment, and (c) learners’ relationships with their country of origin and their first language (L1).

Indeed, we intended to gather information on learners’ opportunities for interaction in the target language as well as their exposure to input in French. Furthermore, by studying those aspects, we hoped to gain insights into learners’ involvement in the host community and learn about the extent to which they have managed to develop ‘insider identities’ and legitimacy (Norton and Gao, Reference Norton and Gao2008; Early and Norton, Reference Early and Norton2012; Norton, Reference Norton2013). Moreover, focusing on how learners feel about their new environment and how they speak about the new culture and the people of the community they interact with is essential to understanding their learning experience, as well as their investment in learning the language and adaptation to the new context (Norton Peirce, Reference Norton Peirce1995; Darvin and Norton, Reference Darvin and Norton2015). SLA research on attitudes has generally shown an interrelation between positive attitudes and the development of L2 proficiency (Gardner, Reference Gardner1985; Oroujlou and Vahedi, Reference Oroujlou and Vahedi2011; Zeinivand et al., Reference Zeinivand, Azizifar and Gowhary2015) and learners’ disposition to learn (Ushioda and Dörnyei, Reference Ushioda and Dörnyei2017). Finally, we consider that information about the way in which individuals relate to their country of origin and L1 for this specific learner profile, given the similar conditions of forced migration that they have all undergone, may contribute to understanding how the participants adapt to their new environment.

2.2. Testing an interview protocol

In any type of research, the choice of data elicitation instruments is dependent on the type of data needed, hence on the research question asked as well as on the theoretical approach adopted (Mackey and Gass, Reference Mackey and Gass2005).

Research interested in understanding language development in relation to sociocultural context draws almost exclusively on qualitative data and narratives collected through qualitative methods, namely interviewing, diary writing or journaling (e.g. Norton, Reference Norton2013; Pavlenko, Reference Pavlenko, Blackledge, Piller and Teutsch-Dwyer2001). In our project, we assume that in order to analyse SLA by Syrian asylum seekers and refugees based on a sociocultural approach, we need to understand how they relate to the community in which they are living and of which they wish to be part. Furthermore, following the work of Norton (Norton, Reference Norton2000; Norton and Toohey, Reference Norton and Toohey2011; Early and Norton, Reference Early and Norton2012; Norton, Reference Norton2013), it is important in examining SLA to understand the world as the learners understand it. For those reasons, we rely on the learners’ representations and understanding of their new environment, as expressed in their own discourse. The data elicitation instrument chosen for those purposes is therefore interviewing.

Interviewing is a widely-used technique in qualitative research, and there are different types of interviews that can be described on a continuum ranging from the fully structured to the completely unstructured (Bell, Reference Bell2005; Mackey and Gass, Reference Mackey and Gass2005). Structured interviews are a rigid type of interview that are comparable to questionnaires, as the researcher sticks to a prepared set of questions, thus allowing to compare the participants’ answers. Semi-structured interviews are based on an interview schedule or guide, but allow for digressions and more freedom of both the interviewer and the respondents. The last type is unstructured interviews, where the interviewer develops and adapts the questions to the respondents. Although no question list is used, the interviewer has to prepare a list of items s/he needs to cover. While unstructured interviews can be more appropriate in preliminary studies, most studies using interviews rely on the semi-structured type, which both allows freedom to the respondents and ensures that the important topics are addressed (Bell, Reference Bell2005).

Overall, given their interactive nature, interviews constitute a flexible tool that allows the researcher to adjust to the respondent and to elicit additional data in case there is need to clarify or to complete initial answers. Nevertheless, this data collection method can also have drawbacks. For instance, the researcher’s subjectivity may interfere with the way the data are collected and interpreted. Moreover, participants may provide information that they believe is wanted or expected by the researcher, through what is termed ‘the halo effect’ (Mackey and Gass, Reference Mackey and Gass2005). Additionally, the questions asked may be inappropriate to certain cultural contexts. The interviewing process and the particularity of the conversation can be challenging for our respondents given that generally, ‘the context of a linguistic encounter contains a tension – that between the uniqueness of a particular interaction and the shared constraints and principles of the society through which communication is mediated’ (Bremer et al., Reference Bremer, Broeder, Roberts, Simonot, Vasseur and Perdue1993).

Finally, in order to elicit valuable data, the interviewer needs to develop good interviewing skills in order to face the different methodological challenges involved in conducting interviews: to know how to play with two very different language registers: the language of research that is used to codify reality and communicate the results to the scientific community, and the language used by respondents; to react in case of silence; and to guide the interview while letting the interviewee speak (Romelaer, Reference Romelaer, Roussel and Wacheux2005; Creswell, Reference Creswell2014).

The literature on research methods and data collection techniques offers a number of suggestions to help the researchers maximise their chances of gathering relevant rich data (e.g., Bell, Reference Bell2005; Mackey and Gass, Reference Mackey and Gass2005). These include suggestions to build a positive and reassuring relationship between the interviewer and the interviewee, to prepare efficient questions that would elicit valuable data and to develop skills in anticipating and solving communication problems. However, as stated by Chaudron (Reference Chaudron2003, p. 773), ‘[s]urprisingly, in most of this literature, there is little description of exactly what sort of protocols, guidelines, questions, or procedures are employed’. Aware of the potential pitfalls involved in conducting interviews, as well as the added challenge of conducting them with Syrian asylum seekers and refugees who have predominantly low L2 proficiency skills, we planned to pilot test out data collection protocol as the first stage of the SOFRA project.

4.2. General description of the dataset

Table 2 describes the dataset in terms of duration of the recordings and corpus size (in the form of the number of turns and utterances).

Table 2.

The data set

Overall, the interviews lasted less than an hour, and contained an average of 266 turns and 572 utterances.

4.3. Data collection procedure

Semi-structured interviews were conducted by the researcher (female, bilingual French/ Arabic) with each participant in person and were video-recorded. An interview schedule of 35 questions was used (see Appendix 1). The semi-structured type of interview was chosen to allow the respondent to talk about what was of central significance to them (Bell, Reference Bell2005) and also to facilitate the interviewer moving easily from one topic to another, paying attention to the flow of the conversation and to the relevance of the respondents’ answers.

We have chosen French as the language of the interviews in order to be able to conduct different types of analyses using software that is not possible to use for Arabic languages (e.g., lexicometric software), and also to analyse the learners’ conversational abilities in the L2. Possibilities of translating the questions in Arabic or English were offered. Furthermore, all the documents presented to the participants, namely the participants’ agreement and the presentation of the project’s objectives, were trilingual (in French, Arabic and English). However, opportunities to switch to the L1 or to English were minimally taken and all the participants chose to hear and answer all questions in French. Some of the participants even reported that the interview was an opportunity for them to speak French and that similar opportunities were rare, especially at the time of the social movement, where access to the university was impossible.

4.4. Interview schedule

The interview schedule contained 35 questions (Appendix 1). Among other things, these questions elicited information about (a) learners’ access to French language and culture and interaction opportunities in the target language; (b) their attitudes towards the new language, their learning experience and new environment as well as their (c) relationship to their culture. We focus, in this article, on only the 19 questions which targeted those three topics (see Table 3). The category (b) was divided into two subcategories: attitudes towards the new culture and environment and attitudes/comments about learning experience.

Table 3.

Questions used in the data subset

Furthermore, in order to easily refer to the listed questions throughout the analyses, each one of them was summarized in the column Targeted information. Finally, a description of each question type is provided in the last column, in terms of complexity (simple or double) and in terms of type of answer required (open-ended or closed-ended). Simple questions focus on only one aspect of the topic addressed. Double questions, however, target two different pieces of information. They are written using two question marks in the interview schedule (Table 3). Four categories of prepared questions were distinguished Open-ended (n=5); Closed-ended (n=2); Open-ended/Double (n=6) and Closed-ended/Double (n=6). These question categories will be used in the analysis of the interviews.

4.5. Data transcription, coding and analysis

The interviews were entirely transcribed in CHAT format using the software CLAN (MacWhinney, Reference MacWhinney2000). The transcriptions were overall orthographic, except for words and chunks that did not correspond to what would be expected in the context of the utterance. These were transcribed phonetically and coded using CHAT conventions. In the examples provided in this article, @u is affixed to such instances.

Respondents’ turns were divided into utterances to have comparable data units across interviews. An utterance is the smallest meaningful speech unit. Most learner utterances are organized around a finite verb, although some are structured with a non-finite verb form/base form or even without a verb (Klein and Perdue, Reference Klein and Perdue1992).

The dataset based on the 19 questions listed in Table 3 was divided into subparts. For each interview, 19 data files were created, each of which contained the turns used to respond to one of the 19 questions. The dataset analysed in the current article thus corresponds to a total of 171 CHAT files.

As a reminder, our objective in interview piloting was to test the interview protocol to ensure that interviews conducted in the main study will allow us to gather coherent, complete, valuable and comparable data across all participants. In order to achieve that, each of the planned questions should be easily understood by each participant and should spontaneously elicit on-topic answers. Furthermore, risks of bias and data variation are two major concerns and have to be avoided. Therefore, we examined how the interviewer actually asked each question and how the interviewee responded. The aim is to assess if the question was clear enough and adapted to the participant’s profile and to examine, in case of difficulties, the interviewer’s reactions and attempts to clarify the question and obtain the most relevant answer possible. Indeed, we elaborated a bottom-up coding scheme to analyse conversations in the different files in terms of the apparent clarity of the questions from the respondents’ perspectives and the relevance of answers from the point of view of the interviewer (whether they were on-topic or off-topic). Parts of conversations that did not directly relate to the pre-set list of questions were coded as digressions. Table 4 presents the information coded in the interviewer’s and the respondents’ speech.

Table 4.

Coding of interviews

* The interviewer’s digressions will not be addressed in the analysis.

6.1. Interview schedule and question wording

The most outstanding result is that many aspects of the interview schedule and question wording needed to be entirely reviewed for the main study, given all the observed difficulties to access meaning and respond to the questions asked.

First, interviewees’ responses and digressions have shown that many different questions in the interview schedule elicited similar content. For example, the three questions on ‘Attitudes about L2 culture’, ‘Attitudes about France’ and ‘Sensitivity to L1/L2 cultural differences’ frequently yielded comparisons of L1 and L2 environments by many respondents. Being actively engaged in adapting to a new environment, acquiring a L2 culture and restructuring their conceptual system (Lantolf, Reference Lantolf and Hinkel1999), it seems natural that attitudes towards that new environment may be expressed through comparisons with the well-known environment. It is obvious that the three questions need to be rephrased, simplified and refocused to avoid redundancy.

In addition, many questions were complicated and unclear because of the way in which they were formulated (e.g. the question on ‘Attitudes towards the French culture’) or because they are in reality multiple questions (what was referred to as double questions in the analyses). The problem with double questions is that in some of the interviews, the interviewer chose to ask each part of the double question separately. In others, she chose to ask both parts from the outset. This lack of systematicity increased the data heterogeneity. Additionally, the interviewer had at times to remind the respondent of the part of the question that was left unanswered. Indeed, most question repetitions occurred with double questions partly because the questions were so complex that the respondents answered them only partially. According to Bell (Reference Bell2005, p. 141), double questions should be avoided in questionnaires as they can make interpretation difficult. Even though our interviews are semi-structured and allow a certain flexibility in question asking, we intend, in the main study, to avoid this question type with the targeted L2 learners in order to reduce question complexity. Furthermore, closed-ended questions were also problematic as going beyond a yes/no answer was highly dependent on the respondent. If the answer is judged as unsatisfactory by the interviewer, she has to ask further questions, and those questions also affect the quality of the gathered data (Romelaer, Reference Romelaer, Roussel and Wacheux2005) and are likely to make data interpretation difficult. Moreover, the lack of clarity of the questions was also due to the use of complex and unfamiliar words (e.g., the word anecdotes in the question related to ‘Anecdotes about L2 use’), or of ambiguous vocabulary (e.g., travailler in the question on ‘Investment in L2’).

6.2 Interview conducting and problem-solving strategies

Our findings presented in section 5 confirm that the interviewer has to cope throughout the interviews with different types of problems. In the pilot study, these included problems related to the participants’ L2 proficiency level and the capacity to anticipate lack of understanding. Without pretesting the different questions, it is quite difficult for the researcher to anticipate some of the potential comprehension problems.

To cope with the communication and comprehension problems, the interviewer used repetitions, reformulations and translations. Translations, which were minimally used as the participants rarely asked for them, were not always efficient in solving understanding problems. Reformulations were the most frequently used strategy to cope with perceived understanding difficulties. While helping to clarify the questions, they added to the heterogeneity of the data collected. In fact, many interviewer reformulations resulted in changes in the questions’ orientation and scope, which consequently influenced the responses. Furthermore, while reformulating and repeating answers ensure the flow of the conversation, some of the interviewer’s turns suggested possible answers to the respondents. These instances of bias show what the interviewer consciously or unconsciously would like to see in the data based on her research questions and objectives (Romelaer, Reference Romelaer, Roussel and Wacheux2005). What is vital in qualitative research, however, is that the researcher manages to keep ‘a focus on learning the meaning that the participants hold about the problem or issue, not the meaning that the researchers bring to the research or that writers express in the literature’ (Creswell, Reference Creswell2014, p. 234). Interviewer’s subjectivity is evidently a problem and it can be hard to overcome as Bell (Reference Bell2005, p. 168) suggests :

Moreover, an interviewer’s reaction to respondents’ speech with expressions like oui and d’accord may also affect the neutrality that is expected in an interview. According to Romelaer (Reference Romelaer, Roussel and Wacheux2005, p. 116), ‘les ‘oui…’ ne doivent pas pouvoir être interprétés comme signifiant l’approbation donnée par l’interviewer aux propos du répondant, ou son étonnement sur les informations qu’il recueille.’ These types of reactions, even though they are meant to encourage the participants, may also influence the respondents and produce the halo effect; i.e., ‘trying to please the researcher by giving the answers or responses they think are expected’ (Mackey and Gass, Reference Mackey and Gass2005, p. 117).