2. Learner autonomy

One of the most popular lines of inquiry in doctoral research within the last few years has been learner autonomy, so this section includes four Ph.D. theses addressing this topic from very different angles. While some of these seek to determine the impact of overall training programs (Ekiert Reference Ekiert2006), others focus upon the contributions of specific instructional resources and procedures as well as the influence of autonomy on attainment in learning a foreign language (Czura Reference Czura2010; Kruk Reference Kruk2010), and others explore the relationship between certain aspects of learner autonomy and the development of specific language skills (Trepczyńska Reference Trepczyńska2009).

Ekiert (Reference Ekiert2006) investigates the role and feasibility of learner training in the context of a private Polish lower secondary school. The study proper, preceded by a pilot study, took the form of an action research project which lasted about two years and involved 36 learners, aged 13 to 15. The study involved three pre-existing learner groups and was conducted in two stages: (1) a comparison of the learning outcomes of an experimental and a control group, and (2) a comparison of the outcomes of a single study group after one year of learning English without learner training, and then after another year with the benefit of such training. Ekiert focused on the learners’ development, first examining their beliefs about learner training as well as their ability to cope with language learning, and then implementing a training program, partly modeled on Scharle & Szabo (Reference Scharle and Szabo2000), with two groups of students. The study aimed to: (1) establish the extent to which training in learning strategies can aid the development of language competence and metacognition, and (2) determine how teachers can contribute to the learning process by creating conditions conducive to the use of effective language learning strategies and autonomous behaviors. The data, which were obtained by means of interviews, self-reports, questionnaires, observations, teacher-, peer- and self-evaluations, personal accounts and teacher field notes, as well as achievement and proficiency tests, were subjected to both quantitative and qualitative analysis and interpretation.

Although the results suggested some improvement, there was no significant difference between the groups that could serve as strong evidence for or against learner training. The final grades in the experimental group, however, suggest that learner training did prove to be effective to some extent. Such an interpretation is plausible in view of the fact that the students’ grades improved in comparison with the previous year, while those of the control groups dropped considerably in comparison with the first semester. The analysis of the qualitative data showed the value of introducing learner training and demonstrated that students are interested in such training, understand its value and, in most cases, willingly participate in strategies-based instruction. On the basis of these findings, Ekiert concludes that although lower secondary school students generally demonstrate a positive attitude towards learner training, a lot of time and effort is needed to design activities that would not only contribute to successful language learning but also be of interest to learners in this age group. She also emphasizes the fact that learner training will only bring positive results if it is implemented across the curriculum, irrespective of the subjects being taught. Ekiert points to the most common obstacles encountered and offers some valuable guidelines on how training can be conducted for the benefit of learners.

The dissertation can certainly serve as a rich source of ideas for teachers wishing to introduce learner training into their classrooms. Despite its numerous merits, however, the study is not without its limitations, especially in that the school where the research was carried out offered much better educational opportunities (e.g. smaller groups, more contact hours and modern teaching equipment) than is typically the case in Polish lower secondary schools. Consequently, the outcomes achieved may not provide a good point of reference for the implementation of learner training in state schools.

Following a line of inquiry that Benson (Reference Benson2001) identified as urgently needed in research on learner autonomy, Kruk (Reference Kruk2010) examines the impact of Internet-based resources on the growth of learner independence among upper secondary school students and investigates the effect of such independence on their achievements in learning English, drawing on the participants’ perceptions of the intervention and observing its impact on individual learners. The study was quasi-experimental in nature, and involved 46 learners in two pre-existing classes from two types of upper secondary school (i.e. specialized and vocational), designated respectively as the experimental and control groups. The intervention took place during 23 regularly-scheduled lessons and extended over a period of 15 weeks, focusing on vocabulary, grammar, pronunciation and the skills of reading, listening, speaking and writing. Although the focus was the same for both groups, the control subjects worked with their regular coursebook, while the experimental students focused on the same areas but had the benefit of using websites dedicated to learning English and Internet-based software as well as tasks created by the teacher. The data on learner autonomy were collected through a questionnaire consisting of 30 five-point Likert-scale statements and open-ended questions administered before the treatment, immediately afterwards, and seven weeks later. Additional data were obtained through individual and group interviews, participant observation, learners’ logs, an Internet English forum and an evaluation sheet. Attainment was measured on a pretest, and immediate and delayed posttests, which focused on the subsystems and skills mentioned above. Statistical analysis revealed that the treatment resulted in increased levels of autonomy among the experimental students, both at the group and at the individual level, although not in all areas. This finding was largely corroborated by the data collected through the remaining instruments, which also showed that the students’ perceptions of the Internet-based resources and tasks were positive. The growth of learners’ independence was accompanied by progress in the experimental learners’ command of English, in particular with respect to grammar, vocabulary, speaking and writing immediately after the intervention.

The dissertation is valuable as it contributes significantly to the still sparse body of empirical evidence on the link between the development of learner autonomy and attainment in learning a foreign language. The research project was meticulously designed, longitudinal in nature, and conducted in real classrooms. It relied upon methodological triangulation as well as a combination of quantitative and qualitative data, and the findings serve as a basis for feasible pedagogic proposals and sound recommendations for further research. As with many other studies conducted in the Polish context, however, no reliability or validity information for the data collection tools is included, although the tests were thoroughly piloted. Another cause for concern is the compatibility of the students in the two groups, given the fact that they came from different schools, which may have affected the results of the intervention.

Czura (Reference Czura2010) seeks to explore the impact of different types of self-assessment on the development of autonomy among lower secondary school students, as well as the relationship between the changes in their level of independence and their achievement in English. The study, which successfully combines the characteristics of cross-sectional and longitudinal research, spanned one school year (ten months) and involved 116 learners attending first and second grade, divided into five groups, four of which were experimental and one control. While the controls were evaluated in accordance with the standard school procedures, the teachers of the experimental subjects employed different methods of self-assessment, namely portfolio assessment, project work, peer-assessment and observation. Quantitative data on learner autonomy were collected by means of a questionnaire designed specifically for the purpose of the study and administered to all the groups before and after the intervention. The questionnaire used 35 five-point Likert-scale items, reflecting the following seven aspects of autonomy: selection and implementation of resources, collaboration with others, the ability to establish learning aims and objectives, engagement in learning outside the classroom, the ability to evaluate the process of learning, the implementation of appropriate learning strategies, and attitudes toward teachers and their role in education. Qualitative data on the development of learner autonomy were obtained through regular observations and interviews conducted towards the end of the study with three or four learners from each group. Finally, information about the subjects’ attainment was obtained by means of a paper-and-pencil test administered at the end of the intervention, which enabled assessment of subjects’ skills in reading and writing and their knowledge of grammar and vocabulary. The main finding was that none of the methods of self-assessment had any statistically significant effect on the overall level of autonomy and therefore it was not possible to establish the relationship between the growth of autonomy and performance on the test. However, some changes were observed with regard to the impact of portfolio assessment: this positively influenced the subjects’ ability to choose and take advantage of relevant resources but at the same time negatively affected their ability to set their own learning goals. Even though other changes were evident in the case of specific statements in the particular groups, they were transitory and failed to transfer to other tasks. As shown by the outcomes of the observations and interviews, these findings are to a considerable extent a consequence of the regular teaching and assessment procedures applied in Polish lower secondary schools, which are very much teacher-centered and oriented towards accuracy.

Czura's study has undeniable strengths: (1) it tackles the issue of the development of learner autonomy in lower secondary school, an educational context which has been neglected by Polish researchers, (2) it submits claims concerning the value of self-assessment in real classrooms to empirical testing, (3) it was conducted over an extended period of time, which makes it possible to trace changes in the growth of learner independence, and (4) it recognizes the multidimensionality of autonomy by breaking down the concept into its components and employing methodological triangulation. Such characteristics are certainly needed in research on autonomy, but their inclusion in relevant studies is by no means the norm (Benson Reference Benson2007). However, the study also suffers from several weaknesses, the most crucial of which is that no attempt was made to validate the autonomy questionnaire, following, for example, the procedure described by Dörnyei (Reference Dörnyei2001). There are also limitations connected with the measure of attainment, due to the lack of a pretest and the failure to complement the data obtained on the test with some kind of evaluation of communicative ability.

Finally, Trepczyńska (Reference Trepczyńska2009) focuses on the contribution of training in self- and peer-assessment to the development of advanced EFL learners’ writing skills. The study, qualitative in nature, was situated within the context of the process approach to writing instruction and was informed by research on peer-response groups and learning strategy training. The project lasted a year and a half and involved 31 first-year English majors, prospective teachers, out of whom a sub-group of ten students was selected for further in-depth analysis in which the focus was on revision and response strategies used by inexperienced FL writers. The students attended a general writing course taught by the researcher, combined with training in evaluation, which included discussions on the role of peer response, guidelines for techniques of formulating comments, and instructor-led as well as pair- and group-evaluation of students’ essays using supplied forms. At each stage of the composition process, the participants were provided with feedback coming from peers or generated by the writers themselves in response to their own texts. Out of the total of 12 essays produced by the students, three were selected for analysis, together with earlier drafts. Using Faigley & Witte's (Reference Faigley and Witte1981) taxonomy, the number and type of revisions as well as their sources (peer vs. self) were analyzed, together with their impact on the overall quality of the text. The different versions of the essays were graded on a seven-point scale and Pearson product–moment correlations were calculated to determine whether the improvement between drafts was significant. Other instruments included (1) surveys with five-point Likert scale statements providing information about the students’ attitudes towards various types of feedback, (2) semi-structured interviews following revision, (3) self- and peer-evaluation forms modeled on those used in similar studies (e.g. Berg Reference Berg1999) with checklists related to content, organization and language use, and (4) the original and revised drafts.

The analysis of the data, both for the whole group and the individual students, yielded many interesting insights into the development of writing skills. Its major findings were as follows: (1) while self-evaluation was the least popular activity among the students in the initial stages, the overwhelming majority of changes were those generated by the writers themselves, (2) the majority of students were primarily concerned with language use (i.e. surface changes), with content coming second, (3) in terms of types of revisions, those relating to surface structure were the most common; the subjects did not trust their peers’ linguistic abilities and ignored their suggestions, (4) as for meaning changes, the authors were the most open to their peer reviewers, but when peer response was a regular element of the course, more substantial changes were introduced, (5) the students seemed to have benefited more from reading and analyzing their peers’ texts than from the feedback received, and (6) although writing improvement was generally noticeable, no correlation between this improvement and the number of changes was found; thus it is the quality rather than quantity of changes that affects the outcomes of written work. On the basis of these results, Trepczyńska (Reference Trepczyńska2009) concluded that: (1) both peer response and self-evaluation seem to hold some potential, but they should not be expected to produce immediate effects, (2) although students benefit mainly from self-evaluation, they recognize training in peer response as valuable for their future profession as language teachers, and (3) the provision of regular opportunities to participate in self- and peer-evaluation tasks can lead to some degree of improvement in writing skills. She also cautions that, given the rather modest outcomes of the study, the pros and cons need to be carefully considered before an innovation of this kind is introduced into teaching practice.

Summing up, the results of Trepczyńska's study seem rather disappointing, which is a major drawback in itself as questions could be raised about the time and effort spent in return for such a meager payoff. It should also be kept in mind that in contrast to teacher feedback, self-evaluation and peer response do not play major roles in the process of teaching writing; they are considered time-consuming techniques that may contribute to the development of learner autonomy, but apparently do not translate into the improvement of writing skills as such, so their use in terms of teacher training is rather limited.

3. Individual differences

Another vibrant line of inquiry in research on SLA in Poland involves empirical investigations of the role of individual learner differences in the process of learning and teaching foreign languages, and this is evident in the six doctoral dissertations under review in this section. It should be pointed out, however, that the motivation for investigating individual variation is not always the same: some of the authors attempt to determine its impact on the use of language learning strategies (Lesiak-Bielawska Reference Lesiak-Bielawska2007; Dąbrowska Reference Dąbrowska2008; Banasiak-Ryba Reference Banasiak-Ryba2010), while others are interested in establishing the influence of individual learner differences on attainment (Kobyłecka Reference Kobyłecka2006; Klimas Reference Klimas2009; Pospieszyńska-Wojtkowiak Reference Pospieszyńska-Wojtkowiak2010).

Dąbrowska's (Reference Dąbrowska2008) dissertation reports on an ambitious attempt to assess the value of strategy training in teaching English to 42 second-year students of English in a teacher training institution. The study was divided into two stages, during which the following issues were investigated: (1) the relationships between selected individual differences, on the one hand, and between these individual differences and patterns of strategy use, on the other, and (2) changes in the use of strategies, beliefs and opinions that are likely to appear as a result of strategic training. The first stage was diagnostic in nature and served to obtain data on the basis of which an experimental program could be designed, for implementation in the second stage. The instruments used included a detailed individual differences questionnaire, an extroversion/introversion test, a right/left-brain dominance test (Brown Reference Brown1994), a learning styles test (Tanner & Green Reference Tanner and Green1998), the Strategy Inventory for Language Learning (Oxford Reference Oxford1990) and a communication strategies survey (cf. Ellis Reference Ellis1985). Quantitative and qualitative analysis of the data established the cognitive and affective profiles of the participants, and also demonstrated the existence of correlations between some individual factors and the use of different categories of language learning and communication strategies, on the basis of which the experimental and the control groups were formed, containing 22 and 20 students respectively. The second stage, eight months in duration, involved an experimental research project, which took the form of a long-term, fully-informed strategy training program devoted to practicing strategies related to the four language skills and primarily based on the model proposed by Oxford (Reference Oxford1990). The data collection instruments included questionnaires, checklists, verbal reports and semi-structured informal interviews, as well as teacher observation.

The main findings of the study can be summarized as follows: (1) both qualitative and quantitative analysis provided evidence that the majority of the students did not use strategies on a regular basis, (2) the training was effective in contributing to a statistically significant increase in the number of strategies used and the frequency of their use in the experimental group, while there was a decrease in these areas in the control group, (3) although the increase in the frequency of strategy use was evident across the board, it was the highest in the case of social and affective strategies, (4) all the experimental group students were observed to use more strategies, compared to the control group, and (5) there were no significant changes in the subjects’ beliefs, the only exception being modifications in the experimental students’ views about listening comprehension. The most important conclusion that can be drawn from the study is that adult language learners can still develop and modify learning strategies in response to well-planned strategy training, which should therefore be incorporated into language courses and also constitute an integral component of teacher training courses. Such instruction is also of vital importance for future teachers, who are clearly in need of the knowledge and skills necessary to diagnose, assess and solve their own learning problems as well as the capacity for self-reflection, self-monitoring and self-evaluation.

Dąbrowska's dissertation is impressive both in its volume and its scope and each of the two empirical stages is a well designed research study in its own right, contributing to a better understanding of the process of language learning and teaching. The numerous strengths of the study can be summarized as follows: (1) it is characterized by complex research procedures for the purpose of which the author developed her own instruments as well as adopting others used in similar studies, (2) it provides insights into the process of skill development and strategy training which may be of value to both teachers and teacher trainers, and (3) it is probably the most comprehensive, in-depth study of individual variation and strategic research. Paradoxically, the major drawback of the study is its scope, in the sense that it seeks to answer too many research questions, making it difficult for the reader to appreciate the main orientation of the issue being discussed, especially as some of the results are presented in the main text, and others in the appendices.

Drawing upon earlier research findings (e.g. Anderson & Vandergrift Reference Anderson, Vandergrift and Oxford1996), Lesiak-Bielawska (Reference Lesiak-Bielawska2007) set out to explore the relationship between learning style preferences and the use of strategies for reading and listening. It was hypothesized that strategies triggered in response to the constraints of a task reflect the learning style preferences reported by learners; they are task-related and linked to other strategies employed to perform a particular assignment. The study involved a group of 49 intermediate adult learners around the age of 21, studying English as a foreign language; it combined product and process orientations to SLA research, the former represented by the scores on a comprehension task and the latter by the relationships between learning style preferences and strategy use. The instruments employed included a background questionnaire, a motivation questionnaire based on Horwitz's (Reference Horwitz, Wenden and Rubin1987) BALLI and Dörnyei & Ottó (Reference Dörnyei and Ottó1998), and the Learning Style Preference Survey (Cohen, Oxford & Chi Reference Cohen, Oxford and Chi2001), which was translated into Polish and thoroughly piloted. Subsequently, the learners were asked to complete reading comprehension and listening comprehension assignments, which were assessed in terms of correct answers. The tasks were followed by surveys of strategy use in post-task reading (97 strategies) and listening (92 strategies), which employed a five-point Likert scale to determine the use of specific strategies (Cohen, Oxford & Chi Reference Cohen, Oxford and Chi2001). Spearman's rank correlation coefficients were then computed to explore the relationship between the students’ learning style preferences and their reading/listening strategy use on each task. The covariance between these two measures was inspected with respect to the learners’ differential success in task completion.

In the more effective subgroup, an overwhelming majority of strategies seemed to be consistently linked to reported learning style preferences. Moreover, many strategies appeared to be compatibly distributed over two or even three learning style preferences, which testifies to the ability to trigger a wide range of related learning style preferences matched to the strategies employed and the task at hand. In the case of the weaker group, such matching was not evident: there was a lack of inner coherence between learning style preferences and reported strategy use, and there was some evidence of insufficient metacognitive awareness on the part of the subjects. The results also indicated that successful foreign language comprehension rests on the skilful integration of top-down and bottom-up processing and that strategies related to local and global aspects of discourse processing do not operate in a vacuum: their successful activation requires a certain level of linguistic and metalinguistic knowledge. When a learner has such knowledge, the strategies triggered in response to the constraints of language comprehension appear to be congruent with learning style preferences, the task in hand, and other pertinent strategies. In addition, it was found that compatibility and inner correspondence in the style–strategy relationships were related to a higher level of target language proficiency as well as a high level of metalinguistic knowledge.

The results of the study seem to corroborate the findings of earlier research, pointing to the relevance of the concept of learning style to second and foreign language learning. It yielded much interesting data contributing to a better understanding of the relationship between learning style preferences and reading and listening strategy use. Because of its complexity – evident in the number of factors being investigated – the study should be replicated in other contexts and with other groups of learners. It has to be noted, though, that the presentation of the results lacks clarity and focus, as Lesiak-Bielawska constantly refers to numerous similar studies, making comparisons with the results obtained by other researchers. A critical synthesis of the results followed by a discussion would have been a better solution. As it is, the reader has difficulty getting a clear picture of the issues being discussed.

In another dissertation seeking to relate individual difference variables to strategy use, Banasiak-Ryba (Reference Banasiak-Ryba2010) conducted a study whose main aim was to compare the vocabulary learning strategies used by learners from different cultural backgrounds. The strategies were analyzed from two perspectives: first, the researcher identified the strategic devices used by the respondents to complete a concrete vocabulary learning task, and, second, she investigated the overall pattern of vocabulary learning strategies used by each of her subjects. The study proper, which was preceded by a pilot study, involved 11 foreign language students from different linguistic and cultural backgrounds and consisted of two stages. In the first stage, a questionnaire was administered on the ways of vocabulary learning in the students’ home countries and the strategies they used. In the second stage, which lasted for three days, the students were given a task of learning ten vocabulary items and asked to document the process by completing a structured record. The results of their learning were then evaluated by means of a vocabulary test administered during a recorded interview. In the interview, the respondents were also asked to clarify certain issues concerning the task, answer questions about their cultural background and express their views about culture- and education-related categories established by Hofstede (Reference Hofstede2001). The results showed that, on the one hand, there were some vocabulary learning strategies used by the majority of the subjects but, on the other, numerous differences were observed on specific tasks, which could be to some extent ascribed to the subjects’ different cultural backgrounds. Interestingly, no correlation was found between the number of strategies used and test scores, which implies that frequency of strategy use does not guarantee success in learning vocabulary. Based on the findings, it can be concluded that students choose vocabulary learning strategies according to the task at hand and their current needs, and it is the suitability and utility of strategies for a particular learner in a particular context that determines successful learning. In other words, cultural background is but one of many factors that can impinge on the choice of vocabulary learning strategies, although it should surely be taken into account when planning strategy-based instruction.

Banasiak-Ryba's study has some clear virtues, the most important of which are: (1) it contributes to the still sparse body of research on the use of vocabulary language learning strategies across cultures; (2) it is based on a taxonomy worked out by the researcher and (3) it explores strategy use related to a specific task. Its findings therefore have pedagogical implications as well as helping us plan further research. The drawbacks of the study are: (1) only a small number of respondents are used, which does not allow generalizations to be made; (2) the respondents are divided into groups according to their gender and professional profile, which diverts attention from the major question – the influence of cultural background; (3) there is a lack of precise information about data analysis, which makes it hard to interpret the data correctly; and (4) the survey of the use of vocabulary learning strategies is not properly validated.

Kobyłecka (Reference Kobyłecka2006) investigated the influence of cognitive, metacognitive and motivational factors on Polish upper secondary school learners’ attainment in English. Inspired by earlier research on self-regulated learning (e.g. Zimmerman Reference Zimmerman1990), she set out to determine which personal variables influence self-regulation in learning English and to establish the relevant correlations in a descriptive study using a sample of 195 learners. The data were collected with a specially designed questionnaire, based on the Motivated Strategies for Learning Questionnaire developed by Pintrich et al. (Reference Pintrich, Smith, Garcia and McKeachie1991). It contained 115 five-point Likert scale items designed to tap the students’ beliefs, desires and attitudes, as well as an open-ended question which aimed to elicit self-regulating strategies the subjects would use to study for a test taking place the next day. Before the final version of the survey was administered, a pilot study was conducted to assess its utility and to detect any irrelevant questions. The quantitative analysis included tabulating Pearson and Spearman's correlations, as well as running MANOVA; the qualitative analysis focused upon self-regulatory strategies, including cognitive strategies, resource management strategies, metacognitive and control regulation strategies.

The findings seem to confirm the understanding of self-regulated learning as a process in which metacognition, cognition and affect play a crucial role. The strongest correlations between components of self-regulated learning and achievement were identified for test anxiety, self-determination, self-regulation and cognitive control, the first of these being the strongest. It was also discovered that both test anxiety and subsequent poor achievement could be attributed to poor preparation, resulting from the learners’ inappropriate study habits. Accordingly, significant differences were identified in the use of self-regulated strategies between high- and low-achieving students, as confirmed by the results of MANOVA. The findings also allowed the identification of the following groups of strategies used by learners: learning strategies, control and regulation strategies, resource management strategies and non-constructive strategies, such as procrastination. In addition, it was demonstrated that the self-regulation may fail in a situation when certain conditions are not met. On the basis of the results, the author makes some suggestions about strategy training, although this section is rather vague and the conclusions are not well justified.

All in all, Kobyłecka's dissertation provides useful insights into the process of self-regulation in learning English as a foreign language. The quantitative analysis of the data allowed the researcher to identify the underlying components of self-regulation, while the qualitative results provided evidence for the idiosyncratic ways in which students activate self-regulation strategies. One of the major drawbacks of this dissertation, however, is that it uses a questionnaire as the only instrument for data collection, a problem that cannot be easily remedied by subjecting the data to both qualitative and quantitative analysis. Additionally, in analyzing the data, Kobyłecka constantly makes references to other authors and their research, which obscures the overall picture and draws the reader's attention from the main issues. Again, it would have been more helpful if the author had made an attempt to keep the analysis and the discussion of the results separate.

Klimas (Reference Klimas2009) invites us to consider the importance of goals and goal-setting, reporting a longitudinal study which comprised two phases, each with its own set of aims, subjects and procedures: (1) examining the willingness of upper secondary school students to set goals in learning a foreign language, the nature of these goals and their compatibility with educational and social objectives as well as the external influences on the process of determining these goals, and (2) exploring the process of goal implementation, the factors aiding or impeding it and the impact of goal-setting on motivation and achievement. The instrument applied in the first phase to elicit data from 161 participants was a survey consisting of three open-ended questions and 95 five-point Likert-scale items, the latter grouped into eight subscales (i.e. students’ readiness to set and achieve goals, contextual influences on goals, integrative/friendship goals, intrinsic goals, instrumental goals, achievement, social goals and educational/learning goals) and inspired by a number of existing instruments (e.g. Dörnyei Reference Dörnyei1990; Ehrman Reference Ehrman1996; Gardner, Tremblay & Masgoret Reference Gardner, Tremblay and Masgoret1997; Elliot & McGregor Reference Elliot and McGregor2001; Brown Reference Brown2002). The second phase was quasi-experimental in nature and involved a total of 59 participants in four groups, two of which were for practical reasons designated as experimental and two as control groups. The instructional treatment spanned the period of one school year (around 10 months) and took the form of regular goal-setting and goal-evaluation sessions conducted by the researcher, who was the learners’ regular teacher. In this case, the data were collected in the following ways: (1) a motivation questionnaire administered before the intervention, after the first semester and at the end of the school year; this comprised 36 items divided into three subscales: motivational intensity, the desire to learn English and attitudes towards language learning, (2) a goal-setting logbook, which enabled the subjects to, first, evaluate the importance of, as well as the possible obstacles to, the goals specified by the researcher for a particular week and, second, indicate the percentage levels they achieved and add their own comments, and (3) as assessment of the subjects’ overall performance in the course measured according to the school criteria and expressed in percentages based on points accumulated throughout the school year.

The findings indicate that the participants were to some extent willing to set their own goals but their decisions were hugely influenced by such contextual factors as the teacher, the course book and the curriculum. As for the nature of these goals, many were pursued simultaneously, with instrumental goals being treated as more important than integrative, achievement goals playing a significant part, and social goals being less important than other categories. It was also found that the goals mentioned were to a large extent consistent with the objectives listed in the curriculum, being conducive to the development of target language communicative ability. An important outcome was that raising awareness about the process of goal-setting was effective, as evidenced in the fact that it led to statistically significant differences between the experimental and control groups, with the former maintaining or strengthening their motivation and achieving better results in learning. An undeniable strength of the research project was that it was longitudinal, conducted in real classrooms and employing a variety of data collection tools. Particularly commendable, though, is the inclusion of a proper piloting stage for all the data collection tools and an attempt to ensure their internal consistency and reliability through calculating Cronbach's alpha for different scales or the use of test-retest procedures. There are also, however, some limitations, related to the use of different participants in the two phases of the experiment, unclear validity and reliability of achievement measures, and the related difficulty of determining whether the experimental and control subjects represented the same or similar levels of proficiency.

Pospieszyńska-Wojtkowiak (Reference Pospieszyńska-Wojtkowiak2010) explored the impact of individual differences on the development of speaking skills. First, she reports the results of a preliminary study which investigated the relationship between individual difference variables, including self-esteem, motivation, reflectivity and impulsivity, tolerance of ambiguity and learning style and success in learning foreign language pronunciation, and examines the factors which influence the choice of the pronunciation activities learners engage in. The data were collected from 25 English majors in the first year of a B.A. program and included: (1) the results of questionnaires which measured the variables listed above, (2) information obtained through unstructured interviews intended to verify the quantitative data yielded by the inventories, and (3) the scores on pronunciation reading evaluation, which involved a passage constructed on the basis of persistent problems in the students’ pronunciation recorded at the beginning and end of the academic year, and a pronunciation speaking evaluation in which the subjects’ individual oral performance on the final examination was recorded; in both cases, pronunciation was evaluated by means of a five-point holistic evaluation scale developed for the purpose of the study. Statistical analysis demonstrated that success in pronunciation learning was related to high self-esteem, extrinsic and intrinsic motivation, as well as a communicative learning style, but only on measures administered at the end of the academic year.

The main study investigated the influence of individual differences on aspects of the spoken performance of 299 students in B.A. programs in two similar institutions of higher education. Individual difference variables were measured by means of Oxford's (Reference Oxford and Reid1993) Style Analysis Survey and included the following learning styles: visual/auditory/kinaesthetic, extraversion/introversion, abstract/concrete-sequential, closure-oriented/open, and global/analytic. Success in oral performance was operationalized in terms of attainment on the pronunciation, grammar, vocabulary and fluency subscales in the course of an oral interview which formed part of the end-of-the-year examination, as well as the composite score on all these measures. Statistical analysis in the form of multiple regression and non-parametric tests of significance showed that extraversion influences fluency and spoken grammar, and that abstract learners outperform concrete-sequential learners on the vocabulary dimension of spoken performance, but it also demonstrated that auditory learners do not outperform visual learners on pronunciation and global learners perform similarly to analytic learners on spoken grammar. The author concludes that such findings should sensitize teachers to the existence of individual preferences in developing speaking skills and encourage them to take heed of these differences in their classroom practices.

In light of the fact that the influence of individual variables on speaking performance is in urgent need of empirical investigation, the dissertation can be viewed as ground-breaking, particularly because it looks at clusters of such factors and examines oral attainment not only overall but by breaking it down into a number of more specific categories. A weakness of the main study, however, is the fact that the data on speaking performance were collected during a high-stakes assessment procedure and, given the fact that the situation is stress-inducing and far from spontaneous, this could have significantly affected the results. An interview conducted specifically for the purpose of the study would have definitely been a better idea, although it would have been logistically difficult and the decision made by the researcher was one of convenience. A study like this ought to serve as a basis for concrete pedagogical implications and directions for future research, but this is unfortunately not the case here.

4. Language skills and subsystems

Another research strand in the doctoral dissertations selected for review here is reflected in studies dealing with the development of target language subsystem and skills. The five theses included in this section touch on issues related to learning and teaching grammar (Mystkowska-Wiertelak Reference Mystkowska-Wiertelak2010), phonetics (Nowacka Reference Nowacka2008; Rojczyk Reference Rojczyk2008), reading (Cirocki Reference Cirocki2008) and writing (Salski Reference Salski2006). The absence of studies focusing upon vocabulary, pragmatics, speaking and listening is conspicuous.

Mystkowska-Wiertelak (Reference Mystkowska-Wiertelak2010) investigated the value of production-oriented grammar teaching, as embodied in the still ubiquitous PPP procedure (presentation – practice – production), and input-based grammar instruction, as implemented in interpretation tasks (Ellis Reference Ellis1995) and processing instruction (VanPatten Reference VanPatten2002). She reports the results of four studies which compared the effectiveness of the two approaches as follows: (1) output-based instruction and interpretation tasks in teaching inversion to 24 English majors, (2) production-oriented instruction and input processing in teaching passive voice to 33 primary school education students divided into one control and two experimental groups, (3) production-oriented, input-based and instruction combining both of these options in teaching causative have to 64 English majors divided into one control and three experimental groups, and (4) meaningful output-oriented and reception-based instruction in teaching causative have to 57 English majors divided into one control and two experimental groups, also in terms of attainment of the individual subjects. Although the design differed somewhat from one study to the next, becoming more elaborate in successive research projects, most included measures of productive and receptive explicit and implicit knowledge on pretests and immediate and delayed posttests.

The findings are sometimes complex and difficult to interpret but they indicate that, contrary to the claims advanced by the proponents of processing instruction, output-oriented and input-based options are equally effective with respect to the subjects’ monitored and spontaneous ability to use the targeted structures in production and reception, both immediately after the treatment and in the long run. The results also pointed to the need to take account of individual differences between learners, such as attitudes, beliefs, preferences and learning styles, as in some cases these mediating variables may turn out to be more important than the mode of instruction. What is particularly valuable about the studies is that they are, as far as we know, the only attempt so far to verify the utility of input processing instruction in the Polish context, such verification being indispensable for determining the feasibility of pedagogic options proposed by researchers, and that they attempt to investigate the role of individual variation in grammar instruction. The weaknesses of these studies are related to their inconsistent operationalization of output-based instruction as well as insufficient emphasis on the role of the inherent characteristics of the targeted structures; however, the latter is a major drawback found in many similar studies.

The theses dealing with instructed acquisition of pronunciation in English as a foreign language were submitted by Nowacka (Reference Nowacka2008) and Rojczyk (Reference Rojczyk2008). The first of these explores the phonetic attainment of Polish university and college students both in terms of production and reception of selected pronunciation features. Nowacka (Reference Nowacka2008) presents the responses to a survey of the attitudes of 91 tertiary level English majors from six different institutions of higher education towards learning pronunciation. She also reports the findings of five experiments which were intended to: (1) trace the development of the productive use of 41 pronunciation features in the careful and fast speech of 25 English majors in a teacher training college over the period of three years, (2) replicate the procedures used in (1) for 62 students of English at university level, (3) evaluate the perception of consonants in listening comprehension (by means of multiple choice items) and sound discrimination tasks (with the help of minimal pair contrasts) by 181 English majors, determine the relationship between the findings of these two, propose a hierarchy of difficulty of consonantal contrasts, individual consonants and the contexts in which they appear, and determine the effects of phonetic training on attainment in this area, (4) examine the perception of fast speech by 60 students of English, relate the scores to attainment, and propose a hierarchy of difficulty for elements of connected speech, and (5) compare the results of the holistic ratings of the pronunciation of 62 English majors by the students themselves, Polish teachers and native-speaker judges. The findings demonstrated that Polish students display positive attitudes to learning English pronunciation and that explicit phonetic training works; although some features are more amenable to improvement than others, it is possible to establish a hierarchy of difficulty for the perception of segmental and suprasegmental features, and students are less critical of their pronunciation than native speakers and their teachers. An undeniable strength of the dissertation is that it represents the most comprehensive empirical investigation of phonetic attainment at the tertiary level in terms of comprehension and production in the Polish context conducted to date. Its limitations mainly concern inadequate control of the non-phonetic variables in the materials employed in the study (such as word frequency or morphosyntactic characteristics), the lack of control groups and the fact that it is largely cross-sectional in character.

The dissertation submitted by Rojczyk (Reference Rojczyk2008), in contrast, is visibly less pedagogically oriented and reports the results of a study which aimed to determine how Polish learners of English perceive the temporal and spectral parameters of the voicing contrast in English, to trace the changes in this perception with the growth of language experience, and to establish whether such experience affects the perception of the voicing contrast parameters which are typical of Polish. The subjects were 11 native speakers of English who acted as the control group, 24 advanced Polish learners of English in the third year of a B.A. program in the Department of English studies, and 26 beginner learners of English attending a language school with only six months’ experience in learning the language. The stimuli employed in the experiment were digital recordings of natural speech samples and nonsense syllables read by native speakers of English and Polish. The subjects were asked to circle the sound they heard in each syllable in a forced-choice identification format (Fowler Reference Fowler1992), with native speakers given English orthographic approximation of these sounds, beginner learners Polish representations thereof and advanced learners the relevant transcribed options. Statistical analysis conducted by means of a Cochran Q test and a Chi test demonstrated a developmental pattern in the perception of the parameters under study, with the caveat that although some of them are relatively easy (e.g. partial devoicing of final fricatives), most pose a challenge even for advanced learners (e.g. vowel duration). At the same time, no evidence was found for backward transfer from English to Polish and no statistically significant differences were detected in this respect between beginner and advanced learners. What needs to be stressed is the innovative character of the research project in the Polish context, its meticulous design and its appropriate choice and use of technical procedures. Its limitations are related to somewhat superficial discussion of the findings and limited pedagogical implications for teaching the perception of pronunciation features in the foreign language classroom.

Cirocki's (Reference Cirocki2008) study is a commendable example of a practicing teacher's involvement in the knowledge-gaining process. He set out to investigate the influence of extensive reading on the development of linguistic and (inter)cultural competence among upper secondary school learners in a research project which was divided into two parts. Drawing upon the findings of a pilot study, background questionnaires were designed with the aim of determining to what extent 100 learners and 100 teachers approved of learning and teaching English through literature, and to what degree techniques based on literature are actually used in language lessons. The results were rather disappointing and confirmed the author's expectations that literature remains underrated, since neither the students nor the teachers were interested in using it, and both groups saw numerous problems connected with its use. The teachers pointed to curricular constraints, believed that reliance on literature was an obstacle to the successful preparation of their students for school-leaving examinations, mentioned lack of proper training, expertise and experience, highlighted the shortage of suitable instructional materials, indicated the insufficient reading habits of their learners, and manifested a reluctance to invest time in preparing out-of-the-ordinary classes. The students, on the other hand, avoided literary texts as they appeared very difficult, boring and outdated. Despite such problems, however, both were of the opinion that literature can foster language development.

Accordingly, the second part of Cirocki's study, in the form of action research, introduced an extensive reading program based on Waring's (Reference Waring2001) Extensive Reading Approach, to determine whether teaching through literature was effective in promoting the development of verbal skills and ensuring the naturalness of communicative situations. The study lasted one semester and involved 20 upper intermediate-level students from one class taught by the teacher-researcher, who devoted two out of three weekly 90-minute sessions to introducing the extensive reading approach. In-class treatment involved language activities based on class readers, giving oral and written presentations about the texts read, and conducting discussions on social/cultural issues, while out-of-class tasks consisted of daily reading practice, preparing oral and written presentations, and writing book reports. The instruments of data collection were the diaries kept by the students and the teacher-researcher, learners’ book reports, reading speed tests, and vocabulary and Certificate in Advanced English (CAE) tests. Pretests and posttests were administered to determine the growth in reading speed, vocabulary development, and gains in the overall target language competence. The data were subjected to qualitative analysis aided by quantitative measures, which demonstrated that the intervention had beneficial results, particularly with respect to such areas as: (1) reading fluency and related positive attitudes to reading in English, (2) vocabulary, the highest gain being 24.16% and the lowest 10%, (3) productive skills, improvement in speaking being positively correlated with the amount of reading, and (4) overall linguistic competence, as demonstrated by the results of CAE tests. Additionally, the data showed improvement in the learners’ (inter)cultural competence and an increase in autonomy and motivation.

Cirocki's study is significant for a number of reasons. Firstly, it demonstrates that encouraging practitioners to carry out research in the classroom is beneficial. Secondly, its undeniable advantage is the fact it was conducted in the context of a Polish state school, which makes its outcomes applicable to the Polish educational context. Thirdly, having been described and documented in a detailed way, it can serve as a model for foreign language teachers interested in introducing innovations in their own local contexts. Finally, the results point to the necessity of introducing changes both to the way languages are taught and to the way teachers are trained in Poland.

Salski (Reference Salski2006) focuses on the development of writing skills among advanced learners of English, adopting as a point of departure the assumption that difficulties in composing texts in a foreign language stem not so much from inadequate mastery of the target language system as from the culturally-determined differences between English and Polish rhetorical patterns, as claimed by the proponents of contrastive rhetoric (Kaplan Reference Kaplan1966), although such an approach should be integrated with a process-based view taking into account writers’ background and preferences. He applied this perspective to digression, which he explored by comparing texts written in English by 30 students of English and Early School Education in a teacher training college to identical assignments penned by 17 American and 15 Polish control participants. Like the study by Kubota (Reference Kubota1992), which provided inspiration for the research project, the data also included the subjects’ responses to a survey examining their writing experiences, preferences and the processes they applied when composing the texts under study. He tested the following hypotheses: (1) Polish writers use more digression in English than American student writers writing in English, (2) Polish student writers use significantly more digression in Polish than American students writing in English as their first language, (3) Polish students are less familiar with process writing procedures than students at American universities, and (4) writers who are taught composition through process-based instruction use less digression. Applying the Framework for the Analysis of Rhetorical Structure of Texts of Gołębiowski (Reference Gołębiowski1998) as well as statistical procedures, Salski failed to find sufficient support in his data for the first two hypotheses. He discovered, however, that digression is indeed more frequent in EFL texts, which he attributed to the subjects’ insufficient level of proficiency in the target language. On the other hand, the analysis of the survey data enabled him to confirm the remaining two hypotheses, which led him to conclude that although process-based instruction is not necessary for good writing, it aids the writer in adopting the reader's perspective and focusing not only on accuracy but also on content and organization. The dissertation holds considerable potential for teaching writing to advanced learners in its proposal that efforts to develop awareness of rhetorical structure should be combined with a shift of attention in teaching writing from the product to the process. As for limitations, it might be worthwhile to examine in more detail individual differences which might have affected the use of digression in the texts, and it should also be noted that the questionnaire was not properly validated and there is no mention of it having been piloted with a comparable group of respondents.

5. Culture

Another area that figures prominently in the dissertations selected for review is culture, as exemplified in the following four Ph.D. theses. It should be noted, though, that interpretations of the notion of culture vary, some authors choosing to focus on the concept of intercultural competence (Maciejewska-Stępień 2007; Owczarek Reference Owczarek2010), another equating it with the sociocultural component of communicative competence (Domalewska Reference Domalewska2009), and another emphasizing cross-cultural differences (Sroka Reference Sroka2008).

Maciejewska-Stępień's (2007) dissertation is devoted to the evaluation of a European project whose aim was to develop a module to be used in the teaching of aspects of intercultural communicative competence (ICC), as proposed by Byram (Reference Byram1997). The PICTURE (Portfolio Intercultural Communication: Towards using Real Experiences) project was assessed in terms of its relevance to the objectives set and its effectiveness and impact. The descriptive-exploratory study discusses thoroughly both the interim stages of module development and the outcomes of the exploratory phases of an investigation which tested the preliminary version with 248 students and 11 university language teachers in Poland. Testing the final version of the module involved 342 students and 14 teachers from Poland, Portugal, Bulgaria and Cyprus. The data collected through questionnaires, teacher interviews and classroom observations were analyzed using both qualitative and quantitative methods. The assessment of the early stages necessitated reliance on documentation, observation of the participants’ plenary meetings, interviews with partners, and an interim evaluation questionnaire to obtain feedback on the materials developed. The evaluation showed that the materials were viewed as relevant to the objectives of the program and were considered to be effective, although some reservations were expressed concerning the uneven quality of the materials, and suggestions for their improvement were made. After several modules had been field-tested, the next body of data was gathered in the form of responses to questionnaires provided by teachers and learners. The teachers were convinced that the materials were pertinent to the objectives of the program, as they promoted the intercultural dimension in language teaching, led to greater understanding between the peoples of Europe and improved the quality of language teaching. The students, in turn, perceived their progress in ICC quite positively, as, thanks to the materials, they learned about other people and cultures. Data were collected again when the final versions of the modules were implemented, with responses to the modified surveys indicating generally positive reception of the materials by both students and teachers. Some problems were also pinpointed, however, related to (for example) the fact that the students perceived the materials as disorganized, irrelevant, ambiguous, inappropriate or too personal, while the teachers described them as superficial and not challenging enough. As a result, changes were recommended in content organization, the elimination of inappropriate or ambiguous statements, the introduction of more variety in language skills and vocabulary, an increase in the level of intellectual challenge and the integration of content with elements of language teaching.

The study has many undeniably good features: (1) it is an example of a well-planned multi-dimensional research project carried out with expertise, whose outcomes are presented in a logical way, (2) it deals with the development of ICC, which, along with autonomy, has become one of the ‘hot’ issues in language education in recent years, (3) it documents the consecutive stages of the development of a teaching module, including modification of data collection procedures, and (4) it offers concrete suggestions for the development of ICC. On the other hand, the dissertation as such suffers from limitations mainly related to its structure (e.g. no clear separation of the theoretical and empirical parts and unnecessary repetition of research questions).

Intercultural communicative competence is also addressed in Owczarek's (Reference Owczarek2010) thesis, which is concerned with whether language education in a teacher training college prepares students to perform as cultural mediators. She carried out a study which aimed to assess this competence, test the instruments of assessment, and single out those aspects of ICC development that need special attention. The research project, divided into three stages, was descriptive in nature, and used a number of data collection procedures, both qualitative and quantitative, to ensure data triangulation. The longitudinal study, carried out over four years and involving 176 students, focused on the evaluation of attitudes (e.g. the ethnocentricity level test), knowledge (e.g. the critical incidents test), skills and the subjects’ critical cultural awareness. The interview stage, whose participants were 20 students taking part in COMENIUS and ERASMUS programs, focused upon their personal and professional experiences during their stay abroad and used data collected over a period of six years. Finally, the cross-sectional stage involved 96 students and focused on the role of literature in ICC development. The participants read a short poem by Rudyard Kipling and were then asked to fill in a poem response questionnaire, in which, among other things, they were requested to reveal the emotions they experienced while reading the poem. Except for the attitude scales, the research tools were all constructed specifically for the research project, with the tests being checked for validity and reliability in a pilot study. Quantitative data were subjected to statistical analyses that involved running a one-way analysis of variance and calculating Pearson product-moment coefficients.

The findings provided a basis for a number of conclusions, of which the following are significant: (1) elements of everyday symbolic culture (e.g. gestures, ways of expressing emotions, body language and proxemics) are crucial and need to be emphasized in teaching cultural mediation, (2) while teacher trainees are prepared to talk about their own country and to discover a foreign culture, they seem unprepared to cope with emotional stress during their stay abroad, and (3) cultural studies taught by means of lectures and emphasizing factual knowledge do not address students’ needs in developing their ICC, as they are mainly seeking opportunities for discussion and dialogue. Additionally, Owczarek identifies the objectives for future research dealing with the assessment of ICC and provides useful recommendations in this connection for future teacher training programs. As for the limitations of the study, these include the use of only a single text to investigate the level of the students’ ICC and their emotional responses; there are also problems with terminology, as there is confusion with respect to different types of research studies.

Domalewska (Reference Domalewska2009) reports the findings of a study which aimed to examine the self-perception of Polish learners of English, defined as their attitudes towards their national group and the coherence of their national identity, as well as variables affecting both of these, including age, gender, place of residence, experience in traveling abroad, self-esteem and participation in religious services. The subjects were 592 lower and upper secondary school learners from a big city, a town and a village in two different geographical regions. They were asked to fill out a questionnaire containing open-ended and closed questions, specifically designed for the purpose of this research project and trialed in a small-scale pilot study, as well as the self-esteem scale constructed by Rosenberg (Reference Rosenberg1989) which comprises Likert-scale items. In another experiment, roughly half (292) of the subjects, acting as an experimental group, were exposed to critical opinions and negative stereotypes about their nationality, and the other half were provided with texts exuding a positive picture of Poles, thus acting as a control group. All the learners were then asked to answer open-ended questions about their perceptions and attitudes. The analysis of the data revealed that young Poles manifest ambivalent attitudes towards their national group which are impacted by the factors listed above. Also, they are more likely to retain information that does not conform to their beliefs, with their general impressions being modified by their existing attitudes. Although they are attached to their family and peers, this is not the case where what are perceived as traditional national values (i.e. religiousness) are concerned. In light of the fact that more and more Poles have opportunities to travel and work abroad, which allows contact with members of other nationalities, the study is valuable in that it provides insights that can help teachers willing to develop the intercultural competence of their learners. As is the case with many other projects discussed in the present review, however, no attempt was made to ensure the reliability and validity of the survey used, and the pilot study seems to have contributed very little in this respect. It should also be noted that the picture of the subjects’ self-perceptions would have been more complete had the author drawn upon some of the latest models in motivational research, especially the notion of ideal language self (e.g. Csizer & Dörnyei Reference Csizer and Dörnyei2005).

A strong emphasis on the role of culture is also visible in the dissertation submitted by Sroka (Reference Sroka2008), who reports on a two-stage study of the development of reading comprehension skills in the first, second and foreign language in the context of Polish and American classrooms. The aim of the first stage was to examine the classroom practices used to develop reading comprehension skills and it addressed such issues as text selection, methods and techniques, assessment, and attitudes manifested by teachers and students towards reading literary texts and existing constraints in this respect. To answer these questions, two case studies were conducted in Polish and American high schools. The data were collected through interviews with 23 Polish and 19 American teachers, 43 hours of observations of classes taught by American teachers and unstructured questionnaires administered to 80 Polish and 43 American students, as well as relevant Polish and American documentation. The aim of the second stage was to explore the curricular and legal constraints determining how reading comprehension skills are developed in the first, second or foreign language in Polish and American culture with respect to such areas as the development of text comprehension, the analysis and interpretation of lexical, grammatical and structural features, elements of historical literary tradition, literary devices and sociolinguistic elements, experiential and critical response, the application of reading and the extent of prescription of texts. In this case, document analysis was carried out focusing on curricula and educational standards, examination standards and matrices, and syllabuses.

A combination of quantitative and qualitative analysis showed that the results of the two stages overlapped and corroborated each other to a considerable degree, revealing some major differences in the ways reading comprehension skills are perceived and developed in the two educational contexts. For instance, it turned out that from American documents that considerably more attention is attached to the development of reading strategies, whereas the Polish educational system places more premium on testing reading skills than on teaching them. Furthermore, in American schools reading is viewed as an opportunity to develop students’ knowledge of vocabulary and grammar while in Poland these subsystems are more likely to be taught separately. Apart from this, in the American context, reading is much more often a springboard for the development of content knowledge, including intercultural competence. At the same time, the development of critical thinking skills and emotional development through reading are emphasized, which is not the case in Poland, where the mere acquisition of knowledge is more important. It should also be noted that American students have more choice in the texts they read than their Polish peers. Finally, in the United States the integration of reading and writing is much stronger than in Poland, but the skill is integrated with speaking in both contexts.

The dissertation closes with explicit guidelines on how teaching reading comprehension skills in the first and foreign language can be improved in the Polish context and concrete implications for future research. While the recommendations are salutary, a question arises as to how far the instructional practices of one cultural context are applicable to another, particularly if no data are available allowing objective comparison of the value of the approaches being compared. Additionally, by the author's own admission, there was an alarming lack of balance in the amount of information gleaned from various sources, with teachers providing the richest data, and American students being much more imaginative and outspoken in their responses than their Polish peers.

6. Assessment

The three Ph.D. theses described below touch on the issue of assessment with respect to specific target language subsystems (Lewandowski Reference Lewandowski2010), examination standards (Poszytek Reference Poszytek2007) and principles of effective assessment (Smolik 2008).

The primary aim of Lewandowski's (Reference Lewandowski2010) dissertation was to examine the feasibility of assessing advanced learners’ knowledge of lexical collocations, analyze the measurement properties of an instrument designed for this purpose, and provide guidelines on how collocational competence can be effectively developed and evaluated. His study describes a systematic process of developing a collocational competence test for advanced tertiary-level students of business English with a view to verifying empirically the statistical properties of a task type selected for this purpose. As most problems with collocations at an advanced level are caused by ineffective organization of the mental lexicon and the inability to recall a collocate, it was decided to adapt a controlled monolingual production recall task, modeled on Laufer & Nation's (Reference Laufer and Nation1999) vocabulary-size test of productive ability, in which test-takers are required to complete a collocation by recalling a specific collocate whose first letters are provided, to match a node presented in a sentence context. As the test was intended for advanced students of business English, it focused on lexical collocations that are likely to be needed in business and communication studies, a solution which allowed selection of a representative sample of vocabulary items. Four consecutive versions of the test, successively comprising a smaller number of items (153, 114, 82 and, finally, 70) were administered to different groups of subjects (native speakers, teachers of business English, students of international business and students of business and economics), with the analysis of the results and the performance of particular items following each trial providing a basis for improving the quality of the measure and examining the usefulness of the test as a whole. The data were subjected to in-depth statistical analysis to assure high quality measurements in terms of reliability, discriminating power, validity, practicality and washback. Consequently, the final version of the test could be seen to meet the main criteria of a good vocabulary test, which generates reliable scores and should therefore have a positive effect on the process of learning and teaching collocations. Lewandowski's study has numerous good features that far outweigh its minor weaknesses, which are mainly of an organizational nature. It has to be emphasized that, to our best knowledge, it is a pioneering undertaking in the Polish context.

Poszytek (Reference Poszytek2007) developed methodology and tools to determine the extent to which the Polish secondary school leaving examination in English takes account of the European trends and standards reflected in the Common European Framework of Reference for Languages (CEFR: Coste et al. Reference Coste, North, Sheils and Trim2000). In particular, he addressed two research questions, namely: (1) whether the Polish examination draws on the European model of language proficiency with respect to underlying constructs, activities and strategies, and (2) whether the proficiency of those taking the examination relates to the European model of proficiency with respect to the tasks included in the examination and the performance of the examinees. The study comprised two stages that involved the analysis of the relevant documents and the results of the examination conducted in the school year 2005–2006. It combined quantitative and qualitative procedures, the former involving the calculation of the average scores of test-takers on the written part of the examination with respect to language proficiency parameters and the CEFR scales for these parameters and sub-parameters, and the latter consisting in scrutinizing the data to establish the validity of the examination and its compatibility with the European language proficiency scale. The findings demonstrated that there were major disparities between the Polish examination and European standards, especially with respect to listening and reading comprehension and, to a somewhat lesser extent, written production, spoken interaction and spoken production, which did not give due attention to measuring sociolinguistic competence. Additionally, the extended level was usually a better reflection of the European standards than the standard level; there was a similar trend for the 2005 and 2006 versions of the examination. As regards the actual results, some deviations were observed from declared proficiency levels in the case of some tasks as well as differences in proficiency levels on different parameters within the same task. These findings led Poszytek to conclude that it would be helpful to create a bank of individual items and calibrated tasks, with each item or task specifying the parameters it aims to measure at each proficiency level, and to include with the test a survey of language education outside school, which would allow a more detailed and valid analysis of student performance. While he regards the CEFR as a useful tool for designing, producing and evaluating tests, he recognizes its shortcomings, and suggests that future research should address such areas as the impact of task content, response format, skill integration and teaching methods on examination scores, as well as further analysis of task items.

The approach adopted by Poszytek is certainly commendable as it has the potential for comparing final examinations to a set of generally accepted standards, thus contributing to the greater validity of testing procedures. However, one has to bear in mind the limitations of the CEFR related, for example, to its superficial treatment of language subsystems and the difficulties involved in using it as a point of reference in test construction (cf. Weir Reference Weir2005). Another contentious issue is the extent to which it is advisable or even feasible to include a unified set of criteria in national assessment procedures, given that the realities and demands of foreign language learning may vary from one country to the next.

Issues related to the final examination in English in upper secondary schools are also the focus of Smolik's (Reference Smolik2007) dissertation. He investigated the rating process in the speaking part of this examination at the standard level with a view to determining the extent to which its scores meet the criteria of validity and reliability or, more specifically, Weir's (Reference Weir2005) concept of scoring validity. The investigation took the form of an instrumental case study (Stake Reference Stake1995) and focused upon such areas as rating conditions, raters and training, tasks and scales, consistency and agreement, rater severity and bias, the use of rating scales, rater perception of interlocutor competence, score resolution methods and the interchangeability of raters and teams. A separate issue was whether the methodology applied in the study – stimulated recall and mixed-method research – is useful in investigating scoring validity. The subjects were 21 teachers, 12 of whom were trained examiners, and 45 final grade students, familiar with the format of the examination. The research instruments, which were carefully piloted and revised, included: (1) four sets of stimulus materials developed in accordance with the examination criteria, which contained three guided conversations and one picture description task, (2) video recordings of mock examinations, (3) two rating scales, one used in the real examination and one intended to provide information on the best predictor for the overall grade for language, (4) competence of the interlocutor rating sheet, included to find out whether the rater's perception of the learner affected the assigned grade, (5) individual stimulated recall reports and team discussions collected from raters, (6) a background questionnaire for teachers which supplied not only demographic data but also information about the examination and rating procedures, and (7) supplementary documents such as instructions, letters, opinions and certificates. The data were subjected to quantitative and qualitative analysis, the former using descriptive and inferential statistics, the latter being applied to responses to open-ended questions, individual stimulated recall reports and teacher discussions.

Although Smolik (Reference Smolik2007) found levels of intra- and inter-rater agreement in the region of 80%, he argues that this does not provide adequate evidence for the accurate mapping of speaking performance onto rating scales, a conclusion that finds support in the data obtained by means of other instruments (e.g. stimulated recall). This led him to conclude that the scores were not very dependable, indicating that caution should be exercised when interpreting them and that they should not be put to practical use, although he is aware that the data is only a fraction of all the available validity-related evidence. As far as research methodology is concerned, Smolik argues that reliance on a mixed-methods approach yielded a clearer and more comprehensive picture of the phenomena under study, even if the generalizability of the findings of any case study is always inherently limited. The dissertation also suggests some concrete implications for testing and teaching practice, testing and validation theory and validation methodology, postulating, among other things, that the effectiveness of examiner training should be enhanced and the rating scales should be subjected to close scrutiny. He advocates caution in the interpretation and use of the results, modification of examination-preparation courses, and reliance on a multi-faceted approach to the study of scoring validity. Again, the value of the research project lies in the insights it offers into the procedures of the examination, providing an opportunity to introduce the changes required. However, the question arises as to whether it can be justified to use such complex statistical procedures with relatively small samples of participants to explore issues which, as the author himself admits, may not easily lend themselves to quantification. The use of both quantitative and qualitative methodologies is, of course, one answer to this, but perhaps in such cases it is qualitative methods that should receive more emphasis.

7. Miscellaneous

Since three of the dissertations under review did not fit any of five main categories created for the purpose of this paper, we decided to group them together in one section. They are the theses by Rokoszewska (Reference Rokoszewska2007), on teaching English to young learners, Jaroszek (Reference Jaroszek2008), on the development of discourse competence, and Horak (Reference Horak2008), on classroom interaction.

Rokoszewska (Reference Rokoszewska2007) compared the effectiveness of the eclectic method (cf. Larsen-Freeman Reference Larsen-Freeman2000: 183) and the Helen Doron (Reference Doron1993) method of teaching English to young learners as used, respectively, at state schools and in private language courses. The research project, which spanned one year and involved 121 subjects aged six to eight, had the following aims: (1) to obtain information about children's attainment in terms of comprehension, production, acquisition of lexis, grammar and language chunks, and (2) to establish whether the two methods resulted in significant differences in these areas. The participants were divided into three ‘regular school’ groups and three ‘Helen Doron’ groups according to age, all of which – six groups altogether – were taught by the researcher. In the case of the former, the regular coursebook was used during two 30- or 45-minute lessons a week, while instruction in the latter was conducted by means of specially designed materials in one 45- or 60-minute lesson a week, complemented by listening to a tape twice a day. Five progress tests, administered every two months, were used to estimate the children's achievement in terms of seven different categories, namely vocabulary, grammar, language chunks, listening, speaking, pronunciation and language acquisition. The analysis revealed that the differences between the results obtained by the two groups (i.e. the regular school and the Helen Doron pupils) were statistically significant, which led the author to the following conclusions. On the whole, the children taught by the Helen Doron method outperformed those taught eclectically on all the measures, except for the ability to correct pronunciation. They demonstrated a clear advantage in their ability to use the target language, including productive use of grammatical structures. In addition, they learned significantly more structures and language chunks, though they were less successful in understanding grammatical items than the regular school pupils. Such findings were accounted for by – among other factors – differences in exposure to the target language, the recycling of the material, the size of groups and their motivation.

Although the study is an example of a meticulously planned and carefully conducted experiment, it suffers from some limitations, chief among which were that the groups of children being compared were taught and tested by means of different coursebooks and procedures, and that the teacher was also the researcher. In addition, it was conducted for a relatively short period of one school year, which does not allow one to draw conclusions about long-term attainment in terms of proficiency. Moreover, there seems to be a problem with the way the author uses the term language acquisition, which, as well as causing some conceptual problems, obscures the results.

Jaroszek (Reference Jaroszek2008) reports a longitudinal study undertaken to explore the development of discourse competence in teaching EFL and determine the factors affecting this development. It was based on the assumption that teachers tend to perceive communicative competence as consisting mainly of grammar and sociolinguistic competences, neglecting discourse competence, which could account for the awkward use of collocations and sentence wording by many advanced learners. The participants were 13 advanced students of English at a teacher training college, whose development in terms of discourse competence in spoken output was documented over a period of three years. The main study was preceded by a survey study to identify the spoken discourse mechanisms employed by the students with the aim of selecting some for further investigation, and a pilot study conducted to check the suitability and reliability of the data collection procedures. An analysis of the recordings of the performance of 12 randomly chosen students on two discussion tasks in the survey study led to the selection of discourse devices such as modality, conjunctions, relexicalization, reiteration, ellipsis and substitution, all of which are to some extent used differently in L1 and L2. The pilot study, in turn, enabled the researcher to design the data collection procedures to be used in the main study, which included weekly student diaries to record types of L2 exposure and classroom interaction, student interviews in both English and Polish, a native speaker interview (to compare the subjects’ L2 competence with that of their English peers), analysis of teacher talk (classes of 12 teachers of theoretical subjects were recorded), and analysis of classroom procedures which was based on observations and served to verify the subjects’ weekly reports.

The main findings were as follows: (1) the development of discourse competence may take a peripheral position and it was taught predominantly with respect to grammar, (2) throughout the study significant progress was made as far as discourse features are concerned, the teacher factor playing a dominant role in this respect in the first year, and the authentic L2 factor being the most pronounced in the third year, and (3) possible cross-linguistic influences in discourse construction were detected (e.g. in the case of the Polish and English deontic modality). As regards the factors contributing to the development of discourse competence, correlational analysis revealed that they included teacher talk, access to authentic input, and L1 cross-linguistic influence and teaching procedures, with the caveat that these factors often interact with one another. It was often not so much the frequency of selected discourse aspects that distinguished L2 learners’ speaking conventions from a natural conversational style as the qualitative features of spoken language output. In the light of these findings, the main strength of the study is that it draws attention to some obvious limitations of classroom teaching and suggests that they can be overcome by intensifying contact with authentic English, whether through individual interaction with L2 native speakers or passive exposure to input. At the same time, the question arises of whether reliable conclusions can be reached, given the rather small number of participants.

Horak (2007) examines the sociolinguistic and psycholinguistic aspects of discourse in the foreign language classroom and learning-teaching processes from the perspective of teachers and students. The study took the form of an action research project involving 30 teachers of English and 781 students. Its main objectives included raising teachers’ awareness of the factors that are most likely to influence teacher-learner interaction in the classroom, as well as encouraging them to reflect on their teaching. Further objectives involved making teachers aware of the discrepancies in the ways their classroom practices are assessed by them and their students, and raising teachers’ awareness of who their students are and what expectations they have concerning classroom discourse. The study, which took place in two upper secondary schools, spanned three school terms and used methodological triangulation, the data being collected with five different tools: questionnaires for teachers and for learners (to supply background information about the subjects and their perceptions of classroom interaction), observations of 60 lessons (to verify the survey responses), semi-structured interviews with teachers (in which they could account for the discrepancies in the data), and a diary kept by the researcher in which she reflected upon the teaching-learning process and her rapport with the students.

The analysis of the data, which involved both quantitative and qualitative measures, led to the identification of a range of student-related and teacher-related variables, psycholinguistic and sociolinguistic in character, which may influence classroom discourse. In the case of the students, the psycholinguistic factors included their attitudes to English, motivation to learn, age and stage of psychological development and affective needs. The sociolinguistic factors were their family background, perceptions of the value of education, individual and language group profiles. Teacher-related psycholinguistic factors were linked to the teacher's personality, his or her attitudes and beliefs about teaching, teaching style and professional self-assessment, while the sociolinguistic factors had to do with the teacher's family background and learning history as well as how he or she uses verbal and non-verbal language to approve or disapprove of student performance, maintain discipline, and assess learners’ work. Horak concluded that, although there are many similarities between teachers’ and learners’ perceptions of classroom discourse, there are also many differences, which speaks to the need for teachers to conduct action research which would help them become reflective practitioners and develop the right relationship with their students. An undeniable strength of the study is that it throws new light on the way in which teachers’ and learners’ perceptions shape the nature of classroom interaction and thus determine learning outcomes, especially because it offers evidence obtained through the use of various data collection instruments. Its main weakness arises from the fact that no attempt was made to link the factors under study with specific aspects of classroom discourse, which could have been done if the researcher had recorded the lessons and then engaged in in-depth analysis of specific episodes. In addition, it would have been interesting to see how factors of this kind translate into learning outcomes, which, again, would have been possible through audio-recordings (indicating, for example, the amount and nature of participation) or by interpreting such factors with respect to measures of achievement.