1. Introduction
This report describes the results of a pilot study from a wider project seeking to link the commitment mindsets of English teachers and the learning outcomes of their students. Although the relationships among teacher commitment, teacher retention, and turnover intentions have been well-established (e.g., Day & Gu, Reference Day and Gu2007; McInerney, Ganotice, King, Marsh, & Morin, Reference McInerney, Ganotice, King, Marsh and Morin2015), there is still a need to better understand the relationship between language teacher commitment and teaching effectiveness; that is, to see whether more committed teachers make better teachers (Moodie & Feryok, Reference Moodie and Feryok2015). The main impetus for this pilot study is to explore how this question can be operationalized and whether the currently adapted methodology will be sensitive enough to pick up meaningful differences concerning possible links between teacher commitment and the learning outcomes of students.
To measure commitment, the study adopts a survey from industrial/organizational (I/O) psychology research which operationalizes commitment into three of its components (Meyer, Allen, & Smith, Reference Meyer, Allen and Smith1993; Meyer & Herscovitch, Reference Meyer and Herscovitch2001). These components are affective commitment (e.g., working in a field because of interest in it), continuance commitment (e.g., working in a field because of a lack of other alternatives), and normative commitment (e.g., working in a field because of a sense of duty towards it). Based on a hypothesis that workers with higher affective commitment are more productive than those with lower affective commitment (Meyer & Herscovitch, Reference Meyer and Herscovitch2001), the pilot study addresses the following research question:
To what extent can teacher commitment predict student learning outcomes given a small, medium, and large bias towards students whose teachers have higher affective commitment than their colleagues?
Inevitably, because of the confounding variables associated with language teaching (e.g., experience, education) and learning (e.g., motivation, willingness-to-communicate, aptitude), a question such as this poses challenging methodological issues. Therefore, for the pilot study, we created a database of mock student testing data which can control for such variables. An additional benefit of using mock (i.e., simulated) student testing data is that we can test the sensitivity of our statistical model before applying it to real-world data.
2. Methods
2.1 Participants
The participant sample comprised expatriate faculty at a private university in South Korea. The university has a standardized English curriculum and it keeps records of pre- and post-semester testing data, which presented a good opportunity to undertake this project. Seventy teachers (18 females and 52 males) completed a survey for the pilot study.
2.2 Data
Data came in two forms: the survey and the mock testing data. The survey, adapted from workplace commitment research (McInerney et al., Reference McInerney, Ganotice, King, Marsh and Morin2015), had three scales, each with ten Likert-type items measured on a scale of 1 = Strongly Disagree to 7 = Strongly Agree. The scales measured participants' affective, normative, and continuance commitment to the field of English language teaching.
The mock test data, created in MATLAB R2017a (The MathWorks Inc., Natick, MA, 2017), were designed to simulate the distributions of test scores and class sizes at the university. First, we set constraints for the number of students per teacher and assigned simulated classes to each participant at random, leading to a sample of 2,041 mock students and an average class size of 29.16. Then, based on preliminary real-world language testing data, we simulated pre-test scores for each student. For this, we set the average pretest score at 15 out of 30 with a standard deviation of 4. Then, to generate post-test data, we assigned a default improvement for each student at random. To do so, we set the range for improvement at −1.5 – 4 points with an average of 1.25 points improvement per student.
Following this, the teacher participants were sorted into three groups based on their total affective commitment scores (i.e., high, medium, and low). The next step was to introduce biases favouring the virtual students whose teachers had high affective commitment scores. For this, we created three sets with an average of a small (0.1, or 0.33%), medium (0.5, or 1.67%), and large (1.0, or 3.33%) advantage added to the default improvement scores.
2.3. Analysis
First, descriptive results of the survey were generated in SPSS, and each item and scale were checked for normality. Skewness and kurtosis were within an acceptable range (−1.000–1.000) for all the scales.
Next, we ran a multiple linear regression. The post-test scores for the mock students were set as the dependent variable, with the pre-test scores and their instructors' total affective, continuance, and normative commitment scale scores constituting the independent variables. We applied backwards elimination, a method which removes variables from the model which have no significant impact on the dependent variable.
3. Results
First, the survey results demonstrated that the scales were reliable measures of commitment, with a Cronbach's alpha of between .827 and .914. The participants' affective commitment scores (M = 44.89) were higher than their continuance commitment scores (M = 36.31), which were in turn higher than their normative commitment scores (M = 30.23).
As for the regression analysis, with the small-effect dataset (plus 0.1 points, or 0.67%), backward elimination removed the normative commitment variable, and continuance commitment had a negligible impact on the post-test scores. Affective commitment had a slight but significant relationship with the post-test scores and could explain 2.2% of the variance. Of course, given how these data were generated, the pre-test values where very strongly related to the post-test scores. When holding the other variables constant, the pre-test could explain 99.5% of the variance in the posttest scores.
With the medium-effect dataset (plus 0.5 points, or 1.67%), all variables were deemed significant. However, continuance and normative commitment explained a negligible portion of the post-test results. Affective commitment, on the other hand, had a larger relationship to post-test scores than in the previous dataset, explaining 9.6% of the variance. This time, again, the post-test scores were largely predicted by the pre-test scores, explaining 99.0% of the variance when holding the other variables constant.
With the large-effect dataset (plus 1.0 points, or 3.33%), again, all the variables fit the criteria for inclusion. The pre-test had a very strong relationship with the post-test results, explaining 96.9% of the variance, whereas continuance and normative commitment each had a slight but significant inverse relationship with the post-test results. Predictably, affective commitment had a larger relationship with the post-test scores than in the other two datasets. In this case, while holding the other variables constant, the teachers' affective commitment could explain 18.6% of the increase in the learning outcomes for the virtual students.
4. Discussion and conclusion
Based on the hypothesis that teachers with higher affective commitment are more effective, the study tested biases with mock posttest scores for students whose teachers had greater affective commitment than their peers. Results from multiple regression analyses predictably showed that as the bias increased favourably towards students whose teachers scored high on affective commitment, so too did the amount of improvement explained by their teachers' affective commitment scores. Holding other variables constant, affective commitment could explain 2.2% of the improvement in the small-advantage dataset, 9.6% in the medium-advantage dataset, and 18.6% in the large-advantage dataset. Although the study oversimplifies complex and dynamic factors related to language teaching and learning, the key finding from these results is that the adopted approach will pick up any statistically significant real-world effects of teacher commitment.
Given the results, we shall conclude with a few remarks concerning the use of simulated testing data in the present study and the implications for the wider project of which it is a part. First, we hope that our documentation of the methods and parameters with which we created the mock data here is of interest to researchers who wish to make use of such procedures. As with this study, using mock student data provides a means of controlling confounding variables while developing and testing the sensitivity of statistical models. As such, these methods can be applied to many types of quantitative applied linguistics and education research involving statistical modelling. For interested readers, we have made the mock datasets and the procedures we used to create them freely available on GitHub. For interested readers, we have made the mock datasets and the procedures we used to create them freely available on GitHub (https://github.com/Rens88/EnglishLanguageTeaching).
Second, based on the positive and significant associations of the affective commitment of teachers with the mock student testing data in this study, we have determined that affective commitment can indeed be associated with meaningful student improvement. The next step, then, will be to see if this hypothesis holds up with authentic testing data, and when other attributes of learners (e.g., proficiency level) and teachers (e.g., education and teaching experience) are included. With some 100 teacher participants expected for the main study, exploring the relationships between these variables and the learning outcomes of their roughly 4,000 students will provide an opportunity to develop a better understanding of the characteristics of effective language teachers.
Acknowledgement
This study was supported by research funding from Mokpo National University in 2018.
