1. Introduction
This paper presents an argument for replication of discourse studies in the area of genre analysis in EAP. It outlines three recent studies on academic writing and a number of ways in which they could be replicated approximately and, in the case of one of the studies, conceptually, and suggests the potential benefits of doing so. The studies are situated in the genre movement known as English for Specific Purposes (ESP) (Hyon Reference Hyon1996; Burns Reference Burns, Byram and Hu2012). They represent a fairly new and as yet relatively untested strand of research within this movement: comparative genre analysis. Replication of example studies would thus be timely. The studies, referred to as the ‘original studies’ in the remainder of this paper, have been published in the two key international journals read by researchers, materials developers and teachers of EAP: English for Academic Purposes Journal and English for Specific Purposes. Each of the original studies focuses on the writing of a particular section of a research report: introductions (Soler-Monreal, Carbonell-Olivares & Gil-Salom Reference Soler-Monreal, Carbonell-Olivares and Gil-Salom2011), results (Lim Reference Lim2010) and discussions of results (Basturkmen Reference Basturkmen2009). Each compares the writing of two groups: Ph.D. students writing in English and in Spanish (Soler-Monreal et al. Reference Soler-Monreal, Carbonell-Olivares and Gil-Salom2011), researchers writing in applied linguistics and education (Lim Reference Lim2010) and the writing of dissertation students and published academics (Basturkmen Reference Basturkmen2009).
Studies can only be replicated satisfactorily if they report their methods in sufficient detail (Mackey Reference Mackey2012). The three original studies include detailed descriptions of procedures and both definitions and illustrations of key categories (moves and steps) used in the analysis, thus making replication feasible. As advice given to EAP teachers, materials developers and academic supervisors is often informed by findings from genre studies, it is important that those findings are demonstrably reliable. Replication designed to verify findings and indicate how far they can be generalised can play an important role in the accumulation and consolidation of knowledge about discourse (Paltridge Reference Paltridge2012).
The original studies are based on the same approach to genre analysis, used similar methods and were subject to similar limitations. These limitations and the wish to understand how far the findings can be verified and generalised motivate most of the suggestions made in what follows.
The remainder of this paper is organised into three sections. Section 2 describes the aims and methods of the ESP genre approach. Section 3 describes the three original studies and their significance to the field, and makes specific suggestions for replication. Section 4 concludes the paper.
2. Background
Genre analysis has become a highly influential approach to the analysis of written text in EAP and many studies taking this approach have been published in English for Specific Purposes and the Journal of English for Academic Purposes, often describing a written genre and providing recommendations for writing instructors, academic supervisors, or developers of instructional materials. The ESP genre tradition, like two other major traditions – Australian genre-based educational linguistics and North American New Rhetoric studies – involves analysis of the situational contexts of texts and the communicative functions and purposes of the text within those contexts (Burns Reference Burns, Byram and Hu2012).
Studies in the ESP tradition draw on the approach to analysis pioneered by Swales (Reference Swales1990, Reference Swales2004) and further developments by other scholars, including Bhatia (Reference Bhatia1993). In this tradition a genre is seen as a collection of rhetorical choices operating as a schema for social action. Analysis aims to explicate and make visible the rhetorical choices that characterise the genre (Swales Reference Swales2009). Research typically aims to describe the context of use, the genre's rhetorical structure (which is portrayed as a sequence of moves and steps) and, in some cases, the salient linguistic features of the genre. Extension and follow-up studies have been conducted to investigate how far a model of moves and steps, developed from the analysis of a genre in one group or discipline, can be applied (generalised) to that produced by another; see, for example, Samraj (Reference Samraj2005) and Basturkmen (Reference Basturkmen2012). One of the difficulties of interpreting some earlier research findings was that although they showed varying numbers and kinds of moves and steps (in the same genre) this may not have been due to the different disciplines subject to investigation but to differences in the methods and definitions used by the researchers (Dubois Reference Dubois1997; Lewin, Fine & Young Reference Lewin, Fine and Young2001; Basturkmen Reference Basturkmen2012).
The identification of moves and steps is initially a qualitative form of investigation based on observations of recurrent patterns across text samples. It involves segmenting the genre (text) into units with specific rhetorical functions and suggesting move and step labels for the units. Initial observations are usually followed by quantification in the form of frequency counts to support findings. Categorisation of moves and steps is to a considerable degree based on inference, since the functions are often not overtly signalled lexically. Analysis requires close and repeated scrutiny of the texts, which are often long and complex and contain subject matter with which the researcher is not familiar. As the aim is to understand how text segments interrelate to achieve the overall communicative purpose of the genre, analysis is very time-consuming: software to aid move and step analysis has become available (Flowerdew & Forest Reference Flowerdew and Forest2009; Upton & Connor Reference Upton, Connor, Biber and Reppen2012) but is generally limited to identification of surface lexical forms that sometimes signal the beginning of units (rather than deep, rhetorical structure). In the short term, at least, analysis of moves and steps will probably continue to be done ‘manually’ by researchers. Such factors generally rule out the use of extensive data sets.
The genre-based descriptions that research has provided have been readily applied in second language (L2) writing instruction (Paltridge Reference Paltridge2001; Johns Reference Johns2002) and a number of published textbooks use this approach in teaching writing (Tardy Reference Tardy, Belcher, Johns and Paltridge2011). EAP writing courses in university settings often draw on genre-based descriptions or findings from research. It is generally accepted that ‘differentiating and describing the genres that students need to write is an important step towards explicating the tacit knowledge that the initiated possess’ (Nesi & Gardner Reference Nesi and Gardner2011: 261). But how much confidence should teachers, materials developers and academic supervisors have in the research findings on which teaching suggestions have been based, given that they are drawn from studies which have gone largely unreplicated?
Recently, a number of comparative genre studies have been published. This is not an entirely new strand of investigation, however (see, for example, Holmes Reference Holmes1997). Comparative studies search for variation in a genre across different groups of writers, and use quantitative means to establish differences. Examples include investigations of genre variation across academic disciplines (Samraj Reference Samraj2002, Reference Samraj2005; Koutsantoni Reference Koutsantoni2009; Bruce Reference Bruce2010; Tardy Reference Tardy, Belcher, Johns and Paltridge2011; Basturkmen Reference Basturkmen2012), languages (Sheldon Reference Sheldon2011; Soler-Monreal et al. Reference Soler-Monreal, Carbonell-Olivares and Gil-Salom2011) and levels of expertise (Basturkmen Reference Basturkmen2009). It is generally argued that the variations observed reflect the differing norms and epistemologies of specific disciplines, or differing roles of the writers in the academic discourse community.
The next section reports three comparative genre studies that found differences between the writing of the groups under investigation. Each study was based on the analysis of a relatively limited number of texts and each found differences between the groups of writers it compared. Replication using a second set of texts and following similar text selection criteria, the same methods and move and step categories (labels and definitions) as the original study could – if similar differences are observed – corroborate the findings and strengthen the claims made for variation across disciplines or populations of writers. In other words, replication may verify that the original results were robust, were not an ‘isolated incident’ (Porte Reference Porte and Porte2012: 7) and were not merely the outcome of the original set of texts selected for analysis. If the replication does not corroborate the original findings it can still serve to stimulate further inquiry into why the findings appeared to be different.
3. The original studies and suggested approach to replication
Soler-Monreal et al. (Reference Soler-Monreal, Carbonell-Olivares and Gil-Salom2011), A contrastive study of the rhetorical organisation of English and Spanish Ph.D. thesis introductions
This study, published in English for Specific Purposes, compared Ph.D. thesis introductions in the field of computing. Its findings have particular significance for the understanding of cultural and linguistic variation in student research writing and, as will be explained below, important implications for academic advisors and writing support teachers in non-Anglophone contexts. Computing is a fairly recent field of knowledge which, according to the researchers, was ‘initiated in the Anglo-American scientific community’ (p. 5), and computing curricula in Spanish universities are founded on this tradition. Spanish computing students may adopt Anglo-American patterns of research reporting, but the distinct student researcher role and expectations of the academic computing community in the Spanish context may affect how the Spanish students present their research. The study aimed to identify similarities and differences between the rhetorical strategies used in the two languages. It was motivated by pedagogical concerns about how best to help the students communicate appropriately in their disciplinary and linguistic setting.
The introductory sections of ten computing Ph.D.s submitted to universities in Spain (written in Spanish) and ten to Anglophone institutions in the US, UK, Australia and Canada (written in English) were analysed (the two sets were comparable in terms of their proportions of applied vs. theoretical theses). In the analysis the researchers used a model of moves and steps based on Swales’ (Reference Swales1990) ‘Create a research space’ model of research article introductions. The model comprises three moves: Establishing a territory, Establishing a niche and Occupying the niche.
Findings showed that the Spanish texts were less complex than the English: in total, 50 moves were identified in the Spanish texts and 145 moves in the English texts. The Spanish texts contained Move 1, Establishing a territory (presenting the research field) and Move 3, Occupying the/a niche (describing the research conducted). The English texts contained all three moves, including Move 2. Only some of the Spanish texts included Move 2, Establishing a niche, suggesting, according to the researchers, that the Spanish students felt less pressure than their Anglophone peers to compete for research space. The researchers argued that, for the Spanish students, ‘claiming knowledge and good performance in a field seems to have a higher priority than establishing the existence of a research gap that needs to be filled’ (p. 8). The English texts invariably used Move 2 (a move which enables writers to present their research as original), leading the researchers to argue that Ph.D. student writers in Anglo-American contexts feel a need to compete for research attention and promote their research.
Approach to replication
The finding that the Spanish writers structured their introductions differently from the English writers could be important. This information would be relevant to academic supervisors who advise students on organising sections of their thesis or those who write guides to thesis preparation. The researchers’ explanation that the Spanish students may not perceive the same need to compete for research attention and promote the novelty of their research is plausible, but the basis for this claim of difference and the explanation for it could be strengthened.
There are a number of directions a replication study could take. First, a limited number of texts were examined in the original study and it could be extended with more texts. Although, as explained in Section 2, investigation of an extensive number of texts is often not feasible in initial genre analytical studies, it is possible in a replication study, since the researcher would draw on the groundwork conducted in the original study (in this case, the identification, definition, labelling and illustration of moves and steps). In this initial replication, the only variable being changed is the number of texts, making it close enough to the original for the findings of the original and the replication to be compared. In other words, we would be asking whether analysis of a larger data set (Ph.D. introductions in computing), following the same methods and using the same definitions of units (moves and steps) as the original study, leads to similar findings.
If the results from the replication suggested above verify those of the original, a second replication could explore whether the findings relating to the Spanish group can be generalised to other non-Anglophone groups (such as Ph.D. students in French or Chinese linguistic contexts), thus changing one variable in the original. This replication would test the explanation suggested by researchers in the original study that claiming novelty for Ph.D. research may be a particularly Anglophone phenomenon.
A further possibility for replication is for the researcher to follow the methods of the original study to investigate students’ writing (one Anglophone and one non-Anglophone group) in a different discipline and/or in writing a different research genre (proposal, results section, conclusion). A replication in a different discipline could indicate whether differences in dissertation writing across the two language groups are unique to computing or can be generalised to other disciplines – important information for academic advisors and writing support teachers working with doctoral students in other disciplines in non-Anglophone countries. A replication of a different research genre could indicate whether sections other than introductions differ across the groups; such information could contribute to the body of knowledge on cultural differences in research writing. Researchers should select a discipline in which they have some background knowledge (or access to subject specialists) and a section of the thesis that has already been described in terms of moves and steps in the research literature.
Such replications could (if similar findings were produced) support the generalisation suggested in the original study, namely, that writers of academic genres in Anglophone and non-Anglophone settings may draw on the same set of rhetorical functions (moves and steps) but differ in how much they use a particular element. If the result shows that the two groups do not draw equally on the same set of functions, students in non-Anglophone countries should be advised to treat with caution the rhetorical descriptions found in some published guides to dissertation writing, which are based on research in Anglophone countries. This is a more ‘approximate’ replication than the first and second, since it involves modifying more than one element (the linguistic context and the discipline, or the linguistic context, the discipline and the genre). ‘With every variation that is successfully introduced into the equation the confirmatory power of a replication increases, and what we observe is generalised to a wider application range’ (Porte, personal communication, 16 August 2012).
Lim, J. M.-H. (Reference Lim2010), Commenting on results in applied linguistics and education: A comparative genre-based investigation
This study, published in the Journal of English for Academic Purposes, compares the rhetorical organisation of the results sections in research articles in two related disciplines, applied linguistics and education. Its findings have particular significance for the understanding of disciplinary and research methodology variation in research reporting, and have led to suggestions for teaching research writing. The results of the study, showing how research writing does or does not vary in relation to research methodology, are of great interest. The study looked at how much writers in the two disciplines commented on their results in the results section (it is widely known that writers comment on results in the discussion section), and whether the inclusion of comments was related to the type of research (qualitative, quantitative or mixed-method). The study was motivated by teaching concerns since, as Lim argues, ‘Without a detailed investigation into such disciplinary and methodological differences, instructors and supervisors would find it difficult to inform novice writers about the permissibility and necessity to incorporate comments of different categories in the Results section’ (p. 280). The study set out to show if there was any correlation between the occurrence of comments and discipline or research type, and to describe how comments were expressed linguistically.
Two sets of data were collected: the results sections from 15 research articles in applied linguistics and 15 from education. Each set included articles from three journals and a mix of quantitative, qualitative and mixed-method studies. The approach to move and step analysis developed by Swales (Reference Swales1990, Reference Swales2004) was used and the study provides a detailed explanation of the procedures used and how comments were defined and identified.
Analysis revealed four step options used in making comments: Step 1, Giving reasons for the findings, Step 2, Expressing views on the findings, Step 3, Comparing findings with the literature and Step 4, Making recommendations for future research. Writers in both disciplines were observed to use Steps 1, 2 and 3, but Step 4 (the least-used step) was only used by the applied linguists.
Findings indicated that the applied linguists made significantly more comments than the writers in education: over half of the education texts were devoid of any comments. No significant differences were found in the occurrence of comments in relation to research type (qualitative, quantitative or mixed-methods studies). It was concluded that researchers in education appeared to avoid making comments, and the study made suggestions for further research to find out whether this could be ascribed to ‘influences from practices in hard sciences and education researchers’ greater dependence on certain style manuals’ (p. 291). Based on these findings, Lim suggested ways EAP instructors and academic dissertation supervisors might foster novice writers’ understanding of rhetorical strategies to use when writing up their results section.
Approach to replication
Two directions for replication can be suggested. Both could offer important insights for the understanding of disciplinary variation in research writing, currently a major focus of EAP research (Johns & Swales Reference Johns and Swales2002; Peacock Reference Peacock2002; Hyland Reference Hyland2004, Reference Hyland2009; Bruce Reference Bruce2009, Reference Bruce2010). Findings from the original study indicated a more discursive function for the results section (writers commented on results as well as reporting them) in applied linguistics than education. How should this finding be interpreted?
The first suggestion is for an approximate replication. This would follow Lim's procedures and definitions of steps (the examples of linguistic realisations in the original study will be of great help for the replication researcher at this stage) to investigate results sections in articles in applied linguistics compared to those in a number of other disciplines, including education. If the replication finds more comments in the applied linguistics than the education texts, the original finding would be corroborated, and if the replication finds more comments in applied linguistics than in most other disciplines, the finding from the original study (that commenting in results sections is more prevalent in applied linguistics) can be generalised: comments are more frequent than in education and most other disciplines.
A second suggestion concerns the relationship between research reporting and research type. The original study did not find a relationship between occurrence of comments and type of research, but that result was based on the examination of texts from just two disciplines, and these were related disciplines which ‘may occasionally overlap’ (Lim Reference Lim2010: 282). The finding may thus not be generalisable to other disciplines. A replication study (using the procedures and definitions of the original study) could investigate this relationship across a range of disciplines that typically use qualitative, quantitative and mixed methods. The results would be of interest to a wider audience: researchers in any number of disciplines can be expected to be interested in links between research methodologies and research reporting.
Basturkmen, H. (Reference Basturkmen2009), Commenting on results in published research articles and masters dissertations in language teaching
This study investigated the rhetorical organisation of discussion sections and compared the use of one particular move – commenting on results – in discussions in M.A. dissertations and published research articles in the field of language teaching. Its findings contribute to the understanding of how and why research writing by experts and novices differs. The findings have implications for the use of expert writing as a model for student writing, a topic of interest to teachers of research writing and developers of research writing materials.
The study aimed to describe how expert writers of published research articles structured their arguments about the meaning or significance of their results in the discussion section (through analysis of step/sub-step sequences in comment moves) and to identify differences between expert writers and novices (students writing M.A. dissertations). Data for the study comprised ten M.A. dissertations from the same university department and ten articles from the same journal specialising in language teaching research. The study adopted Swales’ move–step approach to analysis.
The study indicated that the expert writers typically commented on two or three results (or linked results) using one or more of three step options (Explanation of result, Comparing the result with a result in the literature and Evaluating a result). The bulk of commentary in the expert discussions was in the form of sequences in which the writers proffered alternative explanations for the same result. Comparison suggested that the novices used the same overall rhetorical structure and the same step options as the experts to make their comments. However, the dissertation writers commented on over twice as many results (between five and six) and offered significantly fewer alternative explanations. The original study suggested three reasons why the journal article writers, but not the dissertation writers, may have put considerable rhetorical effort into suggesting alternative explanations: to show their awareness of a range of theories, to provide original explanations or to make the research results appear newsworthy.
Approach to replication
This section makes suggestions for approximate and conceptual replication. Research writing courses and materials developed to guide students in writing their dissertations could draw on findings from this study. However, the findings of the original study were flagged as suggestive only since they were based on limited evidence. The number of texts analysed was small and, since the texts were sourced from one journal (Language Teaching Research) and one M.A. programme, may not be representative. The original study was a small scale, one-off study, and if its approximate replication corroborates its findings, its claims would be strengthened and it would provide a more assured basis for informing materials development and academic supervision. The first suggestion for replication is for a study to confirm/disconfirm the original study's findings about how the discussions of expert and novices differed. The data set would be larger (more discussion sections) and sourced from more journals in the same discipline and more M.A. programmes. The original study involved the development of procedures, an initial search for rhetorical functions, the definition of move and step categories and move and step labels. The researcher in the replication would need to analyse more texts but would have the advantage of having ready-made procedures and categories, as developed in the original study.
A second suggestion is to try to apply the move and step categories from the original study to discussion sections in articles in a different discipline, to understand how far the move and step categories developed in the original can be generalised (a focus of research conventionally used in genre studies – see section 2). If the move and step categories can be applied more generally, the researcher, following the original study, could then compare expert and novice writing in the second discipline. Findings could shed light on whether the kinds of expert/novice differences observed in the original study can be generalised to other disciplines.
The final suggestion is for a conceptual replication: that is, following Porte's definition (Reference Porte and Porte2012: 8), one that addresses the same research objective as the original study but uses different methodologies. As mentioned previously, the ESP tradition views a genre as a set of rhetorical choices that acts as a schema for action. In the studies described above, the writers’ rhetorical choices were derived from the analysis of text. Stimulated recall interviews offer an alternative methodology. Sample sections of the discussion section written by the interviewees (experts and students) would be used as prompts, and the interviewees would be asked to report their aims in discussing the result in the way that they did. The interviews would be recorded and analysed for similarities and differences in the rhetorical choices reported by the two groups. Potential researchers should bear in mind that, although stimulated recall may seem to provide a more direct source of information about the writers’ rhetorical choices (compared to inferences drawn by genre researchers observing text regularities), the writers’ reports would be retrospective, and this may or may not accurately reflect decisions made at the time of writing.
4. Conclusion
In conclusion, comparative genre analysis is an area of research interest in which replication studies would be feasible and valuable in consolidating recent research. This paper has described three original comparative studies in the area of academic writing. Descriptions and suggestions provided in the original studies would be of great interest to EAP teachers, materials developers and research supervisors. However, these were relatively small-scale, single studies (as explained above, initial genre studies are often fairly small-scale, due to the nature of the task). This paper has made specific proposals for replicating the original studies with the aim of confirming/disconfirming results, enhancing the credibility of studies in this area and indicating how far its findings can be generalised to different contexts.
Helen Basturkmen is a senior lecturer in the Department of Applied Language Studies and Linguistics at the University of Auckland, New Zealand. Her main research interests are ESP and genre analysis. She has published two books on ESP (Routledge, 2006; Palgrave Macmillan, 2010) and articles in a number of journals, including Journal of English for Academic Purposes, Applied Linguistics and The Modern Language Journal.
