November 2021 – Volume 25, Number 3
Saraswati Dawadi
The Open University, UK
<Saraswati.Dawadiopen.ac.uk>
Abstract
The effect of high-stakes language tests on teaching and learning a language is usually known as washback. The true nature of test washback can be understood only by taking account of the educational, cultural, political and social contexts in which the test operates. Nevertheless, little research has explored the factors that affect the nature of washback. This paper reports on a study that explored the factors that affect the washback of the Secondary Education Examination (SEE) English test on students and parents in Nepal. The data generated through oral diaries (n=72) and semi-structured interviews with students and their parents (n=24) indicate that several factors including economic factors, social prestige associated with the test performance, parents’ educational background, and perceived importance of English affect the nature of a high-stakes tests’ washback. Finally, some of the pressing policy, pedagogical and research implications are discussed.
Keywords: high-stakes, washback, English as a foreign language, language test
The effect of a test on teaching and learning the language is usually known as washback (Saif, 2006; Saville, 2009; Takagi, 2010; Tsagari, 2012). Washback refers to a range of complex, unplanned phenomena which occur around assessment processes and the impact upon learning and teaching. A growing body of research is addressing the nature of washback.
It has now been widely accepted that high-stakes tests have a huge effect on teaching and learning the languages being studied (Alderson & Wall, 1993; Xie, 2013) as the tests are likely to have life changing consequences to students. However, researchers have different opinions regarding the nature of washback. While some researchers (e.g., Alderson &Wall, 1993; Dawadi, 2018; Khaniya, 1990) argue that language tests have direct effects on students’ learning practices, some others (e.g., Shih, 2010; Shohamy, 2007) argue that there may not always be a direct relationship between a test and learning practices as language assessment is a social activity which is connected to a whole set of variables that interact in a society. In other words, language assessment is affected by the socio-cultural practices in the society where the test exists. However, language testing researchers, for quite a long time, addressed only the measurement issues while overlooking the various roles that language tests can play in society. In the past, there was little interest in examining how language tests are used in a society and what sort of consequences tests can have to students’ lives. Shohamy (2007) argues that language tests should be seen as a powerful tool and they are deeply embedded in social, cultural, political and educational arenas.
This paper explores the factors that affect the washback of the Secondary Education Examination (SEE) English test in Nepal, a small developing country in South Asia.The SEE is a national level examination conducted at the end of the 10th year of school education for the children aged 14-16 years (Dawadi, 2020a). The SEE is a high-stakes test in the Nepalese context as it functions as a gateway to higher secondary education and a basic license for most jobs in Nepal (Bhattrai, 2014). Most people in Nepal believe that good performance in the SEE creates more career opportunities and leads to a better life. Therefore, parents, irrespective of their educational and social background, contribute to the development of such psychology in their children’s minds that their children consider the SEE to be everything in their life; “failure in the exam equating to failure and meaninglessness in life” (Bhattrai, 2014, p.70). However, it is usually argued that the examination brings severe consequences to students and their parents (Shrestha, 2018). Nevertheless, almost no known research has explored how the test affects students and parents, with a particular focus on the washback nature of the test. Only three studies (Dawadi, 2018, 2020b; Khaniya, 1990) have explored the washback of the SEE (previously known as the School Leaving Certificate) English test but all of them focused only on the relationship between the test and students’ English language learning practices. None of them reflected on the social and cultural factors that might affect the nature of the test washback.Thus, the main rationale for conducting this study starts with the need to reflect on the factors associated with the SEE English test washback in Nepal.
This paper aims to contribute to the literature on language testing by reflecting on the factors that affect washback of a high-stakes test. A second potential contribution comprises the research methodology which could be employed by future researchers. In terms of application, the research reported here can contribute to better informing schools and teachers about the nature of test washback, particularly in the Nepalese context.
Theoretical Orientation
The idea that tests have effects over education in general, and individual in particular, was first vaguely used in the 1960s but became better established in the 1990s (Loumbourdi, 2014). Alderson and Wall (1993), having discussed the term ‘washback’ in their oft-cited article ‘Does washback exist?’, stressed that washback is a very complex concept and it has not yet been well established. The article drew the attention of language testers and researchers in such a way that they very quickly began exploring this topic and developed a debate on test washback.
At present, there is an extensive body of research that not only confirms washback does exist but also suggests that washback is a complex and multi-faceted phenomenon. Nevertheless, it is hard to tell exactly what washback looks like as there may not always be a linear relationship between tests and the teaching/learning processes (Shih, 2010).
Highlighting the complex mechanisms through which washback occurs in a real context of teaching and learning the language, Hughes (1993) introduced a concept of the trichotomy and argued for distinguishing between participants, processes and products. In his framework, participants refer to the people such as students, teachers, parents, administrators, material developers and publishers who are directly or indirectly affected by the nature of a test. Thus, Hughes did not limit washback to teachers and learners. The term ‘processes’ refers to any actions that participants or students take for the sake of learning the language (i.e., the language learning processes or strategies that students employ to prepare themselves for the test), such as memorising answers, and seeking help from other people including their parents) whereas the term ‘product’ refers to learning outcomes (i.e., students’ performances on the test). Hughes develops discussion of backwash as follows:
The trichotomy into participants, process and product allows us to construct a basic model of backwash. The nature of a test may first affect the perceptions and attitudes of the participants towards their teaching and learning tasks. These perceptions and attitudes in turn may affect what the participants do in carrying out their work (process), including practising the kind of items that are to be found in the test, which will affect the learning outcomes, the product of the work (p. 2).
In order to illustrate the mechanism of washback as a trigger for change, Bailey (1996) developed a washback model based on Hughes’ (1993) tripartite distinction between participants, processes and products. Bailey’s model suggests that there is a two-way hypothesis; the test itself might be reciprocally affected by its participants. There is no direct effect of a test on learning the language, rather washback can be seen as a process and the process is not always linear. The model does not limit washback to learning the language and suggests that involvement of participants (including students, parents, teachers, researchers, material writers and curriculum designers) make the washback mechanism intricate and comprehensive.
However, Bailey’s washback model is also not comprehensive as it does not reflect on social, cultural and political factors that might affect the washback nature of a test. Having considered the roles of external factors that might affect the nature of washback, Shih (2007) puts forward a washback model. The model indicates that there is no direct relationship between a test and learning the language as several other factors affect the washback nature of a high-stakes test.
It has also been argued that students’ future aspirations and language proficiency and social prestige associated with the language and students’ performances on the test may affect the nature of test washback (Dawadi, 2018, 2020b). Murray, Riazi and Cross (2012) further argue that students’ attitudes towards the test fairness may have impact on students’ learning practices. For instance, having a positive attitude towards a test may encourage students to try their best to become more effective learners.
Similarly, the role of student motivation in learning has been highlighted. There are different views regarding test motivation. For instance, Hsu (2009) argues that students in high-stakes test contexts are less likely to hold positive attitudes towards learning English as the performances in those tests are likely to determine their career or lives. Tsai and Tsou (2009) also claim that high-stakes tests lead to a decrease in motivation to learn communicative English as classes are test-oriented; thus, enhance only test-taking skills instead of developing communicative competence. Nonetheless, Abu-Alhija (2007) opines that large-scale tests in some contexts may motivate students to work harder and more effectively. Some previous washback studies (e.g., Dawadi, 2018, 2020b; Shohamy, Dontisa-Schmidt, & Ferman, 1996) further claim that high-stakes English tests are a strong instrumental motivation for students to learn English.
In summary, as Alderson and Wall (1993) state, washback is more complex than assumed; both good and bad tests may have both beneficial washback (e.g., more motivation, more learning activity) and negative washback (e.g., learning to the test, more learner anxiety, the fear of poor results and the associated guilt, shame, or embarrassment). Nevertheless, the degree of the effects may vary from person to person or from context to context. Furthermore, several factors, including social and personal factors, affect the nature of a test washback. The current study is mainly guided by Shih’s (2010) framework as the framework provides guidelines to explore how social factors may affect the washback nature of the test. Indeed, the framework focuses on three main factors to reflect on the nature of a test washback: social and educational factors (e.g., perceived value of the test, program evaluation, and job market), school factors (e.g., reputation of the school, administrative concerns and school’s support), and parental and student factors (e.g., reactions and rights of parents and students, students’ English proficiency, and students’ financial burden). Following the framework, the current study also considered social and economic factors (i.e., whether parents have economic burden associated with the test) to reflect on the washback nature of the SEE English test.
Empirical Background to the Study
Existing studies on language assessment, particularly in high-stakes testing contexts (e.g., Alderson &Wall, 1993; Dawadi, 2020b; Shih, 2007), have evidenced that the nature of test washback is complex because multiple stakeholders and multiple factors co-exist within the social context where a test exists. Indeed, complex interactions among multiple factors, such as educational and socio-cultural factors, that exist in the society largely determine the nature of washback getting into the classroom (Matoush & Fu, 2012; Shih, 2010; Xie, 2015; Zhan & Andrews, 2014). Hence, it is difficult to establish whether it is the test itself that triggers changes in students’ learning practices, or it is other social or educational factors including the test’s use in the society. Alderson and Wall (1993) rightly point out that other forces exist within society and schools that may also affect the nature of test effect. This means that the nature of washback might be independent of the quality of the test. If a test is designed to affect students’ learning practices in a certain way, this may not automatically affect the language learning practices in the desired way. In other words, simply changing test tasks and content do not necessarily bring about desirable changes in students’ learning practices.
The existence of a test by itself does not seem to have any kind of washback; the amount and type of washback seems to be affected by different factors, such as the level of importance of the test, the extent to which teachers and textbook writers are prepared to innovate, the extent to which teachers think about test preparation (Alderson & Hamp-Lyons, 1996), students’ and parents’ attitudes toward the test (Dawadi, 2019), student motivation (Dawadi, 2020b) and parental involvement or support in the test preparation, particularly in the context of Nepal (Dawadi, 2020a). Messick (1996, p.242) points out, “a poor test may be associated with positive effects and a good test with negative effects because of other things (such as perceived importance of the test and student motivation to learn) that are done or not done in the educational system.” Therefore, it might be plausible to argue that washback is caused by the testing regime rather than the quality of a test (Saville, 2009).
Allen’s (2016) study on the International English Language Testing System (IELTS) test preparation practices in Japan indicated a range of mediating factors that shaped washback to the learners such as perceived importance of the test, test-takers’ interest and learning environment. Learners’ perceptions of the status and stakes of tests seem to affect students’ learning practices; students spend more time developing language skills tested by the high-status or high-stakes tests than they do on lower-status or lower-stakes tests (Shohamy, et al., 1996). Huhta, et al. (2006) further reported that expectations of success seemed to affect the rate and amount of test preparation practices. In addition, parents’ economic status affects students’ learning practices; “high-quality coaching is available only to those who can afford it” (Ingulsrud, 1994, p.72) and teachers’ teaching techniques. Teachers in Sri Lanka used the time-consuming but widespread practice of having students copy test related texts from the chalkboard as the books were too costly for parents (Wall & Alderson, 1993).
Similarly, it has been argued that students’ motivation to learn a language and/or perform well on its test also affects the nature of washback. The term ‘motivation’ is usually defined as a person’s desire or drive to perform a particular task. In this paper, it refers to a student’s desire or drive to learn English and/or perform well on the SEE test. Motivation is usually considered to be one of the most important determinants of students’ success in high-stakes tests (Dawadi, 2018). In a high-stakes test context, “motivation to succeed is not static but occurs along a continuum. In particular, motivation varies according to the complex interaction of test-takers and test contexts based on both the intended and unintended test use” (Cheng, et al., 2014, p.306). Nevertheless, it is usually believed that tests provide incentives to test-takers to improve their performance.
It is worth pointing out that there are different views regarding test motivation. For instance, Hsu (2009) argues that students in high-stakes test contexts are less likely to hold positive attitudes towards learning English as the performances in those tests are likely to determine their career or lives. Tsai and Tsou (2009) also claim that high-stakes tests lead to a decrease in motivation to learn English as classes are test-oriented; thus, enhance only test-taking skills instead of developing communicative competence. Nonetheless, Abu-Alhija (2007) opines that large-scale tests in some contexts may motivate students to work harder and more effectively. Some previous washback studies (e.g. Dawadi, 2018, 2020b; Shohamy, et al., 1996) further claim that high-stakes English tests are a strong instrumental motivation for students to learn English.
The study by Shih (2007) on the effects of General English Proficiency Test on English learning in Taiwan suggests that washback is inextricably linked to the social and educational contexts in which a test is administered, and washback varies from person to person. Shih argues that test washback is intertwined with internal factors (individual differences, personal characteristics and personal perceptions of the test), external factors (socio-economic factors, school and educational factors, family, friends and colleagues) and test factors. Gipps (1999) also argues, “the way students respond to assessment is subject to social and cultural affects (p.355).”
It is important to consider how tests are used in the society where they exist (Ali, et al., 2020). Given the power of high-stakes tests, it is likely that they have a substantial economic impact on families and support networks. The high economic value placed on tests encourages students and their families to go to great lengths to bear the fees and other expenses involved in preparing for and taking the tests (Moore, et al., 2012).
However, almost no known research has explored the issues in the Nepalese context. The literature on assessment indicates that only three studies (Dawadi, 2018, 2020b; Khaniya, 1990) have explored the washback of the SEE (previously known as the School Leaving Certificate Examination) English test but all of them were guided by Alderson and Wall’s (1993) hypotheses as their focus was only on the linear relationship between the test and teaching/learning practices. None of them reflected on the social and cultural factors that might affect the nature of the test washback. Therefore, this study aimed to answer the research question: “What factors affect the washback of the SEE English test in the Nepalese context?”
The Study
Participants
The context for the study is the Secondary Education Examination in Nepal which is conducted at the end of 10-year school education. Participants in this study comprised English as a foreign language (EFL) learners (n=6) and their parents (one parent for each) in Nepal. The students were studying at Grade 10 in public schools. The students were purposively selected for the study ensure that they were representative of gender and ethnic group. All the students had been learning English as a foreign language for 10 years and their age ranged between 14 and 16 years old. All of them were Nepali native speakers and they were from rural Nepal. Among the six parents, three were highly educated (at least SEE/SLC passed), and the rest were illiterate. Their age ranged between 35 to 62 years, and they had different professions: teacher (n=2), farmer (n=2), shopkeeper (n=1) and stonebreaker (n=1).
Data Collection Procedures
The study employed a qualitative approach in order to answer the research question with sufficient depth. Each student was provided with an audio recording device and they were asked to audio-record oral diaries once a week for three months (first month during the usual classes, second during the test preparation time, i.e. around six weeks before the test, and the final one during the test results publication). This means, each of the students recorded 12 diaries and each of them, along with their parents (one parent for each), were also interviewed twice: around six weeks before and around two months after the test. Thus, the study collected 72 diary entries and 24 interviews. Each of the interviews lasted for approximately 45 minutes and diary entries for 5 minutes.
It is worth pointing out that the students in this study were provided with some guidelines to record their diaries, but they were encouraged to express their opinions freely and include as much information as they could about their English language learning practices and the test. During the interviews, some prompts were used to ensure that the required information was collected.
Students were allowed to use the language of their choice while recording their diaries. All the diaries were recorded in Nepali language and the same language was used during the interviews in order to ensure that there is no language barrier to their participation. The researcher herself (a native of Nepal) conducted the interviews. The major focus of both diaries and interviews was mainly on how students learnt English to prepare themselves for the test and how the test affected them. Having used Shih’s (2010) washback framework, the interviews included some discussions around socio-economic factors associated with the test (e.g., social pressure on students and parents, and economic burden associated with the test). It is also worth noting that most of the questions asked to students and parents during the interviews were similar.
The project was carried out in accordance with the British Educational Research Association ethical guidelines (BERA, 2018). Prior to collecting any sort of primary data for this research, informed consent was obtained from all the participants and their parents/carers. Each participant was informed that their participation would be voluntary and that all data would be treated as confidential and anonymised promptly. In order to obtain informed consent, both children and their parents were given a full account of the project.
Data Analysis
A thematic analysis approach was employed and NVivo 10 software was used to systematically organise and enable/identify/develop themes emerging through the analysis. An iterative process was undertaken following analysis stages framed by Braun and Clarke (2006), namely familiarizing with the data, generating initial codes, searching for main themes, and reviewing themes. During the analysis process, a careful attention was paid to the theoretical framework and possible factors that could affect nature of the SEE English test in the Nepalese context.
In order to maximise the overall depth of the analysis, both deductive and inductive approaches of thematic analysis were utilised in this study (Dawadi, 2020c). A deductive approach was used as the starting point which allowed analysing the data in relation to the findings or the test related themes that had emerged through the review of literature done for this study and Shih’s (2010) washback framework, such as economic factors associated with the test, students’ test preparation practices, test pressure on students and student motivation. However, each of the interesting or relevant information (themes) emerged through the data was also considered. Even the unexpected themes were taken into consideration for better understanding of the phenomenon in question. Therefore, a large number of inductive codes emerged when analysing the data.
Each case study student, in this paper, is represented with the alphabet ‘S’ and their parent with ‘P’, followed by a number 1 to 6 to ensure confidentiality in the research. The same number is used to represent a student and their parent i.e. P1 and S1 are parent and child 1 pairing, P2 and S2 the parent and child 2 pairing etc.
Findings and Discussions
The findings of the study indicated that there was no direct relationship between the SEE English test and students’ test preparation practices. However, there were several indications that the SEE was connected to a whole set of other variables associated with Nepalese society. Some of these factors have been briefly discussed below.
English language teaching practices in Nepal
The findings indicate that the SEE English test encouraged students to prioritise memorisation over language skills development. “I have already memorised some answers. I have decided to memorise as many answers as possible for the test (S-5); I have memorised so many things such as short question answers, long question answers, essays, letters and dialogues (S2).
SEE students employed memorisation as one of the main strategies to prepare themselves for the test, and they spent significant time memorising answers. However, this particular practice might be an outcome of the historical and sociocultural settings in Nepal. Indeed, English language teaching in Nepal has long been characterised by a major focus on drilling, note providing, translation, memorisation and rote learning of vocabulary, preoccupation with examinations of grammatical structures and lack of attention to communicative skills, critical thinking (Kandel, 2014) and creativity. It is plausible to argue that poor teaching and learning practices associated with the SEE might have resulted from a pervasive culture within the ELT profession in Nepal rather than the content and format of the SEE English test itself. Memorisation was not therefore an aspect/element/part of SEE English test washback.
Perceived importance of English and its social prestige
This section will set out specific factors about Nepalese society which contribute to washback. There were several indications that students were motivated to learn English. However, it was not clear whether students were motivated by the test or by other (social) factors such as the perceived importance of the language for their career progression.
All the case study students and parents considered English language competence to be important for their future career:
I am very much motivated to learn English these days as I know that English is one of the most important languages in the world […] we need it for different purposes such as travelling to other countries, finding good jobs and communicating with other people (S6).
If you have a very good English, you will be respected in your society (P1).
There has been a high demand of English language in the Nepalese society as knowledge of English is expected to provide better career prospects and choices. Every educated person in Nepal is expected to have learnt some English and those who have good English are thought to be knowledgeable people. English has become the most prestigious language in Nepal. Indeed, the English language has been associated with social prestige, and more people in Nepal seem to be interested in learning English every single day. Dahal (2000, p.176) points out a number of factors that have contributed to the gravity of English in the lives of Nepalese people including Nepal’s diplomatic relationship with several countries, its heavy dependence on foreign aid and job opportunities.
Support from neighbours and peers
Support from neighbours and peers motivated students to work for their assessment and that therefore the social support of students’ community members is a feature of washback in Nepalese society. There were several indications that social factors had a huge impact on students’ motivation to learn English and to prepare themselves for the SEE English test. For instance, parents reported that their children were motivated to work hard for the test because of their neighbour’s support:
Another reason of her motivation could be the support she is getting from our neighbour, who is an English teacher in a school. He is so kind and supportive that he frequently helps her to learn English (P1).
Students’ diary entries indicated that students took a lot of support from their relatives and neighbours. None of the previous studies have reported this kind of support in children’s study. One of the tentative explanations for this finding would be that, contrary to many other countries, Nepalese societies have a good bonding among the people living there. As they are living in strong harmony, it seems quite common for them to take or give help in their community.
A number of other factors were also influential in affecting students’ learning practices. For instance, the students reported that they did not practise speaking as their friends were not interested in speaking English: “I never asked questions to my friends as they were also not good at English” (S3).
These findings are inconsistent with the findings reported by Allen (2016) that EFL students in Japan seek advice from their peers and also share their beliefs, the SEE candidates made very little use of friends as a source of knowledge in the test preparation process. This finding might suggest that the Nepalese students do not usually value friend’s knowledge for learning. One of the tentative explanations for this finding would be associated with their classroom practices in the Nepalese schools which usually promote rote memorisation rather than communication. Despite the fact that “students generally enjoy group work more than individual work” (Life, 2011, p.27), most teachers in Nepal do not usually conduct pair or group work in their classroom. So, students rarely get an opportunity to share their learning experiences with their friends. For instance, Bashyal’s (2018) exploration on the ELT practices in the secondary schools of Nepal indicated only a few examples of role play and group and pair work; teachers “allowed very little time for student talking, there is almost 80% teacher talking time in the classroom” (p.227).
Economic factors
The findings indicate that students used very few language development strategies, such as watching television programmes or movies in English, reading newspapers in English and writing in English. There were several indications that low level of use of these strategies resulted from the lack of money and resources to invest in them. Furthermore, the findings suggest that students’ choice of strategy was affected by their parents’ pressure:
We also bought some CDs and DVDs and allowed her to watch some English movies and some other videos in English during the first few months at Grade 10. However, we have not allowed her to watch those videos these days as the examination is very close and we think that she needs to focus only on reading and writing now (P1).
We do not have a television at home. So, I cannot watch programmes in English (S4).
I have asked him to make notes of all the important answers and then memorise them (P3).
The study suggests that there are important financial implications associated with the test washback. The parents (P1, P3), who were able to afford money for buying a lot of learning resources for their children constantly motivated their children to learn English even in the post-test context. However, the parents (P2, P4, P5), who were not able to afford expensive learning materials did not seem to encourage their children to learn English in the post-test context. Indeed, there were individual differences between the parents. For instance, S1’s parents provided her with some CDs and DVDs in English and bought some magazines or newspapers in English from the city. They also bought so many learning materials for their child while P2 could not provide all those things to her child except some learning materials and the textbook; they did not even have a television at home. Consequently, students’ learning motivation and English language learning practices were affected. For instance, the students who had better access to resources, such as CDs, DVDs and television, seemed to be more interested in developing their listening and speaking skills than the students who had less or no access to the resources:
I am very much interested in improving my listening and speaking skills in English; so I keep on watching DVDs in English (S1).
I have so many materials to read and practise for the test […] I am very much interested in learning English and I am trying hard to prepare myself for the test (S3).
I focus more on reading and writing. I do not have any material to practise listening and speaking in English. Actually, I do not have extra materials to practise my reading and writing as well […] English is the most difficult subject for me. I am not interested in learning English these days. (S4).
Similar findings have been reported by Ferman (2004) and Choi (2008) that there are financial implications associated with high-stakes tests. According to the authors, high-stakes tests in Korea have made parents spend an enormous amount of money on their children’s learning as parents in Korea are extremely sensitive to their children’s performances on high-stakes tests. In other words, parents’ support to students may influence students’ learning practices.
Parents’ educational background and encouragement
The findings suggested that, in the pre-test context, all the six parents tended to motivate their children to work hard for the test: “I always try my best to motivate my children to learn English as it is important for them (p.6); “My father always encourages me to learn English and work hard for the test”(S1); I simply motivate my daughter to improve her English speaking skills (p1). It can be assumed that parents’ encouragement to learn English might have affected their learning practices. In the context of Nepal, as Dawadi (2020a) points out, parents are heavily engaged in students’ learning practices. This means, students’ learning practices are influenced by their family environment.
The findings in the study also indicated that the parents with high education supported their children to develop their listening and reading abilities while it was beyond the capacity of the parents with low education. As a result of this, some individual differences among the students could be seen in terms of their learning practices. This implicates family members as an essential resource that can mediate the washback of an examination that involve productive skills (Allen, 2016). In other words, the child’s preparation for assessment involves the active support of family members and that family members’ English language competence mediates the child’s likely performance in the assessment.
Parents’ and teachers’ expectations from students
Students’ learning practices were also influenced by their teachers’ and parents’ expectations (i.e., My teachers are expecting me to do well on the test as I usually get the highest scores in my class tests– S1; My parents have invested a lot of money on my education, and they are expecting me to do well on the test. Therefore, I am working hard for the test– S5). There were several indications that expectations from teachers and parents created some sort of pressure on students to perform well on the test (e.g., Both my teachers and parents are hoping that I will perform well on the test. Therefore, I am very much anxious about the test and I am working very hard for the test– S6). Therefore, students seemed to bring changes in their learning practices.
Consistent with these findings, the literature on language testing indicates that students in a high-stakes test context are usually forced to work hard for the test by their family members and teachers. For instance, the Greek students in Loumbourdi’s (2014) study and the American students in Mulvenon, Stegman and Ritter’s (2005) study stated that they suffered much pressure from parents and teachers to perform well on high-stakes tests. Kirkpatrick and Zang (2011) further reported that the Chinese students seemed to feel moderate pressure to excel in school. Parents’ and teachers’ expectations of students in their assessment performance is a clear feature of washback in the Nepalese context.
Students’ perceived importance of the test and social prestige associated with the test performance
Students’ learning was affected by the perceived value of the test and the need to do well to achieve their future aspirations. Furthermore, students’ motivation to learn the language seemed to be affected by their perceived social prestige associated with the test performance:
I am very much worried about the test as it is very important for my future. If I cannot do well on the test, I will not be allowed to study science at Grade 11. I think, my future will be dark (S3).
My parents will be respected more in our society, if I can perform well on the test(S6).
If she performs poorly on the test, it might be shameful for us (P1).
They were willing to invest energy and effort as most of them seemed to consider the test to be very important. Indeed, parents emphasise the importance of the assessment so much that children considered the SEE to be everything in their life: “failure in the exam equating to failure and meaninglessness in life” (Bhattrai, 2014, p.70).Within Nepalese society, poor performance on the test is seen as a failure in an individual’s life. Most students consider the SEE to be a landmark in an individual’s life as they believe that the examination provides the ladder for one to get higher education and opens up the vista of developing his/her career (Bhattrai, 2014).
The social prestige attached to good performance in the SEE is also encouraged by schools. Individual students’ performances are publicised by most schools;(in some cases individual students’ photographs along with their Grades in the SEE are even displayed on the school walls and in different public places. Schools hope to attract students for admission by displaying the high grades achieved by their students. This demonstrates that the achievement of high scores/grades on the SEE English test receives high prestige. However, the manner in which high grades can be attained e.g. through rote learning in the Nepalese society suggests this is without necessarily considering the knowledge and skills the student has in the related subject.
To reiterate, the study indicates that there is no linear relationship between a test and students’ learning practices as several factors (such as parents’ education background, perceived importance of the test and the language, social prestige associated with the test performance and economic factors) are more influential than the test structure itself. Therefore, it sounds meaningful to end this discussion with Messick’s (1996) argument: “A poor test may be associated with positive effects and a good test with negative effects because of other things that are done or not done in the educational system” (p. 242). Thus, this study provides further evidence to the claim made by Shih (2007) that test washback is linked to the educational and social contexts in which the test is administered.
Limitations of the Study
The findings of the study cannot be generalized as this study was limited to 6 students and their parents. This study could be extended to a greater number of participants so that the findings could be generalized. Additionally, this study lacks classroom data and does not capture teachers’ (one of the primary stakeholders of the test) voices. In other words, it would have benefited from additional classroom data collected through observation and teacher interviews. Furthermore, it would produce a more comprehensive picture of the test washback operating therein, if the study was extended to other stakeholders of the test including teachers, teacher trainers, policy makers and test designers. However, this study can be utilized as a baseline, and thus provides guidelines to pertinent future studies.
Conclusions and Implications of the Study
The findings of the study indicate that there is not always a linear relationship between a test and teaching/learning practices; involvement of several factors make the washback mechanism intricate and comprehensive. Therefore, it is essential to study the social, cultural and political aspects of the society to reflect on the true nature of washback.
The findings of this study indicate students’ overdependency on memorisation for learning English and there were several indications that students were influenced by the classroom practices. Therefore, there is a need to focus on English language skills in classroom teaching, and teachers need to encourage students to develop their English language skills. However, there might be a need for training or professional development opportunities for teachers to support them introduce changes to their pedagogical practices.
This study also has important implications for testing researchers. They might be interested in the research methodology used in this study. By bringing parents’ and students’ views together, this study produced a comprehensive picture of the factors that affect the nature of washback. Furthermore, the study has some theoretical implications. The findings of the study indicate that several socio-cultural and economic factors affect the nature of the test washback.
The study suggests avenues for further research in the field of language testing to verify the findings of the study and to explore the factors that affect the nature of washback in greater depth. Previous researchers have highlighted the need for multiple research studies on the same test to produce a more comprehensive picture. For instance, Cheng (2008) argues, “It would be the best use of resources if a group of researchers could work collaboratively and cooperatively to carry out a series of studies around the same test within the same educational context” (p.360). In the first instance, what is lacking in this particular context is that this study could not collect and benefit from classroom observation data that would add more insights and make the evidence of the test impact clear.
About the author
Saraswati Dawadi is currently a research associate at the Institute of Educational Technology, The Open University, UK. Her research interests include language assessment, girls’ empowerment, inclusion in education and professional development.
To cite this article
Dawadi, S. (2021). Factors affecting washback of a high-stakes English as a foreign language test. Teaching English as a Second Language Electronic Journal (TESL-EJ), 25(3). https://tesl-ej.org/pdf/ej99/a1.pdf
References
Alderson, C. J., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Language Testing, 13(3), 280–297.
Alderson, C. J., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14 (1), 115–129. https://doi.org/10.1093/applin/14.2.115
Abu-Alhija, F. N. (2007). Large-scale testing: Benefits and pitfalls. Studies in Educational Evaluation, 33(1), 50–68. https://doi.org/10.1016/j.stueduc.2007.01.005
Ali, M. M., Hamid, M. O. & Hardy, I. (2020). Ritualisation of testing: problematising high-stakes English-language testing in Bangladesh. Compare: A Journal of Comparative and International Education, 50(4), 533-553. https://doi.org/10.1080/03057925.2018.1535890
Allen, D. (2016). Investigating washback to the learner from the IELTS test in the Japanese tertiary context. Language Testing in Asia, 6 (7), 1–20. https://doi.org/10.1186/s40468-016-0030-z
Bailey, K. M. (1996). Working for washback- a review of the washback concept in language testing. Language Testing, 13(3), 257–277. https://doi.org/10.1177/026553229601300303
Bashyal, G. P. (2018). ELT methods and techniques in Nepali secondary school. In D. Hayes (Ed.), English language teaching in Nepal: Research, reflection and practice (pp.219-238). Kathmandu: British Council, Nepal.
Bhattrai, Y. B. (2014). The school leaving certificate (SLC) examination of Nepal: Exploring negative consequences. An unpublished MA dissertation, Carleton University, Canada.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. http://dx.doi.org/10.1191/1478088706qp063oa
Cheng, L. (2008). The key to success: English language teaching in China. Language Testing, 25(1), 15-37. https://doi.org/10.1177/0265532207083743
Choi, I. (2008). The impact of EFL testing on EFL education in Korea. Language Testing, 25(1), 39–62. https://doi.org/10.1177/0265532207083744
Dahal, R. K. (2000). Language politics in Nepal. Contributions to Nepalese Studies, 27(2), 155–190. Retrieved from: http://himalaya.socanth.cam.ac.uk/collections/journals/contributions/pdf/CNAS_27_02_02.pdf
Dawadi, S. (2018). The impact of the SLC examination on English language teaching and student motivation to learn English. In D. Hayes (Ed.), English language teaching in Nepal: Research reflection and practice (133-163).British Council.
Dawadi, S. (2019). Students’ and parents’ attitudes towards the SEE English test. Journal of NELTA, 1-2, 1-16. https://doi.org/10.3126/nelta.v24i1-2.27677
Dawadi, S. (2020a). Parental involvement in national EFL test preparation. RELC, 51(3), 427–439. https://doi.org/10.1177/0033688219848770
Dawadi, S. (2020b). High-stakes test impact on student motivation to learn. European Journal of Educational and Social Sciences 5(2), 59 – 71.
Dawadi, S. (2020c). Thematic analysis approach: A step by step guide for ELT research practitioners. Journal of NELTA, 25 (1-2), 62-71.
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In L. Cheng, Y. Watanbe, & A. Curtis (Eds.), Washback in language testing: Research context and methods (pp. 191–210). Lawrence Erlbaum Associates.
Gipps, C. (1999). Socio-cultural aspects of assessment. Review of Research in Education, 24(1), 355–392. https://doi.org/10.3102/0091732X024001355
Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript. University of Reading, England.
Hughes, A. (2003). Testing for language teachers. Cambridge University Press.
Huhta, A., Kalaja, P., & Pitkänen-Huhta, A. (2006). Discursive construction of a high-stakes test: The many faces of a test-taker. Language Testing, 23(3), 326–350. https://doi.org/10.1191/0265532206lt331oa
Hsu, H. (2009). The impact of implementing English proficiency tests as a graduation requirement at Taiwanese universities of technology. An unpublished PhD thesis, University of York, England.
Ingulsrud, J. E. (1994). An entrance test to Japanese universities: Social and historical contexts. In C. Hill & K. Parry (Eds.), From testing to assessment: English as an international language (pp. 61–81). Longman.
Kandel, R.J. (2014). Resolving ELT challenges through critical thinking strategies. Journal of NELTA Surkhet, 4, 1-12. https://doi.org/10.3126/jns.v4i0.12854
Khaniya, T. R. (1990). Examinations as instruments for educational change: Investigating the washback effect of the Nepalese English exams. An unpublished PhD thesis, Edinburgh University, Edinburgh.
Kirkpatrick, R., & Zang, Y. (2011). The negative influences of exam-oriented education on Chinese high school students: Backwash from classroom to child. Language Testing in Asia, 1(3), 36–45. https://doi.org/10.1186/2229-0443-1-3-36
Life, J. (2011). Motivation and EFL university students in North-East Asia. Asian EFL Journal, 13(2), 11–41. Retrieved from: https://www.asian-efl-journal.com/main-editions-new/motivation-and-efl-university-students-in-north-east-asia/
Loumbourdi, L. (2014). The power and impact of standardised test: Investigating the washback of language exams in Greece. Peter Lang.
Matoush, M. M., & Fu, D. (2012). Tests of English Language as significant thresholds for college-bound Chinese and the washback of test-preparation. Studies in Culture and Education, 19(1), 111–121. https://doi.org/10.1080/1358684X.2012.649176
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256. https://doi.org/10.1177/026553229601300302
Moore, S., Stroupe, R., & Mahony, P. (2012). Perceptions of IELTS in Cambodia: A case study of test impact in a small developing country. IELTS Research Report Volume 13, 1–109. Retrieved from: https://www.ielts.org/-/media/research-reports/ielts_rr_volume13_report6.ashx
Mulvenon, S. W., Stegman, C. E., & Ritter, G. (2005). Test anxiety: A multifaceted study on the perceptions of teachers, principals, counselors, students, and parents. International Journal of Testing, 5(1), 37–61. http://dx.doi.org/10.1207/s15327574ijt0501_4
Murray, J. C., Riazi, A. M., & Cross, J. L. (2012). Test candidates’ attitudes and their relationship to demographic and experiential variables: The case of overseas trained teachers in NSW, Australia. Language Testing, 29(4), 577–595. http://dx.doi.org/10.1177/0265532212440690
Prodromou, L. (1995). The backwash effect: from testing to teaching. ELT Journal, 49(1), 13–25. https://doi.org/10.1093/elt/49.1.13
Saif, S. (2006). Aiming for positive washback: a case study of international teaching assistants. Language Testing, 23(1), 1–34. http://dx.doi.org/10.1191/0265532206lt322oa
Saville, N. D. (2009). Developing a model for investigating the impact of language assessment within educational contexts by a public examination provider. An unpublished PhD thesis, University of Bedfordshire, England.
Shih, C. M. (2007). A new washback model of students learning. The Canadian Language Review, 64(1), 135-161. http://dx.doi.org/10.1080/15434301003664196
Shih, C. M. (2010). The washback of the general English proficiency test on university policies: A Taiwan case study. Language Assessment Quarterly, 7(3), 234–254.
Shohamy, E. (1993). The power of tests: The impact of language tests on teaching and learning. NFLC occasional paper. NFLC Occasional Papers.
Shohamy, E. (2001). The power of tests: A critical perspective on the uses of language tests. Longman.
Shohamy, E. (2007). Language tests as language policy tools. Assessment in Education: Principles, Policy & Practice, 14(1), 117–130. http://dx.doi.org/10.1080/09695940701272948
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language Testing, 1 (3), 298–317. https://doi.org/10.1177/026553229601300305
Takagi, A. (2010). A critical analysis of English language entrance examinations at Japanese Universities. An unpublished PhD thesis, University of Exter, England.
Tsagari, D. (2012). FCE-exam preparation discourses: insights form an ethnographic study. ESOL Research notes 47. Retrieved from http://www.cambridgeenglish.org/images/22669-rv-research-notes-47.pdf
Tsai, Y., & Tsou, C. H. (2009). A standardised English language proficiency test as the graduation benchmark: Student perspectives on its application in higher education. Assessment in Education: Principles, Policy & Practice, 16(3), 319–330. https://doi.org/10.1080/09695940903319711
Xie, Q. (2013). Does test preparation work? Implications for score validity. Language Assessment Quarterly, 10(2), 196–218. http://dx.doi.org/10.1080/15434303.2012.721423
Xie, Q. (2015). Do component weighting and testing method affect time management and approaches to test preparation? A study on the washback mechanism. System 50, 56-68. http://dx.doi.org/10.1016/j.system.2015.03.002
Zhan, Y., & Andrews, S. (2014). Washback effects from a high-stakes examination on out-of-class English learning: insights from possible self-theories. Assessment in Education: Principles, Policy & Practice, 21(1), 71–89. http://dx.doi.org/10.1080/0969594X.2012.757546
Copyright rests with authors. Please cite TESL-EJ appropriately. Editor’s Note: The HTML version contains no page numbers. Please use the PDF version of this article for citations. |