Over the past two and a half decades, numerous studies have detailed the role of learners’ strategies in a variety of L2 domains. However, research on strategies in the process of test-taking, TTS, has been rather scant, though such strategies might influence test scores and consequently test validity. That has motivated this work to explore TTS and their impact on test construct validity. To accomplish its objectives, this research operated within the sequence framework hypothesised by the PPP model, in which a number of variables represent Presage factors, which are likely to influence the test Process embodied in the TTS. These variables, in turn, may affect the test Product, i.e. the test scores obtained.
A process-oriented methodology via triangulated verbal data collection was exploited to gain insight into test-takers' mental processes when tackling an EFL multiple-choice vocabulary test. To collect the data required, eight different instruments were utilised with English major university students in two English departments at KSU and UQU in Saudi Arabia. The study contrasted two sources of test item stimuli: one made by professionals at Michigan and Cambridge universities, in USA and UK, and the other locally by EFL instructors at two Saudi Arabian universities. The investigation also explored the influence on the dependent variable of two other independent variables: size of EFL lexicon as determined by Nation’s vocabulary test and general level of EFL proficiency as measured by TOEFL.
The analysis revealed sixty-two different TTS of six types: managing the test as a whole, reading the stem, handling the gap, examining the alternatives, selecting a response and strategies after choosing an answer. It also revealed that test-takers with lower L2 proficiency were more active TTS users than the higher proficiency testees. There was a clear difference between the two types of test stimuli, mainly in some of the TTS used to select a response. It was also found that there was a marked effect of the TTS on the test product. Approximately 40% of test items testing an unknown lexical item were answered correctly based on rational linguistic and non-linguistic strategies rather than on random promising choice.