Technological advances have revolutionised methods of both teaching and testing in languages, and practitioners have eagerly embraced the opportunity to provide more innovative ways of doing this. The unique features offered by technology make it increasingly possible to test for a wide range of language skills required for a specific purpose. With the increasing need to test for English language proficiency and the importance placed on this facility, technology is being utilised to address issues of practicality, speed and efficiency. However, such advantages should not be embraced without due consideration for the essential qualities of any test; validity and reliability. With the inevitable shift towards computer-based testing, certain areas need special consideration. While computer-based tests can provide constructive diagnostic information to complement the language learning process, they should be used more selectively in other contexts, for example in high-stakes tests, examples of which are International English Language Testing System (IELTS) and Test of English as a Foreign Language (TOEFL). These tests provide an assessment of English language proficiency, a key component of university admissions criteria. This paper examines a number of issues related to the design and application of computer-based tests, with particular reference to construct validity, computer familiarity and practicality. It is recommended that in the short term at least, test-takers be offered the choice of test medium in the interests of fairness and equity.