Vol. 4. No. 2 R-2 November 1999

Return to Table of Contents Return to Main Page

It is probably inevitable that this book would appear. Like many of the commercial guides to standardized tests, this one promises to maximize the performance of test-takers who study it and carry out its practice exercises. Most bookstores have shelves dedicated to books on how to take the GRE, the GMAT, and the TOEFL. These guides are not only published by commercial publishers such as Barron, Kaplan, and Arco, but by the Educational Testing Service (ETS) itself, which produces test preparation books and software for all three. The book reviewed here promises to maximize the performance of non-native English speakers on another test produced by ETS, the Test of Spoken English (TSE), or its institutional version, the Speaking Proficiency English Assessment Kit (SPEAK).

ETS developed the Test of Spoken English in the 1970s to measure the ability of nonnative speakers to communicate in English. Unlike the TOEFL, which does not measure speaking ability, it serves as an indicator of oral communication skills. It is used to evaluate and screen people who wish to work in the health professions in North America (medicine and nursing, for example), and to evaluate and screen international students for positions as teaching assistants in North American colleges and universities. There are thousands of international students who aspire to teaching assistantships, and likewise many foreign professionals seeking employment in health professions in North America, so it would seem likely that there would be a large market for test preparation materials aimed at the TSE and the SPEAK. It is worth noting that such materials have not previously been marketed, even by ETS itself. This does not appear to be accidental.

I was trained as a TSE rater in 1989, and in the 1990s began administering and scoring SPEAK tests for international teaching assistant (ITA) candidates for a large university. I was curious to see Toward Speaking Excellence because it seemed to me that extensive preparation for these tests could defeat their purpose. I still think so, but will try to review the book on its own terms as well as mine.

The book consists of a two-page introduction followed by eight chapters, two appendices and a very brief bibliography, aimed directly at test-takers, not language professionals. The whole is succinct, well organized (both in terms of text and of visuals), and well edited. It is written in a consistently friendly, respectful tone. From an educational standpoint, I would consider the book an effective tool for its purposes--very thorough and user-friendly. [-1-]

The introductory paragraph states that the book is designed to maximize the testee's potential on the TSE and the SPEAK. Further on it claims to be useful in preparation for "other oral exams or oral interviews" or for those "who desire to improve their communication skills in general" (p. 1). The next part of the introduction explains that the book contains communication strategies and test-taking strategies for "all the sections of the TSE and SPEAK" (p. 1). It justifies this on the grounds that students will become more "test wise," buttressing this justification with a quote from the Division of Measurement and Evaluation at the University of Illinois in Champaign-Urbana (1994): "Test-wiseness is positive when it allows students to better demonstrate their knowledge of course material" (p. 1). This seems inappropriate since the TSE and SPEAK do not measure students' knowledge of any course material, but rather attempt to measure oral communicative ability in general. Any educator would agree that it is important for students to be test-wise in the sense of being informed about a test and being familiar with its format. ETS explains test procedures and provides a sample test on its web site and in its bulletin to familiarize students with the test format, but providing templates for answers to questions designed to elicit spontaneous speech--which in some cases this book does--seems qualitatively different from what most educators would understand as helping students become "test-wise."

Chapter 1 explains the organization of the book, which follows the organization of the TSE itself. On the test (often given in language laboratories), the test-takers listen to instructions and their responses are tape-recorded. The whole thing lasts 20 minutes. The book provides ways of handling the different sections of the test, beginning with the warm-up questions (chapter 3), then the map section (chapter 4), the picture section (chapter 5), the graph section (chapter 6) and the announcement section (chapter 7). Chapter 1 goes on to present the test's four rating criteria: a) language function (giving directions, defending an opinion, recommending, and announcing, for example; some of these criteria, like announcing and giving directions, are on all the tests; others vary); b) appropriateness; c) coherence/cohesion; and d) accuracy. The first chapter ends by suggesting general test preparation strategies, such as bringing a watch, and imagining you are speaking to a person in front of you. It also suggests one strategy uniquely possible on the TSE and SPEAK, but not in other oral interview situations: since all the questions are printed out in the test booklet, testees are instructed to follow the printed version of the narrator's questions so they can give attention to the pronunciation of words, and so they can use the printed words to match their responses. This is undoubtedly good advice, for purposes of passing the test, but it seems to me to degrade the test's function as a measure of communicative competence, since it diminishes the need for listening comprehension, which is an important factor in the ability to communicate. The opportunity to do this is ETS's doing, however, not the author's. Still, I am sure that there are students who, without this advice, would not follow along so closely and consequently not recognize word boundaries, say, or a word itself, information which would be of value. [-2-]

Chapter 2 is a straightforward presentation of the scoring scale for the TSE and the SPEAK, which generally follows ETS's published explanation. Typical score interpretations are provided (60 is considered very good, 40 middle-of-the-road, 20 bad). Some possible responses are given and scored, and they appear to closely reflect ETS guidelines.

Chapter 3 is the first dedicated to a particular section of the test, the warm-up questions. Even though the warm-up questions are not scored, very detailed instruction and practice are provided. The author explains that this is, first, to build up confidence, and second, just in case the rater does hear these responses, to prevent the rater from getting a bad impression. Students are warned, "Do not try to say too much; the more you say, the more chance you have to mispronounce sounds and confuse the raters if they happen to listen" (page 18). This is one of many strategies offered in the book guaranteed to reduce spontaneity of communication. The warm-up questions, which are meant to do just that--warm the students up--become another site for test anxiety. At the same time, the author has a point. Whether or not the raters intend it, they could possibly be influenced by coming across these responses when they rewind the tapes for rating. This chapter provides extensive practice in how to read and stress numbers, particularly four-digit numbers. This is a useful skill for any professional or incoming teaching assistant in any case. The same is true of the strategy of using question stems, such as I began . . . in response to When did you begin to study English? Ten possible responses which could be used to answer this question are supplied, and students are warned again not to add any extra information to their answers because it would "increase the opportunity for errors." The author recommends a "short, complete answer" exclusively because it "sounds" fluent.

Chapter 4, on the map section, offers advice about "common themes" for U.S. street names, including instruction on names of states, presidents, and ordinal numbers and their stress patterns. This instruction seems more specifically useful for the test than the ability to read numbers, particularly since the people taking the test are more likely to be asking for street directions in English than giving them. Templates are offered for giving directions, in the form of "sample responses": "Exit the coffee shop . . . Walk west . . . turn north . . . At Twenty-Third and Main . . . you'll see . . . on the east side. On the west side is . . ." (p. 35), and so on. Similar templates in the form of sample responses are offered for making recommendations and giving opinions: "I think you really ought to. . . First of all . . ." (p. 30). "If you haven't [X] . . . I highly recommend it" (p. 37). Organizational markers such as "The first reason . . . The other reason" (p. 37) are suggested and students are strongly advised to sound enthusiastic. Six elaborate practice sets give students plenty of opportunity to rehearse ways of recommending things, justifying recommending them, giving directions, and expressing opinions.

In chapter 5, the picture section, two tense schemata are offered for narrating a picture story, with the caveat that whichever is used (narrating in the present tense or narrating in the past tense), it be used consistently. However, "it is easiest," the author advises, "to stick with basic past tenses" (p. 48). Two perspectives are offered as alternative modes of response, first and third person. For sounding cohesive, the use of because, as in "Because it was hot . . . " is suggested. Other expressions such as although and while are recommended. Following these patterns would definitely limit the range of responses that might naturally occur during this task. [-3-]

Test-takers are advised that the picture sequences will always represent some kind of mishap or conflict and told that one of the important things they should do during the period allowed to study the pictures, is to conceive of how the situation could have been avoided, as they will have to offer some solution. Templates are offered for answering this, including the useful needs to . . . in order expression: "The biker needs to ride more carefully" or "The bike path needs to be maintained better" "to avoid" or "in order to avoid" (pp. 52-53). Students are warned not to be demanding in the common situation of role-playing a person trying to persuade someone else to do something. A list of polite expressions is provided. Connective expressions are recommended, such as "Earlier today . . . if you're going to . . . Furthermore . . . So I would like to . . . Well, if . . . " (p.55). A number of Papajohn's suggested transition expressions throughout the book were ones which would be more common in writing than in speech. We don't say therefore very much, or furthermore, or although. In speech we would be more likely to say so, or since, also, and even though. He is right to assume, though, that the use of the words he suggests would probably impress raters. He also is probably right in his amusing advice to test-takers who do this "persuading" role-play that they should end on a positive note, so as to suggest to the rater that they indeed have the ability to persuade in English.

For what are supposed to be the more open-ended questions, "sample response" templates are provided for discussing pros and cons of issues. It is suggested that these take the form of noting "advantages and disadvantages." Lexical phrases such as "two important factors . . . On the other hand . . . [X] is another factor . . . " are provided. Issues are described as having two sides, artificially simplifying issues that might otherwise receive a more nuanced response. Following these directions would be a lot like following those for the traditional 5-paragraph canned essay, and lead to the same dead discourse. The advice is: "choose the side that is easiest for you to talk about" (p. 73). This subverts the intent to measure communicative competence. In the real life tasks the test-takers are being screened for, they cannot simply choose to communicate those facts about physics or economics or autopsies that they find easiest to express.

In chapter 6, test-takers are instructed on how to read coordinate graphs. Like reading numbers (and unlike narrating picture stories), this is a skill directly relevant to the professional demands that will be made on test-takers and is one part of the book where Papajohn's instructions transcend simply getting through the test. If they can do it on the test, they are genuinely likely to be able to do it in other contexts. Also in this chapter, test-takers are given instructions on how to define and explain field-specific concepts. Useful lexical phrases and a five-step procedure for defining and explaining are provided.[-4-]

Chapter 7 treats the announcement section of the test with detailed instructions for how to open with a greeting statement (being mindful of appropriate register). Many lexical phrases for presenting schedules are suggested and detailed practice given in the pronunciation and stress of months and dates. As in the previous chapters, five practice sets follow the instruction. The practice sets and tests throughout the book seemed well thought out, reflecting the kinds of questions, situations, graphs, and maps ETS provides in its tests. The exceptions were a number of topics that probably would not have passed the ETS cultural bias antennae: whether or not the U.S. should continue to use pennies (p. 92); expressing opinions about people in geographical regions called counties (p. 92); why home ownership is part of the American Dream (p. 91); peanut brittle (p. 78); carpooling (p. 75); ATMs (p. 71); and the "pros and cons of purchasing versus leasing a car" (p. 64). The book ends with two appendices: one which includes two practice tests and the other containing test registration information.

On page 2 the author quotes Hopkins, Stanley, and Hopkins (1990): "Wider experience and training in preparing for and taking tests of all kinds is likely to increase accuracy of measurement and, therefore, the fairness of scores for the students tested." Given that this test is an attempt to measure communicative competence, I would say that the use of this book would lead to a decrease in accuracy of measurement since its instructions limit spontaneity and decrease (personal) authenticity of response. As a result, the scores may not be fair to the test-takers (who are not evaluated as well as they otherwise would be and may, as a result, be faced with demands they are not ready to fulfill) or to employers (who may be led to assume more competence than exists) or to colleagues or patients or coworkers of test-takers (who depend on the test-takers' competence in many ways) or to people who take the test and have not been given these instructions and who therefore may not disguise their real competence (or lack thereof) in similar ways. It's anybody's guess how much difference the book might make for a given testee. It is not likely to turn a score of 20 or 30 into a 60, but very high and very low proficiency levels are never the hard choices confronted by raters or score users. It is the intermediate cliff-hangers who present the most difficulty. These are the people who would be most likely to have their scores inflated by the extensive rehearsal the book recommends. Putting someone who is not competent to be there in the position of lab instructor to students handling dangerous equipment and substances can have serious consequences. It is also very dangerous to employ medical personnel who do not communicate well. You want to have the most and the best information available to you to make such decisions. And while ETS routinely warns score users that they should use other indicators as well, the fact is that most institutions give great weight to scores on the TSE and the SPEAK.

In sum, I would say that this book is likely to do very well what it promises--to maximize test scores on the TSE and the SPEAK. It will probably also be a good seller for the University of Michigan Press. I am sorry to see it appear, though, because I think it will achieve higher test scores for students at the expense of degrading the quality of the information the tests provide.

References

Hopkins, D. D., Stanley, J. C., & Hopkin, B. R. (1990). Educational and psychological measurement and evaluation. Englewood Cliffs, NJ: Prentice-Hall.

Sharon Myers
Texas Tech University
<ntmye@ttacs.ttu.edu>