What Teachers Need to Know, and Be Able to Do, About Norm-Referenced Tests
By Natalie Kuhlman, Ph.D., ELL Outlook™ Contributing Writer
“How do I make decisions based on this test?” is a common demand among teachers in general, and even more so among teachers of English language learners (ELLs). Many teachers of ELLs may wonder what the tests they are required to administer are measuring: an ELL’s English language proficiency? Academic achievement in a specific subject area? Something else? Furthermore, teachers want to know how, if at all, the results of standardized norm-referenced tests should affect their curriculum decisions and teaching methods.
This article will first present some of the background knowledge necessary to understand norm-referenced tests, then offer some contrasting opinions on how these tests should-or shouldn’t-alter classroom instruction, and end with some practical advice for teachers on how best to prepare ELLs for norm-referenced testing.
What Do Teachers Need to Know About Norm-Referenced Tests?
Norm-referenced tests are often confused with criterion-referenced tests. What are norm-referenced tests? What do they measure? What is their purpose?
What are norm-referenced tests?
Some teachers misunderstand norm-referenced tests, believing that these tests are instead criterion-referenced. Criterion-referenced tests are based on specific criteria that students answer either correctly or incorrectly; the number of correct or incorrect answers determines whether students pass or fail the test. Although there are some criterion-referenced achievement and language proficiency tests in use these days, most standardized tests are norm-referenced. With these tests, a particular student’s score is determined through comparison with the students in the group that was used to establish the norms. Norms are based on a bell curve, which means that the same percentage of students will receive a 90th percentile score as will receive a 10th percentile score. If a norm-referenced achievement test hasn’t been normed with ELLs, then ELLS will be unfairly compared to the sampling population.
What knowledge and skills do norm-referenced tests measure?
First of all, items on norm-referenced tests are usually not grouped by objective and are not focused on a given skill level. Some test publishers do group items by content or objective and align them with state standards, but in order to cover a broad array of standards, norm-referenced tests usually have only one or two items per objective or standard. Chase (1999) claims that there need to be at least ten items on any one objective (such as an aspect of grammar structure) for it to inform instruction. With most norm-referenced tests, there are rarely enough items for any one objective to have an impact on instruction or to help in diagnosing strengths and weaknesses.
Second, norm-referenced tests that are not specifically designed and normed with ELLs may end up measuring their English language proficiency instead of their subject-area knowledge and skills (Rhodes, Ochoa, & Ortiz, 2005). For example, do test results reflect students’ knowledge of fourth-grade mathematics, or the fact that they don’t have the academic English proficiency in fourth-grade mathematics to know what the questions are asking?
What are the purposes of these tests?
Both norm-referenced standardized achievement tests, and the English language proficiency tests that are given yearly, are intended for large scale assessment (Gottleib, 2003). In other words, the test administrators want to be able to compare large numbers of students, as is required by No Child Left Behind (NCLB) legislation. The current demand for “accountability” is satisfied through these standardized tests. Unlike criterion-referenced tests, norm-referenced tests do not show a learner’s progress over time; they simply show how one student compares to another.
Some Opinions on Using Standardized Test Results in the Classroom
Should teachers take much-needed time away from teaching ELLs English to prepare them to take an achievement test? History tells us that ELLs will do poorly on such a test anyway because the tests are in English.
On the other hand, should teachers who don’t help their students prepare for norm-referenced tests just let students fail? Isn’t a certain amount of test preparation a necessary evil to ensure that ELLs have some level of preparedness for a potentially very foreign assessment?
What do teachers think?
Anita, a second-grade teacher, says, “I receive their scores once they are already in third grade. There is nothing I can do to help them at that point. Incoming second-graders don’t have scores. They test, and the scores are useless to a second-grade teacher.” If teachers receive testing results after their students have already moved up a grade level or if they don’t have the scores for new students, they have no way to tell how well students are doing compared to their peers.
Furthermore, Anita continues, “The test results must be reported in an individual student item analysis where teachers are informed as to what questions/standards students missed (need to identify trends and patterns). Simply being informed that they are below basic in Language Arts or Math does not significantly inform instruction or let the teacher know individual student strengths/weaknesses”. Teachers need concrete data that show which standards students are struggling with in order to adjust instruction.
In addition to the issue of whether these tests actually reflect what students know and can do, another issue is whether they are even fair measures for ELLs. As Linda, a teacher quoted in Wright (2002, p. 15), says:
I think we need to ask ourselves, “What are we testing for?” Are we testing language? Then let’s call it a language test, and let’s make it oral. Are we testing reading? Then let’s talk about what we’re testing for. To try to purport that we are testing for knowledge and acquisition of skill, when we’re really testing for language and reading ability is deceiving the public, it’s deceiving the taxpayers, and making it look like we’re not providing a solid education.
Linda makes a valid point relating to the purpose of these norm-referenced tests when used with ELLs. In general, the perception of English language proficiency is monolithic- either you are fluent in English or you aren’t. Many times, success or failure on an English language proficiency test-or, for that matter, any academic achievement test-is seen as the absolute and conclusive measure of an ELL’s academic potential, regardless of the original purpose or content of the test. Consider, for example, what scoring at the advanced level of English language proficiency on the California English Language Development Test (CELDT) means. Is this student fully prepared for a mainstream classroom? Is his or her previous lack of English language proficiency a problem of the past? Is this student simply fluent in the areas that the CELDT happens to test?
How Should Teachers Prepare ELLs for Norm-Referenced Tests?
Whether teachers are knowledgeable, or whether these tests are a valid measure of what students know or can do, norm-referenced standardized tests are not going to go away. If you teach to the test, students may not gain all of the necessary skills and strategies needed to succeed in the long term. If you don’t teach to the test, students may perform poorly due to lack of test-taking skills. Ideally, any such assessment should reflect what goes on in the classroom, and no special preparation should be needed for students, be they English language learners or students who are proficient in English. But we all know that isn’t the case.
That said, we do have an obligation to demystify the process at the very least. Students shouldn’t be made to feel badly because they don’t understand what to do or why they are doing it. They do need to know what a multiple-choice test is and how to avoid errors that have nothing to do with their knowledge of the subject. They also need to know that these tests are not going to affect their grades; rather, that these tests are intended to inform the education of students (or at least one hopes that is the case).
The following are some suggestions of what teachers can do:
- Build test-like structures into everyday instruction in nonthreatening situations and activities.
- Show ELLs how to use dictionaries in general and bilingual dictionaries specifically, because in some states and/or districts these are allowed.
- Translate (when possible), or scaffold the kinds of instructions the students will hear in the testing situation, so that ELLs get used to the terminology and it doesn’t become a hindrance when the tests are given.
- Demystify multiple-choice tests, including the “tricks,” in a positive environment. We can’t assume because students are in the first, fifth, or even tenth grade that they know what a multiple-choice test is.
- Show students how to fill in a bubble.
- Model how to make choices from four available options, using the process of elimination. This can be done within classroom instruction by setting up games in which students have to make choices about vocabulary or match pictures with printed words.
- Provide direct instruction with modeling and practice about how to make intelligent choices when the answer is not known.
- Help students use context to understand test items, and teach them how to guess wisely.
- Show students how cognates help demystify vocabulary. Cognates will help with vocabulary that is common between a student’s two languages. Anita suggested the importance of learning about idioms. Understanding idioms can help demystify culturally laden vocabulary.
- Teach specific genres and grammar structures that are commonly found on standardized tests. This will also enhance students’ English language development.
- Scaffold background knowledge specific to test structures.
- In some cases, ELLs are given extra time for these tests, since it takes longer to read in a second language. But students need to learn how to use this time wisely. For example, they should answer the easier questions first, and then go back to ones they don’t know.
The political emphasis on accountability, using norm-referenced standardized tests as the ruler, is unfortunate. Dillon (2006) reports on schools where subjects such as science and social studies are disappearing in the interests of ensuring that students score well on the reading and math tests mandated by NCLB. Teachers frequently bemoan, “No child left untested.”
But when teachers become knowledgeable about what standardized tests can and cannot do, they can begin to make decisions about what to do about the tests. I am not suggesting that teachers spend direct instruction time on preparing students to take standardized tests and take time away from good English Language Development (ELD) instruction. Rather, implementing some of these suggestions may give teachers ideas to build on what they are already doing in the classroom.
At the least, teachers need to help their students take these tests, because not to do so is to do our students a disservice.
Chase, L. (1999). Contemporary assessment for educators. Longman.
Dillon, S. (2006, March 26). Schools cut back subjects to push reading and math. The New York Times.
Gottlieb, M. (2003).
Rhodes, R. L., Ochoa, S. H., & Ortiz, S. O. (2005). Assessing culturally and linguistically diverse students: A
Wright, W. E. (2002). The effects of high stakes testing in an inner-city elementary school: The curriculum,
Kuhlman, N. (2005) Unpublished interview data
Gottlieb, M. (2003) Large Scale Assessment of English Language Learners: Addressing Educational Accountability in K-12 Settings. TESOL Professional Papers #6. TESOL.