Some Issues with Norm-referenced Tests

Below is part of the  post of my classmate Ian Kevin Magabilin on 5 November 2013 regarding norm-referenced test. I find his work worth taking because it highlights some problems with the traditional norm-referenced assessment. To view the full post of Ian, please click http://myportal.upou.edu.ph/mod/forum/discuss.php?d=72638

"Here are some of the issues with norm-referenced tests:

Tests can be biased. Some questions may favor one kind of student or another for reasons that have nothing to do with the subject area being tested. Non-school knowledge that is more commonly learned by middle or upper class children is often included in tests. To help make the bell curve, test makers usually eliminate questions that students with low overall scores might get right but those with high overall scores get wrong. Thus, most questions which favor minority groups are eliminated.

NRTs usually have to be completed in a time limit. Some students do not finish, even if they know the material. This can be particularly unfair to students whose first language is not English or who have learning disabilities. This "speededness" is one way test makers sort people out.

The items on the test are only a sample of the whole subject area. There are often thousands of questions that could be asked, but tests may have just a few dozen questions. A test score is therefore an estimate of how well the student would do if she could be asked all the possible questions.

All tests have "measurement error." No test is perfectly reliable. A score that appears as an absolute number -- say, Jamal's 63 -- really is an estimate. For example, Jamal's "true score" is probably between 56 and 70, but it could be even further off. Sometimes results are reported in "score bands," which show the range within which a test-takers' "true score" probably lies.

There are many other possible causes of measurement error. A student can be having a bad day. Test-taking conditions often are not the same from place to place (they are not adequately "standardized"). Different versions of the same test are in fact not quite exactly the same.

Any one test can only measure a limited part of a subject area or a limited range of important human abilities. A "reading" test may measure only some particular reading "skills," not a full range of the ability to understand and use texts. Multiple-choice math tests can measure skill in computation or solving routine problems, but they are not good for assessing whether students can reason mathematically and apply their knowledge to new, real-world problems.

Most NRTs focus too heavily on memorization and routine procedures. Multiple-choice and short-answer questions do not measure most knowledge that students need to do well in college, qualify for good jobs, or be active and informed citizens. Tests like these cannot show whether a student can write a research paper, use history to help understand current events, understand the impact of science on society, or debate important issues. They don't test problem-solving, decision-making, judgment, or social skills.

Tests often cause teachers to overemphasize memorization and de-emphasize thinking and application of knowledge. Since the tests are very limited, teaching to them narrows instruction and weakens curriculum. Making test score gains the definition of "improvement" often guarantees that schooling becomes test coaching. As a result, students are deprived of the quality education they deserve.

Norm-referenced tests also can lower academic expectations. NRTs support the idea that learning or intelligence fits a bell curve. If educators believe it, they are more likely to have low expectations of students who score below average."

 

References:

Suskie, L. (2003). What is “Good” Assessment. Retrieved fromhttp://faculty.ccp.edu/dept/viewpoints/f03v4n1/suskie.html

Merrell, A. (n.d.) Traditional Assessment. Retrieved from

http://audreymerrell.net/INTASC/INTASC8/Assessment/traditionalassessment_files/traditional.html

Fairtest. The National Center for Fair and Open Testing (2007). Retrieved from http://www.fairtest.org/facts/nratests.html

 

 

Guilty as Charged

Assessment of students’ skills and performance through observations and other traditional testing methods is not only for the concern of the learners but also for the teachers. This is because the outcome of the evaluation does not only reflect the level of students’ understanding but also indicate the quality of instruction received by the learners. Wary of this implication, teachers do their best to see to it that their students’ performance and skills meet the standards set by school officials. To achieve this, more often than not, teachers result to teaching to the test. Is this cheating?

 

Teaching to the test refers to giving instruction to students of the contents of the standard tests that they shall take in a given time. Nothing is wrong in this definition. The problem lies when the teachers or instructors give the exact items or similar to it during class discussion or review. Some academicians assert that this kind of tactic is wrong because it is tantamount to cheating.  Popham (2001) argues that item testing is unacceptable because it takes away valid inferences about the outcome of the assessment because the result assumes that the students also understand the other contents of the curriculum not just the items appearing on the test.



Veteran educator Jeanne Clements (2013) argues in favor of teaching to the test and recommends its wide use. However, she is not talking about reviewing the exact items on the standardized test but on the specific contents of the test. She stresses that the test does not rely on rote memorization of facts and concepts. She further asserts that no one condemn licensure examinations for professionals when the reviewers also result to teaching to the test.



As a student at Tapinac Elementary School in Olongapo City, I had never experienced that my teachers  were actually using the outmost form of teaching to the test, which we can also refer to as “cheating to the test.”  We can all infer that my teachers knew before hand the contents or subject matters of the standard or achievement tests to be given at designated time that they needed to teach us students. However, they never drill us on the actual test items or “clones” of them. They just presented us with the format and the directions of the actual test to familiarize ourselves to save time. Since we knew before hand that the result of the test will not become part of our final grade (which we never knew if ever true), test anxiety was diminished.



Nowadays, teachers’ performance are also gauged against the results of their students’ evaluation. As a result, they result to “cheating to the test” or during performance evaluation conducted by principals and other district supervisors. One of the tactics employ during visits of high ranking school officials and achievement tests is to tell low-performing pupils not to come to class during the date of the visit or test. A drill before the visit is perfected by simulating the visit’s scenario through giving high achievers their roles, that is, who will answer first, who will ask question and what, etc. This results to high satisfactory rating to students, teachers and schools.



To overcome their shortcomings as administrators, principals and teachers, there are many instances that sample or the actual test papers are being reviewed by the students and teachers. Because no one from the students up to high ranking officials of the district want to get a failing or unsatisfactory rating or remark from those above them, the performance maybe conceived as cooperative. This kind of drama becomes unacceptable when the learners do not gain from the experience. As long as the students understand the materials being presented and taken during class and these items are not the actual items,  teaching to the actual test items is not wrong. This is also what is happening during review classes for the professional accreditation and certification. Teaching to the test is reprehensible if the test items being reviewed are the actual and same items or their clones.

As some teachers are guilty to some form of teaching to the test, as a student, I am also occasionally guilty to “learning to the test.”  This is especially true when the teacher provides you with so many reading materials and that you do not have enough time to read them all. As a result, I tend to read and study only those that answer the study guide questions and the learning objectives. This becomes challenging because I also write what I read to fully understand them. I believe that this strategy is not at all faulty since I cover the essential knowledge and skills that need to be learned. This is different from high school where you are assigned one particular textbook to study. However, I do hope that a compressed and comprehensive reading materials is provided that tackle the essential elements of the module without sacrificing content and higher order skills, if time is limited to comprehend various materials and media.

 Images from

1) http://static.guim.co.uk/sys-images/Education/Pix/cartoons/2012/10/12/1350042607200/Ros-Asquith-Lines-cartoon-001.jpg

2) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj52DQ3SCyNi2790LSQSabHOqI6IGPM13uKTA0gvS58Pp3rBo8reMV2L4G-OunM9hBz_yHBP8escNW_sz0dFSvixAKTYolUSwjiNKnI-sbj611BZvZZIs2Skw1ogmmSYhtivXl_yIM3yM21/s1600/david-sipress-prospective-teacher-being-interviewed-by-principal-is-given-impossible-to-cartoon.jpg

3) http://seattleducation2010.files.wordpress.com/2012/04/testing1.jpg

4) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUtXYFDpRCXiOqkQtsxIz83YljcIxcRW-jFVj98lfDb_03byLsqgbm5VmLD8ztquKZ84XaKj6MxKQZcorn-7ZnH6dyxjhumBArGvpTFTbAEf28PYAJRFA0VOa-Il6BJ3Ki0OEN8zNaQFI/s640/1b2.jpg

5) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVsfoztllNHOFjX6sKT2Eg0WoX3u45BHAaKXWv0mZF9KiYGqVKVQgXl1mOu8_O9vREkWdgnY-PYTZNWnwcScufO1zIXdBKyir_6S9kDQq94eSk_4_0DV0RilqHeHmKIvYNkwIjuG9XAwUe/s1600/multiple+choice+cartoon.png

6) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgASvgC4_WgHfSDyNqAckGWDKPgJ79PWp7OMmzyhqN1awk01pYV5DffPYY_n9o6V-uYkVKTZ3Z0uubBhVbe7SrerN7UrDE3ha9NmrZSn5h0rMrNvvgrfMPnBi6jC1qqJRnJ6hzm90X1m-I/s1600/Learning+to+Take+Tests+Sign.jpg

References 

Clements, J. (2013). Common Core: Veteran Educator Urges Teachers To ‘Teach To The Test’. In CP Opinion. Retrieved from http://www.christianpost.com/news/common-core-veteran-educator-urges-teachers-to-teach-to-the-test-100938/

Popham, W.J. (2001). Teaching to the Test? Helping All Students Achieve. 58 (6), 16-20. Retrieved from http://www.ascd.org/publications/educational-leadership/mar01/vol58/num06/Teaching-to-the-Test%C2%A2.aspx

2)

How I Viewed Educational Assessment

Although educational assessment has been dreaded by many students from preschool to higher level, it is a very significant part of the teaching-learning process. Assessment as defined in the University of Lincoln’s website pertains to the process of gathering of information to measure student’s “knowledge, skills, attitudes and beliefs.” Simply, it refers to what teacher does to monitor the progress of his/her students, aid them in their learning and re-design his/her instructional methods and /or assessment strategies to attain his/her predetermined learning goals.



As an elementary student, I was not afraid of the test, be it short or long, surprised or not, because I listened to my teachers in class, did my homework and I was confident that I could answer most of the questions. What I hate was when the test given was so simple compared to what our teachers hinted us to study. For example, our Social Studies teacher would normally gave us 5 chapters to review and then gave a 20 item- multiple choice questions. It drove me mad because I felt I was short-changed probably because Social Studies was one of my favorite subjects and found studying the different regions of the Philippines interesting.

It was the reverse in high school. I found Social Studies class less interesting because I was poor in politics. I prayed not to be called to recite during class. It was because we were no longer remembering products and capitals. We were analyzing the socio-cultural and political systems of our country and I was poor in analysis. One short answer was not enough. You have to defend your argument. Surprisingly, I still got good grade.



Although I got good grades in Mathematics, Geometry was one branch that I was struggling. You’ve got it right! Because it involved analysis. I usually could not prove a theorem. No matter how hard I tried to study, I still could not get it. I managed to get an 85, though.

What I liked most in high school was Filipino. Oh, no! Not Dr. Jose Rizal’s Noli Me Tangere and El Filibusterismo but writing essays and short stories. My love of writing in the national language was developed in high school. I was chosen as associate editor of our school organ in Filipino. I also won an award in the Regional Secondary Schools Press Conference. Our school paper won the Best in News Writing in the national contest. I got Best in Journalism – Filipino during my graduation.



Getting good grades in elementary and high school was very important to me. Not only for my own satisfaction but also to the pride and happiness that they gave to my parents. High grades are testimonies that you studied hard. All your sacrifices and hard work had been rewarded. Getting high grades in high school was my primary goal because it was a way to free college education. Belonging to a poor family, my parents could not send me to college without scholarship. Because of the dedications that I committed to my studies, I was chosen as one of the recipients of the UP Experimental Democratization Students Scholarship (UPXDS) in 1977. Getting high grades in the University was very important to maintain my scholarship. I studied Bachelor of Science in Business Administration and graduated in 1982. And whether we like it or not, grades are one of the indicators of employability in the labor market.



During the course of my studies, I never had a failing grade but I got an "Incomplete" in Computer Studies. Again, you got it right. Because this subject involved analysis and I was so poor in that. As a result, I re-enrolled it one summer and got a 3. Whew!!! My only “tres”. I also got an ebb and flow grade in Algebra. My lowest marks happened when the subject matter involved word problem. Yes, you have got it right once more – analysis.

I shall be frustrated and less motivated if I got a failing grade in any subject especially when you did your utmost share in the learning process. You could blame the professor especially when he/she thought that all “iskolar ng bayan” were outright smart and left you on your own (which was always the case then, I could say).]



It seemed that I was really poor in analysis. I could solve mathematical problems when the equations were all set up but I usually did poorly when I have to set them up by myself. Well, I could say now that it was because my mind then was only at level 3 of Bloom’s taxonomy of learning as revised by Lorin Anderson(2000) as cited by Don Clark. My learning was only in the remembering, understanding and application stage.



Ultimately, at this stage of my learning process, grades are not that significant to me compared to my previous formal education. Although they will always give you some sort of pride and achievement, high grades are now measurements of how the teacher perceived you to have attained his/her learning goals rather than indicators of how much you learned.

Images from

1) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjF0hwlZ5uvv3v1mo6qJ56NoN_iMXEbA8p9nTN2tI1M4CBPo06_-Zuhs1uzhB2HDlMS8e-K6aByfBDpJN2HF8ijekaR9IOn9VjKy_lEt_X1P8ZYoPThubDGE5w1Yoz8fpWqPNE2vCIbo1re/s400/nclb-test-cartoon.png

2) http://math.pppst.com/banner_math_geometry_formulas.gif

3) http://blogs.themailbox.com/themailbox/wp-content/uploads/2009/01/report-card-clip.gif

4) http://xa.yimg.com/kq/groups/10246900/homepage/name/161183?type=sn

5) http://www.nwlink.com/~donclark/hrd/learning/id/bloom_taxonomy.jpg

6) https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4znyXFQCYcdyGbuvBNAXza6Ub858KosIFjj2O6VRPXUz-usySzyhpSDdPttFFYOs3rcovDuNeP2B0pGdIiCOEncv8CbPcCanp2zbST0bNgKM3UGsdvXvB5P2-RFX7HQwgmpcBP7DQbs4/s400/Cartoon+Teacher.gif

References

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., Wittrock, M. C. (2000). A Taxonomy for Learning, Teaching, and Assessing: A revision of Bloom's Taxonomy of Educational Objectives. New York: Pearson, Allyn & Bacon.

The University of Lincoln.  Educational assessment. In academic room. Retrieved on 21 September 2013 at http://www.academicroom.com/topics/what-is-educational-assessment