Some Issues with Norm-referenced Tests

Below is part of the  post of my classmate Ian Kevin Magabilin on 5 November 2013 regarding norm-referenced test. I find his work worth taking because it highlights some problems with the traditional norm-referenced assessment. To view the full post of Ian, please click http://myportal.upou.edu.ph/mod/forum/discuss.php?d=72638

"Here are some of the issues with norm-referenced tests:

Tests can be biased. Some questions may favor one kind of student or another for reasons that have nothing to do with the subject area being tested. Non-school knowledge that is more commonly learned by middle or upper class children is often included in tests. To help make the bell curve, test makers usually eliminate questions that students with low overall scores might get right but those with high overall scores get wrong. Thus, most questions which favor minority groups are eliminated.

NRTs usually have to be completed in a time limit. Some students do not finish, even if they know the material. This can be particularly unfair to students whose first language is not English or who have learning disabilities. This "speededness" is one way test makers sort people out.

The items on the test are only a sample of the whole subject area. There are often thousands of questions that could be asked, but tests may have just a few dozen questions. A test score is therefore an estimate of how well the student would do if she could be asked all the possible questions.

All tests have "measurement error." No test is perfectly reliable. A score that appears as an absolute number -- say, Jamal's 63 -- really is an estimate. For example, Jamal's "true score" is probably between 56 and 70, but it could be even further off. Sometimes results are reported in "score bands," which show the range within which a test-takers' "true score" probably lies.

There are many other possible causes of measurement error. A student can be having a bad day. Test-taking conditions often are not the same from place to place (they are not adequately "standardized"). Different versions of the same test are in fact not quite exactly the same.

Any one test can only measure a limited part of a subject area or a limited range of important human abilities. A "reading" test may measure only some particular reading "skills," not a full range of the ability to understand and use texts. Multiple-choice math tests can measure skill in computation or solving routine problems, but they are not good for assessing whether students can reason mathematically and apply their knowledge to new, real-world problems.

Most NRTs focus too heavily on memorization and routine procedures. Multiple-choice and short-answer questions do not measure most knowledge that students need to do well in college, qualify for good jobs, or be active and informed citizens. Tests like these cannot show whether a student can write a research paper, use history to help understand current events, understand the impact of science on society, or debate important issues. They don't test problem-solving, decision-making, judgment, or social skills.

Tests often cause teachers to overemphasize memorization and de-emphasize thinking and application of knowledge. Since the tests are very limited, teaching to them narrows instruction and weakens curriculum. Making test score gains the definition of "improvement" often guarantees that schooling becomes test coaching. As a result, students are deprived of the quality education they deserve.

Norm-referenced tests also can lower academic expectations. NRTs support the idea that learning or intelligence fits a bell curve. If educators believe it, they are more likely to have low expectations of students who score below average."

 

References:

Suskie, L. (2003). What is “Good” Assessment. Retrieved fromhttp://faculty.ccp.edu/dept/viewpoints/f03v4n1/suskie.html

Merrell, A. (n.d.) Traditional Assessment. Retrieved from

http://audreymerrell.net/INTASC/INTASC8/Assessment/traditionalassessment_files/traditional.html

Fairtest. The National Center for Fair and Open Testing (2007). Retrieved from http://www.fairtest.org/facts/nratests.html

 

 

Some Basic Concepts and Terminologies on Assessment

As part of our research and understanding regarding basic concepts and terminologies on assessment, I caught the definitions of my classmate Regina Josephine Benito because she differentiates the various terms in a simple and yet comprehensible way.

Regina describes assessment as the gathering of information, analyzing and interpreting the understanding of the student. She stresses that assessment is used to:

• discover what the student needs regarding his learning (diagnostic);

• position the student in a learning environment to maximize his potentials (placement);

• evaluate the effectiveness of the program, and to

• give an opinion on student’s study.

Regina further describes formative assessment as student’s learning monitoring during the course while summative assessment is done at the end of the course. According to her, criterion-referenced assessment rests on specific or particular tasks or concepts where students are evaluated against a set of criteria or standards while norm-referenced assessment is derived on general skills or concepts.

For further reading, please visit Regina’s original entry at http://myportal.upou.edu.ph/mod/forum/post.php?reply=248794#mform1

Image from http://www.bisd303.org/cms/lib3/WA01001636/Centricity/Domain/765/report%20card.gif

Can we teach metacognition?

Another insights worthy of note is from Ian Carlo Illastron on the topic of teaching metacognition. Of all the other students who shared his view on the concept, I chose his thoughts because they elucidate my notion about metacognition. Although he did not elaborate much further how to teach the concept to students, his is a starting point. I just added the image in his writing . Original entry here: http://myportal.upou.edu.ph/mod/forum/discuss.php?d=47300

" Can we teach metacognition?

by Ian Carlo Illastron - Saturday, 15 June 2013, 11:38 PM

yes i think we can, its possible,

teacher need to evaluate students response to the tasks given and observe if the student is developing metacognition.



John Flavell, researcher of metacognition, believes kids need awareness in three areas: -

1. An awareness of knowledge — understanding what they know

2. An awareness of thinking — understanding cognitive tasks

3. An awareness of thinking strategies — understanding approaches to directing learning

and to develop awareness, teachers must;

1. Model our own thinking- Kids learn by watching us, Saying your thoughts out loud shows kids what you’re thinking, or “thinking aloud



2. Scaffold the thinking- Step by step scaffold the learning. We start at the beginning of noticing the thinking strategies, noticing what we know, notice if our strategies worked. We don’t start trying to implement new strategies before we know what we’re currently using as a strategy.

3. Facilitate and provide opportunities to notice thinking -

http://imaginationsoup.net/2012/01/teach-kids-to-think-about-their-thinking-metacognition/"

Image from http://edweb.sdsu.edu/people/bdodge/scaffold/How_To_Scaffold.jpg

Teaching for Transfer

Below is the complete response of Maria Angelita Carlos on the discussion board on teaching for transfer. I noted her inputs because they are very informative and relevant to my study. I do hope that the readers will also find this worthy of note. Original entry can be find here : http://myportal.upou.edu.ph/mod/forum/discuss.php?d=47302



☼ Teaching for transfer - Teachers will develop ideas about how to facilitate transfer in their own classrooms and how to build bridges for their students between concepts, activities, and lessons.

♦Transfer is the ability to extend what one has learned in one context to new contexts. The ultimate goal of schooling is to help students transfer what they have learned in school to the everyday settings of home, community, and work, we have much to learn from the non-school environments where people work.

♦The challenge of transfer: How can students use what they have learned by applying it to solve new problems? Given the vast array of knowledge needed in life,

♦The teacher’s challenge is to determine what is the least amount of material that she can teach really well that will allow students to use that knowledge in the widest possible range of situations.

♦For instance, in elementary school, students learn addition, subtraction, and multiplication and then at some point they also learn long division. Learning long division requires transfer because students have to take what they already know about adding, subtracting, and multiplying, and apply all three of those processes to learning a new kind of skill called division.

♦We can transfer an idea from one situation and use it in a new, but similar context. If a student has learned about the notion of a revolution while studying world history, she can transfer or apply the notion of revolution to her study of Philippine history. Later, she can apply these
understandings to other context when studying the another country's revolution.

♦We can also transfer within a subject matter, as with the concept of revolution, as well as across subject matter areas.

inspirations:
learnweb.harvard.edu/alps/thinkin
www.cdtl.nus.edu.sg/link/

Thiagi's Seven Laws of Learning

For facilitating experiential learning activities for many years, Sivasailam Thiagarajan came out with Seven Laws of Learning, and I quote:



“1. Laws of Reinforcement – participants learn to repeat behaviors that are rewarded.

2. Law of Emotional Learning – events that are accompanied by intense emotions result in long lasting learning.

3. Law of Active Learning – active responding produces more effective learning than passive listening or reading.

4. Law of Practice and Feedback – learners cannot master skills without repeated practice and relevant feedback.

5. Law of Previous Experience – new learning should be linked to ( and  build upon) the experiences of the learner.

6. Law of Individual Differences – different people learn in different ways.

7. Law of Relevance – effective learning is relevant to the learner’s life and work.”

I believe that the above laws will be helpful in devising learning materials and creating suitable environment for learners. Aware of these, the teacher can align his/her teaching styles to accommodate some of these.

Although Thiagi’s laws have no scientific basis or study by his own, his experience is enough to prove his points. To know more of about S. Thiagarajan and his thoughts, please go to http://squarewheels.com/articles2/thiagi.html

Learning and the Senses, A Fraud?

I have came across the entry below in Reynaldo Flores' blog at http://reynaldojrflores.wordpress.com/2013/06/08/module-2-learning-styles-theories-part-1/. Rhed cited this at http://web.cortland.edu/andersmd/learning/Introduction.htm



Accounting for the differences of individual learning styles in relation to the five senses, it is interesting to note that on the average, as enumerated by Rief (1993), we tend to retain:

• 10% of what they read


• 20% of what they hear


• 30% of what they see


• 50% of what they see and hear


• 70% of what they say


• 90% of what they say and do

I tried to find out more about Rief and his observation in the internet. Specifically, I would like to know what measurement/assessment was used for this result; how many respondents were involved; and from where.

My search landed me nowhere. Instead, I found out an article who disputed this result and labeled it as fraud.I Will Thalheimer tried to trace the "history" of this quote in his blog. He convinced me that this observation  was indeed a fraud although it was being circulated worldwide and had been accepted as true. This was partly due to the fact that no scientific method or research had been done. Also, the identity of the first person who coined it was unknown.

Read more of Thalheimer's perception at  http://reynaldojrflores.wordpress.com/2013/06/08/module-2-learning-styles-theories-part-1/ and have your verdict.

Educational Tablets Versus Textbooks

I was fascinated by the result of the observation of my classmate, Jessica Mae Ylagan, regarding the behavior of Thai grade school students. She observed that those who used educational tablets in their study performed well on outdoor activities while those who used textbooks excelled on academics (Ylagan, 2013).



This is interesting because the Department of Education of the Philippines is currently experimenting on the use of the tablets on some elementary pupils because of lack of textbooks. I do hope that they will not get shocked when the pupils using them are outside of the classroom playing or doing some sports.

The observation of Jessica is noteworthy because it somewhat opens my eyes on one of he real advantages of technology in the learning process of children. However, empirical data and additional research should be performed to validate such a claim.

Source:

Ylagan, J.M. (2013).  On Learning & Maturation. A Discussion Forum. Retrieved on May 16,2013 from http://myportal.upou.edu.ph/mod/forum/discuss.php?d=44705

Image from http://www.teleread.com/wp-content/uploads/2010/01/image66.png