Testing Philosophy

Criterion-Referenced Testing

An alumnus offers to finance a trip to a national track meet for all team members who have the ability to compete at that level. The track coach holds tryouts. He picks everyone whose time in the 100 yard dash falls above the 90th percentile. This is norm-referenced testing. His requests for funds is rejected because he does not know whether his runners can compete at the required level.

He runs a second try out, having established that 10 seconds in the 100 yard dash is competitive in the event. He now picks those who run the dash in 10 seconds or less. This is criterion-referenced testing. He knows the runners he selected can compete. He gets the funds.

Valpar's products rely heavily on criterion-referenced testing, as opposed to norm-referenced testing. Criterion-referenced testing, unlike norm-referenced testing, uses an objective standard or achievement level. An evaluee is required to demonstrate ability at a particular level by performing tasks at that degree of difficulty. Scores on criterion-referenced tests indicate what individuals can do — not how they have scored in relation to the scores of particular groups of persons, as in norm-referenced tests.

Norm-referenced tests compare an individual's performance to the performances of a group, called the "norm group." For the results to be meaningful, it is necessary to know the specific composition of the norm group. For example, if an individual scores at the 87th percentile on a math test, what can you say about his or her math abilities? Nothing, until you know something about the norm group. One conclusion will be reached if the norm group is a collection of fourth graders. An entirely different conclusion will be reached if the norm group is a collection of university seniors majoring in physics.

Even knowing about the norm group is no guarantee of proper test interpretation. The TABE is a norm-referenced test published by McGraw Hill. It reports test performance in terms of grade levels, and it gives scores as high as grade 12.9. An analysis of the test items reveals that none of the questions are above the eighth grade level (no algebra, no geometry, no trigonometry), so what does a score of 11 mean? McGraw Hill says it means that the individual performed on the test like the average 11th grader, NOT that the person is doing 11th grade math. Many users of the TABE incorrectly make the second interpretation. The TABE relies on the notion that an 11th grader can do more simple problems in 15 minutes than an eighth grader can.

Criterion-referenced testing avoids all this confusion. Concrete criteria are established and the individual is challenged to meet them. For example, in a math test, the 11th grade proficiency criteria might be to solve algebra, geometry, or trigonometry problems. The test will give such problems and award a score of "11th grade" only if the test taker succeeds on these particular problems.

MTM — Methods-Time Measurement

MTM engineering is a well established method of analyzing work tasks to determine how long a well trained industrial worker will require to complete a certain task at a rate that can be sustained for an eight hour work day. It is the most widely used and reliable such system in the world.

Most Valpar work samples have been analyzed with the MTM method, so an individual's performance can be compared to a competitive industrial standard. This gives an accurate quantitative judgment of a person's ability to perform a job, and is particularly valuable in an employment screening process or a return-to-work determination.

MTM scores also provide a good way to track improvement over time, as might be done during an injured worker's rehabilitation.

[Home]

Copyright © 1999 - Valpar International Corporation; All Rights Reserved.