The Art and Science of Testing:
Why Assessing a Child Is Not Child’s Play
By Dr. Jadis Blurton
Founder and Clinical Director of Therapy Associates, Hong Kong
Declare the past, diagnose the present, foretell the future, practice these acts.
As to diseases, make a habit of two things— to help or at least do no harm.
-Hippocrates, Epidemics Book I
Parents are at their most vulnerable when their child is struggling. They want information, and they trust experts to tell them what is wrong and to give advice about how they can make it better. But they need to be very careful about finding the information they desperately seek and the advice they fervently want: there is both an art and a science to testing, and results are not always what they seem. My first experience with testing was as a parent, not as a professional, and it was an experience I will always remember.
When my daughter, Jessica, was about twenty months old, she had her first IQ test. She sat on my lap as the test administrator brought out the first items: blocks! Jessica enthusiastically grabbed the blocks and started to build, but the tester stopped her and told her to make her blocks look like the tower the tester made. Dubiously, Jessica made the tower as requested, then began to make her own construction again. The test administrator, however, had checked off her item and no longer needed blocks. She grabbed the blocks from in front of Jessica, put them away, and pulled out the next task materials: a cup and a small ball. Jessica was upset at having the blocks taken, but was intrigued by the ball and cup. The administrator covered the ball with the cup, and asked Jessica to find it, which she did. Then she pulled out a couple more cups and began to hide the ball again. Jessica liked this game, and after finding the ball again she was ready to play by hiding the ball from the administrator. However, the administrator had checked off her item, and put the ball and cup away. Needless to say, after a few of these encounters, Jessica was having none of it. In fact, as the administrator put the new material in front of her, Jessica simply picked them up and threw them at the administrator. (Even twenty-six years later, Jessica is not known for her ability to suffer fools gladly.) Pretty much every item from then on was marked with a zero. Of course, the test results indicated that Jessica had a very low IQ, as she was “unable” to complete these items.
Obviously, this test administrator had not mastered the art of testing. She was testing something, but it was not what she had meant to measure (Jessica’s intelligence). She may have been testing Jessica’s patience or compliance or ability to follow directions, but she was not testing whether or not Jessica could figure out the puzzles she was placing in front of her. Luckily, I was in the room and watched the whole proceeding, so I knew not to trust the results. If I had not been there though, as a young mother with a disabled child, I would have been devastated by these results. Here was an “expert” who was telling me my child was very mentally delayed.
When Jessica was six years old, she was tested again, this time by the public school system’s educational psychologist in the U.S. I don’t know if this person had mastered the art of testing, because this time I wasn’t in the room. But I was the person to whom the psychologist explained the results. Jessica had taken the WISC, an IQ test that has several subtests and subscales. On some of them, she had scored above the 95th percentile, in the gifted range. On others, she had scored below the 5th percentile, in the severely impaired range. When you averaged the high scores together with the low scores, the result was 100—a perfectly average score. The school psychologist (Dr. Somebody) explained to me that the school couldn’t possibly make special accommodations for Jessica because she had a perfectly average IQ. When I pointed out that she was not average in any area—she was either gifted or impaired—he told me that it didn’t matter, because it all averaged together. Again, I was lucky in that by that time I was running an evaluation center at Arizona State University and was clear on the fact that this man didn’t know what he was talking about. He didn’t understand the science of testing. So I moved Jessica to a private school that would address her special educational needs.
The Art of Testing
“In order to test kids, it is important to like kids, to be interested in kids, and to understand how to elicit the best performance possible from kids." The skills every teacher needs—patience, a sense of humor, a bit of firmness, a touch of playfulness—all are important in fostering an environment where a child feels comfortable. If a child does not feel comfortable, the child will not be performing at his or her best. And if the child is not performing at his or her best, then what are the tests measuring? The test used in a psychoeducational assessment are complex enough that there is really only one way to score well on them—to score well, a child must have the necessary abilities or skills. But there are always many reasons that a child might score badly on these tests. The child could have a cold or be overly tired, or be shy or scared or overexcited. He or she could have had a fight with a sibling or parent that morning, a cranky car ride, or a nightmare. I once tested a teenager who (as I found out after the testing) was hung over. Emotional problems such as depression, anxiety, or obsessive-compulsive behaviors might affect test results. (Has everyone seen the episode of Monk in which he takes his police exam?) In other words, there are many ways in which psycho-educational testing can underestimate a child’s true abilities and skills. Part of the art of testing is to recognize (and note down) the things that might be interfering with a child’s best performance and to minimize those things if possible. If the child is old enough, the test administrator should always attempt to assess emotional problems that may be interfering with test results at the same time as he or she measures problem-solving and achievement.
There is a lot to know about testing, and most of it does not concern the actual administration or scoring of the tests themselves. In fact, these tests are beautifully designed, so most of them come with very simple-to-follow manuals, scoring guides, and even computer writers, so all the administrator must do is enter the scores in the computer and push “return.” Anyone reading this article, really, could administer and score these tests with a very small amount of training, just as we could all probably look up medications in a Physician’s Desk Reference and find out about how to treat arthritis. So why should only a licensed psychologist supervise and interpret these tests and only a medical doctor prescribe drugs?
Obviously, a good test administrator should know where the tests come from (test construction) and what the statistics mean. That’s just basic. Otherwise, the results cannot be explained and the test itself cannot be evaluated. For example, it seems that the psychologist who tested Jessica when she was six did not understand how the test was constructed to reflect different ability areas that could not be averaged if they were highly divergent.
A good test administrator also has to know what factors are being measured by each subtest, and this is important because the administrator then has to rely on his or her part to eliminate all other factors. For example, there are many subtests that purport to measure processing speed. For older kids, these are fairly straightforward. Younger kids, however, do not always understand the concept of time or what they are doing in a “race” to see how much can be completed in a given period. They can do the practice exercises (four or five of them) very well and very quickly, but if you turn the page to something that contains a hundred problems and say, “Let’s see how many you can complete in a minute,” the child looks at the page and says, “That’s too many. I can’t do that." He doesn’t understand the concept of racing the clock. (This would be similar to entering a library and telling the child that we were going to measure how many books he can read in a minute, only to have him look at all the books in the library with dismay and say he just can’t read all those books!) This being the case, it is useful to play with the stopwatch with younger kids—to see how many times a child can run across the room, or to allow the child to find out how many times the tester can pat her head, or how many times we can both hop up and down in fifteen seconds. With a few of these exercises, the child is much more eager to begin a longer process and see how many problems can be completed in a minute. If we simply allow the child to moan that there are too many problems, we may be measuring maturity or time-concept, but we are not measuring processing speed, which is the purpose of that subtest.
In another example, there is a subtest that I often use that purports to measure working memory. The test is presented on a tape, and has longer and longer strings of items that must be manipulated and remembered. The tape leaves a space of time for the response, but only allows for the amount of time typically taken by the average child. If a child is working slowly, he or she might bump into the next item on the tape before having completed the response. The subtest’s purpose is to measure working—not response—speed. In order to correctly measure working memory, the administrator should pause the tape if the child is taking longer than expected. Otherwise, the child becomes anxious or worried, tries to hurry, and/or simply doesn’t complete the task in the time allotted. If this happens, the test may be measuring something but it will not be measuring working memory. There are actually many test administrators who fail to pause the tape, thereby completely mismeasuring the true working memory of the child. Their reason? The company that produces the test made the tape and therefore they feel they are following the dictates of the company. But the truth is that the company that makes the test explains in its own training sessions that the timing on the tape is arbitrary and that the administrator should (of course) pause the tape if the time is not appropriate for a child. Again, it is important to understand the factors being measured, rather than to simply follow the instruction booklet (or tape) blindly.
Of course, for comparison purposes, there are some very strict guidelines that must be observed in any standardized testing situation. The trick is to know what can be added in or changed, remembering always the basic purpose of each subtest. Jessica’s first test administrator, for example, would not have been bending the rules had she brought out the cup while Jessica continued to play with the blocks and, with a little enthusiasm and playfulness, gotten Jessica interested in the new task before putting away the last. Similarly, playing with a stopwatch before testing does not interfere with the standardization of the processing-speed tasks. It simply encourages the child to attend to them.
Once the administrator has appropriately approached each specific factor being measured, however, it is also important that he or she understand the factors and how they relate to each other and how they relate to cognitive development and learning. The field of cognitive development is complex and interesting, and doesn’t just consist of a bunch of numbers in a chart. There are many theories and much research concerning the development of thought and thinking abilities. How does working memory develop, and how is it related to learning new materials, or to emotion? How is auditory attention related to general intelligence? Is the relationship the same at all ages? Would we expect the same range of phonemic awareness for eighteen-year-olds as we would expect for six-year-olds? These may sound like esoteric issues, but in fact they can make a big difference when interpreting the numbers or making a diagnosis!
And finally, if the test administrator is going to give advice to parents and schools, he or she has to know a little about education. What are the pedagogical controversies surrounding dyslexia, for example? How can a teacher’s job be made easier, not harder, by the recommendations? What is normal behavior in a classroom and what can we do about situations or needs that are not normal? So, if working with teachers, the psychologist needs to have had significant training in or experience with schools. Similarly, if advising parents, the administrator needs to have had significant experience in the field working with children and/or be a parent.
A Final Note
Finally, and this recommendation is true no matter who is providing your testing: trust what you know about your child. No matter how qualified the practitioner, or how carefully the testing was done, the results are based on tests that are not perfect, in a situation that may not be optimal. The results are good guides, and often very helpful in finding reasons for things you may be seeing at home or at school. But if they tell you something that does not make sense to you, they may be wrong. Tests are designed to predict or reflect reality. If they do not match reality, it is not sensible to say that reality is wrong. A good practitioner will be cautious in his or her interpretation of the results, and you should be cautious as well. If I had not been, I would have believed that Jessica was globally mentally impaired when she was two and “average” when she was six. I assure you, now that she is an adult, it is clear that she is not globally impaired and is anything but average.
Along the same lines, take diagnoses with the caution they deserve, and remember that a diagnosis should be helpful, not harmful. One of my favorite quotes is the line by Goody Nurse in The Crucible by Arthur Miller, a story about the Salem witch trials. When she sees the girls writhing on their beds, seemingly stricken by the devil, she says, “I have eleven children, and I am twenty-six times a grandma, and I have seen them all through their silly seasons, and when it comes on them they will run the Devil bowlegged keeping up with their mischief.” (Of course, the town does not listen to Goody Nurse, and the resultant trials kill many innocent citizens!) Goody Nurse’s point is that children do have transient periods of odd or dysfunctional behaviors. It is important to address them, one by one, for what they are, not for what they might be or could turn into or could signify. It is important to proceed with caution and love, forcefully and intensively addressing the problems that have been clearly identified, but without attempting to second-guess what these particular problems may mean about the future of your child. The diagnosis does not define your child: it should be meant to clarify and help, not to limit or dismiss. “Don’t turn a silly season into a witch trial,” and don’t allow others to do so either, no matter how professional or expert they may seem.