<p>One of the problems with high stakes attached to testing, is that tests are only as good as the makers of them. If nonsensical, unanswerable questions are not recognized as such by the testmakers, what does that say about their critical thinking skills?</p>
<p>I’ve been drafted to review exam items, and more recently to write them. It is nightmarishly difficult to make certain that you are assessing what you want to assess, and only that one thing. Because so much is proprietary, it is difficult to get a “real world” evaluation of the questions before the exam is published. I ran some of mine past the Happyfamily, but even that was stretching the bounds of my contract.</p>
<p>Bless all those kids who recognized nonsense when it was in front of them! And bless Pearson for choosing a reading that was not stodgy, even if they couldn’t get the questions right.</p>
<p>Oh, I see. In order to fully understand the question, the 8th graders must be smart enough, or experienced enough in life to have a grasp of the expression about having "a trick up one’s sleeve. Easy as pie for those of us that remember Rocky and Bullwinkle. But if these particular students are completely unaware of the expression, the question would make no sense.</p>
<p>Last Wednesday evening, here in NY State, I was teaching a dozen 8th Graders from public and private schools, at a Jewish afterschool. We were reading some Holocaust literature together, the day before Holocaust Remembrance Day. As you can imagine, the classroom mood was very serious. </p>
<p>Suddenly, one boy raised his hand and asked me, “But Mrs X … DO pineapples have sleeves?” I felt deeply annoyed at him for – I thought – deriding our class material. The whole class burst into heated discussion about owls and rabbit races. I thought I’d gone through the Looking Glass. </p>
<p>But I quickly surmised they were talking about their ELA (English Language Arts) NY standardized test. I forgave the disruption and assumed, since it was ELA, it must be something really stupid that had bothered them. When I read the test text later on an NPR radio transcript, I saw a very poorly written story and questions without foundation from the story.</p>
<p>I was amused to read that the original story-author had submitted a much better text a year before. With the small paycheck received, he’d taken his wife out to dinner. His story, a healthy satire of the Tortoise and Hare race, had an eggplant, not a pineapple, and fewer meaningless side characters. As he recalls, his wife ordered eggplant at their celebratory dinner. He was diplomatically amused at how NYState had rewritten his story for their test.</p>
<p>Of course–it’s not just do we know whether pineapples have sleeves or not. Many of the questions were serious attempts to parse nonsense as if it were serious. My husband and i picked different answers for several of them.</p>
<p>It’s a start that this part of the test was stricken from their scores, but how much time did earnest students spend trying to answer non-answerable questions, and how did that experience affect the rest of their test-taking?</p>
<p>Happymom–guess I can’t begin to bless Pearson for a “non-stodgy” story, placed in the midst of an inherently stodgy situation. The only answer possible for most of these questions is, as Pinkwater says, laughter. But it’s a shame that students, who know that, are reduced to frustration trying to pick the “right” one from impossible choices.</p>
<p>And bless the NY state kids for speaking out–this passage has been making the test rounds for several years now.</p>
<p>Pearson and their ilk make a ton of money from these things. The only criterion, as profit-making businesses, is: does it sell? And apparently it does.</p>
<p>This was stupid, but imo, the much greater stupidity is in the actual grading of writing. It floors me that a student with three terrible arguments in response to a prompt, and a number of factual mistakes, can still earn a higher score than the student who crafts a beautiful paragraph with two well-chosen, beautifully articulated arguments and no errors of fact. But if you look, that is exactly what happens on many state writing tests.</p>
<p>I used to score tests for Pearson. My favorite was a second grade story problem about bears getting in a boat to go to a picnic. The kids were supposed to write a sentance with the answer. The correct answer was supposed to be something like “there were six bears in the boat,” but kids could get credit for just writing “6”. We were grading tests from Mississippi and things went fine until we got to a school district obivously on the water some where. All the kids wrote something like this “NO 6 bears in boat. Mississippi Park Ranger say NO 6 bears in boat it is not SAFE.” All these answers were given no credit because they wrote NO 6 and the muckety-mucks intrepretated that as the kids were saying the answer was not 6.</p>
<p>Wow, I had no idea how to answer some of these questions. I also scored in the 99th percentile on the verbal portion of the GRE. I think that says enough right there.
(Not trying to brag, just put things in perspective.)</p>
<p>It was bad enough that they mangled the story, but the questions were ridiculous - I had no idea what they were looking for. I’ve always gotten top scores in verbal tests. </p>
<p>This is not a first for NYS - long ago my kids had an official practice test for the 4th grade reading test. The story was about animals on a merry-go-round and which ones didn’t belong because they weren’t native to New York State. There were seven wild animals and one domestic one (a cow I think). One of the wild animals was clearly not a native animal, and I am sure that is the answer they wanted, but cows were brought over by the Europeans, and was clearly also a correct answer.</p>
<p>I’m not a big fan of Pinkwater, or this sort of nonsense writing.</p>
<p>I read a while back about a question withdrawn from a standardized test. It went something like this, as best as I can remember:
A man has $35000 to buy a yacht. They are sold only in 10 foot long sections, at $10,000 per section. How big of a yacht can he purchase?
A) 30 ft
B) 35 ft.
C) 40 ft
D. Not enough information</p>
<p>The question was stricken on the grounds that it was an unfair question to black testtakers.</p>
<p>Wow, what an absurd story & even more absurd revision. I can see why the kids didn’t like or “get” it & wonder why it ever got put on ANY test. The 6 bears question also is crazy! Evaluating & scoring writing is very challenging–did it as a parent when I was active at my children’s grade school.</p>
<p>Pearson has a completely machine scored exam for English proficiency that is marketed as a competitor of the TOEFL. Just imagine how that will score answers like “NO 6 bears”.</p>
<p>I still think it is nice that Pearson’s writers tried to jazz up the reading section a bit. However, the heavy hand of the editors is obvious in the conversion of the eggplant onto a pineapple, and is probably responsible for the stupid questions as well. I’ve seen truly stupid editing in questions I was asked to review, and our team was told that we were the last round! I couldn’t believe those mistakes had gone past at least three previous levels of review.</p>
<p>When my oldest D was in 4th grade she came home from a day of standardized testing and told me that one of the questions was something like this:</p>
<p>If a pencil is standing on it’s tip and a light source is on the left of the pencil, in which direction will the pencil’s shadow fall?</p>
<p>Well…If a pencil COULD stand, UNASSISTED, on it’s tip, then the answer would be that the shadow would fall on the right of the very, very special pencil. </p>
<p>My D left it blank because she thought it was a trick question.</p>
<p>All I could think was “what a bunch of A$$es.”</p>