Every once in a while, I read a news story that is so dumbfounding, so ridiculous, so unbelievably filled with morons and tomfoolery, that I have to read it multiple times to even understand what is going on. And with a second reading, and sometimes even a third, and as I finally begin to understand it, my horror at what I’m reading, my pure stunned amazement, my anger at idiotic bureaucratic incompetence, grows by leaps and bounds.
Today, we have “State gets do-over on ISAT scoring; wide swings in results prompted retabulation” by Stephanie Banchero in the Chicago Tribune.
Illinois education officials made the unprecedented decision Friday to rescore the state’s nearly 1 million elementary school math and reading exams after outside auditors determined the scoring process was seriously flawed.
They hope that regrading the exam will temper the wild swings observed in the preliminary results of the Illinois Standards Achievement Test (ISAT) taken by 3rd- through 8th-graders in the spring. Initial scores revealed inexplicable fluctuations of up to 10 percentage points over last year.
The do-over means test results for some students and some schools will increase. Others will decrease.
The state has not publicly released individual student and school results yet, so parents and educators won’t notice the changes. State education officials said the annual school report card will still be released on time Nov. 1.
“These are high-stakes tests, and we have an obligation to get them right,” said Christopher Koch, state superintendent of education. “We have to be able to stand by these scores and ensure they are accurate.”
The Tribune first reported the questionable results last week, raising doubts about the reliability of a testing system used to rate schools, apply federal penalties and put children in summer school.
Dozens of states have had testing difficulties nearly every year since the No Child Left Behind Law required them to test students more often.
OK, time out. What is so complicated about scoring tests? Right answers get a point, wrong ones don’t. Right?
Oh, no. Not even close.
Creating and scoring a state achievement exam is a complicated process that involves complex calculations. Not every question carries the same weight, for example, and each year’s exam is scored just a bit differently from the previous year.
“Complicated process”. “Complex calculations”. Weighted questions. And the cherry on the sundae: “each year’s exam is scored just a bit differently from the previous year”!
That’s just great.
So do all these layers (a) add to, or (b) subtract from, the ability to measure student achievement in an accurate way? I think I know the answer to that one, and it isn’t complicated, or complex, or weighted, or scored differently from one year to the next.
In my world, answers to test questions are either right or wrong. There really isn’t supposed to be lots of wiggle room in there. Unless, of course, they’re using essay questions on these tests.
Hahahahaha! That’s a good one. Essay questions. You crackin’ me up!
The problem this year can be traced to the decision to use a new version of the exam, state officials said.
The ISAT is developed by combining state-created questions with those taken from the national exam called Stanford 10. This year, the state used a new version of the national exam.
When it came time to score the test, the state picked the formula the testing company used. It turned out to be the wrong one, state officials said.
Oh my GOD. The formula used to score the test just “turned out to be the wrong one”!
Come on now, who hasn’t used the wrong formula to score a test before? Scoring tests is, like, hard and stuff!
The result: dramatic fluctuations up and down in the math and reading scores. The state’s grade school science exam did not change this year, and preliminary scores were in line with previous exams.
Koch said state officials and the testing company immediately began looking into the problem when it was detected in late June. The state also asked two independent test auditors to investigate at a cost of about $20,000.
“They both came to the same conclusion, so I felt confident we knew what the problem was,” Koch said.
David Hakensen, a spokesman for the state’s testing company, Pearson, said the firm agreed with the decision to rescore the exam.
“I know you are trying to point fingers,” Hakensen said, “but this isn’t anybody’s fault. A standard methodology was used. Now, we have agreed to use a different one.”
So a standard methodology was used. Hmmm. That’s just great.
Hey, just brainstorming here … what about using the right fricking answers?!
It’s so evil and diabolical, it just might work!
I’ve got another crazy idea, maybe they should try this as a test question: When “experts” in education testing can’t even figure out how to score the tests, how can we, as taxpayers who pay for all this, have any confidence that the results are telling us anything useful? Explain, without using words like “methodology”.
Still, Pearson said it will pick up the cost of rescoring the exam.
Pearson has made numerous scoring errors in the past. The company gave more than 4,000 students lower scores than they deserved on the SAT college entrance exam two years ago and produced a flawed answer key that lowered student test scores on a different exam in Arizona.
I’ve got a sneaking suspicion that this testing company is not the sharpest tool in the shed. And since this is Illinois we’re talking about, a healthy suspicion about corruption and under-the-table deals is always a good idea.
Bob Schaeffer, public education director for FairTest, said Pearson is ducking its responsibility in Illinois.
“They are supposed to be the testing experts,” said Schaeffer, whose non-profit group has been critical of over reliance on standardized tests. “And you would think they would have flagged the flawed methodology before the scores got out.”
In Illinois, results have been delayed and riddled with errors, but this is the first time that serious miscalculations were made in scoring.
These people are educating our children.
But relax, it shouldn’t be a problem, really. They’re from the government, and they’re here to help us!