The Intersection of Technology and Test Scores

September 13, 2011

“In Classroom of the Future, Stagnant Scores” blared the headline in New York Times on September 4th. The paper’s editors decided that the top-of-the-fold story on Page 1 also warranted two full pages inside, plus four color photos and a graph. That’s a huge part of the news hole on any day, but particularly on Sunday, when circulation is at its highest.

The long piece is worth reading, but at the end of the day what stood out for me was what the article failed to take note of: the unimaginative uses of the technology, essentially digital versions of routine stuff: One teacher gave a true-false quiz but handed out wireless clickers for students to record their answers. In other classes, kids were playing a math game (“Alien Addition”) and an interactive spelling game, while other students were videotaping a skit that they could as easily have simply performed for the class.

In none of the examples presented were teachers using the technology to burst the boundaries of their classroom to connect with students in other cities, or even elsewhere in their district. None were using the Internet to do original research. I’ve written about this before, and Learning Matters producer John Tulenko helped craft a great piece related to the topic:

It seemed to be all about entertainment or delivering more efficiently what the adults had decided the kids need to know, rather than allowing and encouraging students to follow their own interests — at least occasionally. I fault the reporter for not drawing that distinction and for not pressing the adults who are spending all this money on the paucity of imagination.

But my real point is that the Times reporter could — and should — have written a very different story:

“Schools spend billions on technology but use it to do the same old stuff in more entertaining ways!”

Why is this happening, the reporter could have asked? Is it because teachers don’t understand the technology’s power, or because they want to make sure the kids learn what the adults have decided they must learn — or because they are ruled by fear of low test scores?

Running throughout the article is a constant refrain about the limitations of test scores. Adult after adult complained that “Test scores were not an adequate measure of the value of technology” but then went on to say, in effect, “Well, that’s what we have to live by.”

That really gets my dander up. They are endorsing spending billions on technology — it’s not their money — and they complain about the tyranny of bubble tests, even while their pedagogy is focused on test scores.

If they understood what today’s technology can do, and if they were enabling their teachers to go there, and if scores were still stagnant, that would be a story. (But the story might now be about how inappropriate bubble tests are to measure this new learning.)

Something must be done. The Times reports that school systems spent $1.89 billion on software in 2010 and perhaps five times that amount on hardware. That’s real money, especially at a time when school districts are going to four-day weeks, cutting art and music, eliminating Advanced Placement classes, and making other draconian cuts.

And then this expensive technology is used in woefully unimaginative ways!

Establishing a ratio of dollars for training to dollars for software and hardware is not the answer, because there aren’t sufficient incentives for teachers to try new approaches — at least not as long as their main job is to get those test scores up.

To find the solution, go back to the whining mentioned above, the constant complaints about the lack of adequate measures.

That brings me to a conversation I had last week with a leader in the reform movement. I asked his thoughts about the erasure scandals in Atlanta, Washington D.C., Philadelphia and elsewhere. He said it was a wake-up call and a clear message that we need better security. “Since those scores count for so much,” he said, “systems have to do a better job of protecting the tests.”

He’s not alone. A few days ago a panel of experts in New York recommended tighter security, including giving all tests on the same day and requiring proctors to certify that they have been trained in ‘security procedures.’

Wrong, guys! That barn door is off the hinges and the the horse is long gone. As long as adults’ jobs and students’ promotions and graduations are determined by test scores, there will be cheating. Students can use wireless devices to share answers, for example, while ‘fully certified’ proctors can still nudge nudge wink wink their way around the room, helping students pass.

We ought to be searching for multiple measures of academic progress, measures that are valid, reliable and reasonably affordable.

Who should be doing the searching? Wonderful as the U. S. Department of Education’s i3 ‘innovation’ grant program sounded, it was never set up to support risky investments of the sort I think will be required. It bet on such ‘innovations’ as Teach for America and KIPP, and that’s fine, but what’s needed here is some real risk-taking.

I have three candidates:

  1. The companies now making megabucks on testing, Pearson and McGraw-Hill, ought to be protecting their revenue stream by finding better ways.
  2. Apple, Microsoft, Dell and others hawking their products have a strong interest in public evidence of the power of technology

But the best candidate might be the NewSchools Venture Fund, who I think are the brightest folks on the block. That organization has never been shy about taking chances, probably because it exemplifies the spirit of its founder, John Doerr. In the Venture Capitalist world, only a small percentage of investments hit a home run, and the NSVF gets that. It’s putting dollars behind a number of new approaches to teacher training, for example, in the expectation that some of them will be a distinct improvement on the current approach — while others will fall short.

(I don’t know how NSVF finances work, but maybe Apple, McGraw-Hill, et alia should be making large donations to that organization?)

We need that venture capitalist mentality and approach to the world of measurement. So what if most of the schemes don’t pan out, as long as we emerge with a few that actually work?

This matters because right now school systems have almost no incentive to trust technology — because they don’t know how it will affect those test scores.

Look, educators are excessively literal and overly reactive. They haven’t gotten where they are by taking chances, so don’t expect them to take the lead now. Society has been telling them that we want good reading scores (we haven’t said, “we want kids who love to read,” just good reading scores). So why are we surprised when they drill kids on reading tests?

Bottom line: schools will never realize the power of technology until they get out from under our current way of holding them accountable. We need accountability, but what we are now doing is stifling learning and teaching. It’s making public education worse, not better.