Friday, April 27, 2012

Teaching to the Test

What does that mean anyway - "Teaching to the Test?"

It's most often used in a pejorative manner, summing up most of what many folks see as wrong about our current public education system. The criticism, as I understand it, is that because of the significant consequences of low scores on standardized tests, educators are motivated to change their methodology so that the primary objective is for kids to score well on the tests, and that raising the actual knowledge level of the kids has become of secondary concern.

A wise person once told me: "The problem with incentive systems is that they work."

The point is that any system created to influence behavior will function as designed if the incentives - positive or negative - are great enough. The challenge is in creating a 'scoring' system for an incentive system which accurately relates consequences to the desired behavior, while advancing the objectives of the larger organization.

For example, you have three kids in your household, one who keeps his/her room pretty neat and tidy, but the other two are slobs - clothes and trash everywhere. Finally, you get tired of the smell, and tell the two: "If you keep your rooms neat for one month, I'll buy you each an iPhone."  That's a pretty meaningful gift for most kids, and there's a good chance that the two slobs will do exactly what you ask, at least for one month.

But what about the neat kid?  One reaction - the one the parents are assuming - might be for the neat kid to think: "I'm glad our parents are finally getting my sibs to clean up their rooms, it was getting pretty disgusting."

Or it might be to think: "Gee, I'd like an iPhone too, and it seems like the easiest path is to make my room a mess as well."

That's called an 'unintended consequence.'

Testing has always been a part of education. The little kids take simple tests to evaluate their readiness to enter kindergarten. Teachers make up tests to evaluate which kids are learning, and which are not, as well as to figure out whether the teacher's approach to a topic is working. That is, if a test indicates that most kids are grasping a concept, then maybe just the few kids who aren't need some special attention. On the other hand, if few of the kids are showing understanding of a concept, perhaps the teacher needs to use a new strategy. Testing is a valid method for the teacher, students, and parents to give and receive feedback as to how the students are doing.

When I was in school forty-odd years ago, I don't recall there being very many standardized tests. Certainly tests made up by the teachers were a constant - I'm sure I took hundreds of them in my 12 years of public education. But there definitely was no such thing as a 'state graduation test.'

I think this is the difference of then versus now:  I don't believe that decades ago the results of those standardized tests were used to evaluate the performance of a teacher, a building (ie the administration), or the school district. At least not in any public way, or in a way that motivated teachers and administrators to materially change their approach to teaching.

Today, it seems that teachers are tasked with not only conveying subject matter knowledge to their kids, but also coaching the kids in the use of strategies that help them perform better on the test.

There are high stakes associated with test scores on all the standardized tests, which are administered here in Ohio from third grade on. It's not just the students who get evaluated. The buildings individually and the school districts as a whole also get a standardized report card, prepared by the Ohio Department of Education.

To call these state report cards 'standardized' is a bit of hyperbole in my opinion. They've been tweaked a number of times since first being created a few years back. The motivation for change hasn't always been to gain a clearer picture of how the school district is performing as much as it has been to modify the political message.

As I have been writing for years, suburban school districts like ours are being required to fund an ever-increasing fraction of our school budget with local money - all property taxes in the case of our district. That means putting an operating levy before the voters every 3 years or so. Depending on how good of a job we do controlling expenses, these levies will be larger or smaller, but chances are that there will still be one of some size on the ballot every few years - at least if the scope of programming and services in our district remains about the same.

It's tough to get a levy passed if the State Report Card says a school district is not doing well. You can bet that there is plenty of lobbying on the part of the Ohio School Boards Association, the Alliance for High Quality Education (representing suburban districts, including us), the Ohio Education Association (the teachers' union), and all the other special interest groups formed in connection with public education - all with the mission of getting a State Report Card structure crafted that helped get levies passed, or a least doesn't get in the way.

That's the reason more than 90% of Ohio's 600 school districts are currently rated as "Excellent" or "Excellent with Distinction."

The Ohio Department of Education is now bringing forward yet another State Report Card structure, this time using letter grades from "A" to "F" to rate districts, on the theory everyone who has ever received an individual report card will understand this scale better than the "Excellent" etc system in use now.

The trouble is that no school district in Franklin County would receive an "A" if the proposed, tougher scoring system were to be applied, and that has caused great concern. We would get a "B." After being rated "Excellent with Distinction" for four years running, we suddenly would be knocked down to second-class status.

A public relations nightmare.

Not to fear. Already changes are being negotiated, as reported in a recent story in the Columbus Dispatch. What that means is that pressure is being applied to key politicians, including the State Superintendent, and deals are being crafted. We'll see how much this new system, if it actually gets implemented, is 'softened up' to prevent alarming the voters. Even if some alarm is warranted.

If we're having this much trouble sorting through the politics of using standardized test scores to evaluate whole school districts, imagine the level of struggle associated with trying to use standardized test scores to evaluate individual teachers.

We'll leave it to another time to discuss the merits and problems of measuring teacher effectiveness with standardized tests. The point here is that if standardized tests are used, and the test results have significant influence in terms of teacher compensation and working conditions, then the teachers will be powerfully motivated to instruct their students in a manner that maximizes standardized test scores, regardless of whether that results in a 'thorough and efficient' education for our kids. I'm not sure that's a good thing.

The problem with incentive systems is that they work.


  1. Paul,

    Did you see the latest in the Dispatch recently about the possibility of a new evaluation system that may be implemented? There would be grades for Student Achievement, Student Growth and Gap Closing (explained in the article) all figuring in the Overall Grade. The state would get a waiver from the federal No Child Left Behind Act with the implementation of the new system. Report Card grades would drop significantly (based on last year's performance). Hilliard and Dublin would have gotten a "B", with Upper Arlington and Bexley garnering a "B-".

    In our district, we would see a wide range. Darby would have gotten an "A", Bradley a "B", and Davidson a "C-". At the elementary level, Washington would have gotten an "A", Hoffman Trails and Horizon an "A-", and JW Reason a "D-", among others.

    1. We have to remember that these things as subject to the same political forces as any other legislation. I don't think it's necessarily a bad thing to "stretch out" the scale a little, but it creates a public relations challenge. That is, the community needs to be educated why the grading comes out the way it might, and what has to be done to improve.

      Kinda the same problem as the kid with the 4+ point GPA all through high school that struggles to keep a 3.2 GPA in college. The kid didn't get dumber - the material and the grading got harder.