Saturday, December 15, 2007

Value-Added Evaluation

As readers of this blog know, I make no claim to understand the art and science of teaching. My focus is on the governance and funding of Hilliard City Schools - in other words, on making sure we have the right folks in leadership and that the system has the resources it needs to deliver the performance we in the community expect.

There is an intersection between these two elements - the feedback to the community about how our schools perform, measured against a set of standards applied to schools all over the state. We want to know as parents that our kids are getting a good education, and as taxpayers we want to find out if our money is being well spent.

The 'State Report Card' is one mechanism for answering those questions. Now the State of Ohio is adding another performance measurement called the 'Value Added measurement.' The Columbus Dispatch recently ran articles about this new measurement. Hilliard City Schools was given the top rating, GREEN, meaning that our schools are exceeded what the state would expect, according to their complex formula.

Dave, an educator in southwestern Ohio, writes the blog Into My Own, and in his blog has written an excellent series on the Value Added measurement system. You are encouraged to read it.

Thanks Dave.


  1. Ahhh... So we have another element on the State Report Card to try and digest and understand. They seem to be getting more complex.

    So first we measured performance by counting the number of standards met by the district. Then we introduced something a little more complex, the Peformance Index Score, but still pretty easily understood once the calculation was explained. Then, we throw in Adequate Yearly Progress along with District/School Improvement status. And if that wasn't complicated enough, let's throw in the Value-Added measure.

    My concern here is that the these calculations are getting increasingly complex, yet we assign a very simplistic designation (A, B, C) to describe how a school district is doing. These simple designations however don't seem to really convey what is going on and they seem often be misleading or at least, confusing.

    I noticed that Worthington City Schools got pinged on this measure (getting tagged with a "RED" value-added designation), after missing AYP again earlier this summer. If you look at the number of indicators and Performance Score of Worthington, you would conclude one thing. Look at the AYP and this new measure and something different seems to be indicated. Overall, Worthington is rated 'C' with a "RED". Yet, from almost all accounts, Worthington CSD provides a high-quality education. How much more can these numbers be massaged before parents totally give up and conclude that the State has concocted a system to confuse everyone?

    (This is coming from someone who is really trying to understand what all these numbers mean and their significance.)

  2. Back in the years when I was involved in the annual planning effort of a large corporation, we used to joke that our CEO practiced 'Jeopardy Budgeting' because he already had the answer in mind - our job was to come up with the questions that fit.

    As I've become more aware of politics, at the local and state level at least, I realize that it works much the same. The lobbyists and special interest groups have an outcome in mind, and set about influencing lawmakers to create legislation that leads to that outcome, usually in a fairly obscure way.

    For example, I think the Big Darby Accord is being sold as a environmental conservation program, when what it really does is extend the area in which developers can build homes in suburban school districts without burdening the City of Columbus with the cost of streets, police and fire protection.

    The desired outcome is for developers to get water/sewer services without blowing up the Win-Win Agreement, nor yielding additional territory to the suburbs and potentially cutting off Columbus growth.

    I think these school measurement systems have been politicized in the same way. The idea is a good one - figure out a way to objectively measure school system performance. It should be valuable information to both parents and state officials.

    But I think the lawmakers already know what answers they want the measurement systems to produce - the answers that get more funding to the schools in their districts. For example, divert more money to Columbus City Schools because we've finally found a measure that shows that they're making good use of the money we've sent their way.

    Worthington is an unfortunate aberation. What lawmakers want to see are results like Hilliard, where a Green rating is being achieved in spite of state funding cutbacks, because that justifies sending even less money our way.

    There is no research behind this assertion. I'm only describing a scenario which potentially answers your question as to why this school performance measurement system has gotten so complex.

    Politics at high levels is a game of misdirection: make folks think the motivation is X when it is really Y. Tell only enough truth to get buy-in from a public that largely doesn't care.

    I'm less concerned that people agree with me than I am that they get sufficiently engaged to put a stop to this brand of politics in our community.

  3. Here's another one for you and your readers, Paul. Really excellent post that shows why some traditionally underperforming districts are excited about Value Added.

    Anonymous, you're right, this system is almost as confusing as the state school funding system. :) You're not actually supposed to understand it. You just have to take their word for it.

  4. I think the quality of schools has far more to do with the make-up of the students than the make-up of the teachers. Teachers are all graduates with degrees and thus have mostly "succeeded". Students run the gamut from immigrant unable to speak English, to the gangbanger actively hostile to school, to the A-student striving for the Ivy league.

    And we all know that good students are not equally distributed across the state of Ohio. They are lumped together in the richer neighborhoods and have parents who likely have instilled the importance of learning.

    Schools are not like private businesses in the sense that they don't have control over their raw materials. They can't fire a student or students. It's like judging the quality of a beer on the technical expertise of the makers while forcing them use whatever hops and barley and corn, no matter how poor or excellent in quality.

  5. Interesting perspective.

    In many countries around the world, kids have to pass aptitude exams to stay on an academic track past 8th grade. Kids who don't pass and those who choose not to take the tests are redirected to vocational education.

    I don't think that's all bad. A good friend of mine worked in the budget office at OSU for many years. When we got on a conversation like this, he told me how much money was spent at OSU educating freshman students that couldn't add fractions. In my day, such kids were there to avoid the draft, not get an education.

    OSU has gotten itself out of the remedial education business, but the point is that resources are consumed trying to teach kids skills that will have little value in them getting their first job.

    I'm not saying once a kid is put on the vocational track, their whole life is programmed (e.g. the board game "Life"). Sometimes the very best college students are the ones who, while not ready for academics in their youth, spent some time in the trades and then went back to school.

    Our inner cities are rotting because there aren't jobs for all the kids who grow up there. While population control is a critical issue for our society, once the kids are born, there must be decent jobs awaiting them or the effort of school just seems pointless.

  6. I mostly agree, but regarding this:

    ...there must be decent jobs awaiting them or the effort of school just seems pointless.

    The unemployment rate for Ohio is 5.9%, which would've been cause for euphoria any time before the 1990s. We've never had a period in which school was worth more, given the low unemployment rate (by historical standards) and how crucial education is in our post-industrial society. Less than a mile from the inner city there are corporations with job openings.

    I suspect the real problem is the dissolution of marriage and corresponding lack of fathers in the inner city, but there is also severe peer pressure - studious black kids are teased for acting white.