Friday, August 30, 2013

The New State Report Card

It's out, but what does it mean?  Should we puff up with pride the way we have been able to with our string of "Excellent with Distinction" ratings?  Or should we freak out because there are a couple of "D"s on our new report card?

Two answers:
  1. If the previous state report card evaluation criteria were still being used, we would have achieved an Excellent with Distinction rating for the sixth consecutive year - ever since this the creation of this rating category;
  2. We don't fully understand how to interpret the new report card yet. 
I hope we will spend some time as a Board having Dr. Marschhausen and the curriculum team walk us through this new rating system. For now, perhaps the best I can offer is a comparison with some of the other central Ohio districts: Upper Arlington, Westerville, Worthington, Olentangy, Pickerington, and Dublin (click on each to see their report card).

There is no overall grade yet - that won't be coming until 2015. I imagine that there is still some debate going on as to how each component should be weighted. Perhaps it is also wise to force folks to look at the components individually the first time around, as once the overall grades start being presented, few will bother to look at any more detail - just like most folks didn't look past Excellent with Distinction.

As with the previous report card, this new approach starts with a set of 24 state standards. This is one of those "idiot light" indicators. It's either on or off, okay or not okay - like the dreaded "Check Engine" light on our car's dashboard which can mean one or many things need attention. This set of indicators is mostly about the standardized tests. An "ok" means 80% of the students scored a rating of 'proficient' on the state test. Some feel the bar for being called 'proficient' is set too low, and that's another debate which needs to be had.

On this dimension, Hilliard Schools and all the comparison districts hit 24 of 24 indicators, and therefore received a letter grade of "A" for this component. Good start.

Next is the Performance Index, another component which is carried forward from the prior report card. This one again deals with the standardized test scores, but is an indicator of the distribution of scores. The higher the PI, then the more students who scored at higher levels on the test. All seven of the districts were rated "B" for this component, and none were all that close to getting an "A."

We all got "A"s for Overall Value Added. This is an indicator of whether the students are gaining a year's worth of knowledge in a school year.

Where we are differ is when the Value Added Score is isolated by various subgroups. Hilliard was the only one of these districts to be graded a "D"  for the Gifted students. Worthington was the only one to get an "A" for this subgroup. This one concerns me, and I want to understand more about what it means.

For Gap Closing, or Annual Measurable Objectives, we also received a "D", but so did Worthington and Dublin. Of this group, none received an "A". This measurement dimension seems to follow the philosophy of No Child Left Behind, in that a high grade requires that performance is high not only from a district-wide perspective, but also that each of the defined subgroups of students are progressing as well. In other words, issues in one minority segment of our student population can't be masked by the excellent performance of the majority.

Several things will happen in the coming months. One is that we'll need to continue to analyze what the new report card means, and to figure out what we should be doing in response. Trust that there will also be a political response to this new report card as more of Ohio's school districts find out how their communities react to this new body of information. I'm very glad we don't have a levy on the ballot this fall.

If you haven't already done so, I recommend that you read Dr. Marschhausen's commentary on the new Report Card, as well as the story which appeared in This Week Hilliard. In particular, I appreciated this comment from our Superintendent:
[Marschhausen] said while the report card is "one tool to evaluate" a district, no rating system is perfect, and state report cards "can't tell the complete story of a student's educational experience (or) define the effectiveness of our educators."

I agree.


  1. Paul,

    I do not understand the lack of comments on this topic. Either nobody cares or they are ok with the results. I vote for everyone is OK, as they reviewed the posts by Dr. M.

    His frank, open and honest approach is what people are ok about.

    The lack of propaganda is what I like. No spin!!

    This another example of why he is the right guy at the right time.


    1. Dave: Thanks for your support of Dr. John - we certainly like what we're seeing.

  2. Paul,

    I have also been surprise by the lack of comment on this post - this is important stuff. I have been closely following the assessment and reporting details of Ohio K-12 education for that past few years and am always interested in hearing thoughts of others.

    A common criticism of the Ohio OAA and OGT state assessments is that they are not “properly aligned” with the state curriculum (for every grade) and that the minimum proficiency levels are nowhere near national assessment standards. For example, for most districts like Hilliard, the 2013 5th grade OAA results for both math and reading were significantly lower than the 4th grade and 6th grade results – why the discontinuity in performance just for that grade?

    On average, students only need to answer 37 percent of the items correct on OAA/OGT state assessments to pass and be considered “proficient” according to 2012 ODE data.

    So how do our students do against national standards? For the last 3 years, Hilliard 8th and 10th grade students have been taking a version of the ACT (ACT-EXPLORE for 8th graders and ACT-PLAN for 10th graders) that can help determine if they are on a “college readiness path”.

    The table below is for data that I compiled for 2011/2012 that compares how those students performed on both OAA/OGT and ACT-EPLORE/ACT-PLAN testing in that same year. As you can see, there are significant difference in results between state national assessments for our students – barely 50% proficiency in math and reading in both grade, and science as low as 25% proficiency for our 8th grade students.

    |Hilliard 8th Grade Students |
    HCSD 2011/2012 |----------------------------|
    District Test Results | Reading | Math | Science |
    State OAA Proficiency | 93% | 94% | 86% |
    ACT-EXPLORE 8th grade | | | |
    College Path Benchmark | 55% | 56% | 25% |
    Testing | | | |
    source: Hilliard District Office and ODE

    |Hilliard 10th Grade Students|
    HCSD 2011/2012 |----------------------------|
    District Test Results | Reading | Math | Science |
    State OGT Proficiency | 94% | 93% | 90% |
    ACT-PLAN 10th Grade | | | |
    College Path Benchmark | 65% | 53% | 37% |
    Testing | | | |
    source: Hilliard District Office and ODE

    Districts who do this type of ACT testing use it for internal assessment purposes and do not usually publish results. (I annually request our district’s ACT related data)

    Beginning in 2015 the state report card will begin to include similar “college and career readiness” measures for each district. In addition, state OAA/OGT assessments are planned to be replaced with the new “PARRC” assessments that will be more fully aligned with the new Common Core Curriculum, and with the minimum proficiency level raised much higher and closer to nationally standardized assessments.

    Without quick and major academic improvement in the next 2 years, our district’s state proficiency rates are likely to drop significantly and similar to the data shown in the above tables.

    If you are a school parent, how do you know if your child is on a “college ready” path? What kind of assessment data for your child would you find useful to help track and support their academic progress, and how often would like to receive updates.

    I would be curious in hearing other folks thoughts on this.

    Steve B.

    1. Thanks Steve - I appreciate all the time you spend on this analysis.

  3. PAul,

    Looks like the formating of the table did not make it! Here is the same data in a different format.

    Hilliard 8th Grade Students
    State OAA Proficiency:
    Reading-93%, Math-94%, Science-86%
    ACT-EXPLORE College Path Benchmark:
    Reading-55%, Math-56%, Science-25%

    Hilliard 10th Grade Students
    State OGT Proficiency:
    Reading-94%, Math-94%, Science-90%
    ACT-PLAN College Path Benchmark:
    Reading-65%, Math-53%, Science-37%

    Also forgot to mention that Dr. Marschhausen is actively looking at providing an alternative and more meaningful report on district progress to parents and community stakeholders. This report would include other measures than just OAA and OGT data.

    I think that students in our district will benefit greatly under his leadership in years to come.

    Steve B.