Print
Return to online version

November 30, 2009

What Might Merit Mean?

Justin Katz

In a comment to my liveblog post, Thomas Schmeling asked me to "provide some information on the 'merit system' of compensation that [I] support." The short answer is that I don't have a tremendous amount of detail to put forward.

For one thing, I volunteered for the Tiverton School Committee Subcommittee on Evaluations in part to develop my understanding of the various considerations involved, and we have yet to meet. For another thing, a fair portion of he details should be left to administrators to hone according to the actual forces and dynamics in their own districts.

Basically, I just support the idea that compensation and professional advancement should be related to capability — not longevity. (Although one would expect longevity to result in escalating capability in most cases.) I don't think the one-time bonus structure is very effective, especially if the bonuses are small, and group bonuses probably wouldn't prove very effective unless the groups are very small. There should be an individually based spectrum, ranging from firing and probation to raises and promotion.

The "afterthought" of my earlier post included a sketch of factors that would be considered while adjusting pay, but the long and short of it is that everything should have an effect, from standardized scores to demonstrable extra work, to student and parent reviews, to peer reviews. By some process that suits the school and district, administrators would factor in these various considerations — a good amount of which could easily be incorporated via objective scales — and produce an annual raise and promotion result.

Comments

Well, what are we trying to achieve with merit-based compensation? What's the current problems that we're aware of? Most people refer to the standardized tests. If performing better on the tests is the goal, then reward that. If getting more kids into college is the goal, then reward that. If having a higher student retention rate, or graduation rate is the goal, then reward that.

If a teacher gets students who are performing three grades below where they should be and then at the those students are only one year behind, that's progress to me. Some would say the student is still a year behind, but I'd say that teacher did a great job.

I think the other thing that would help is to divide the students by their abilities better, if that's possible. When a teacher has 25 students and 5 are advanced, 10 are really lacking and 10 are average, then how do you challenge all 25? Split 'em up.

Just getting the conversation started...

Posted by: Patrick at November 30, 2009 3:01 PM

We already calculate "value added" -- the difference between test scores predicted by the socio-demographics of the students in a class, and their actual scores.

I'd even be willing to weight this 25%, along with a principal evaluation, peer evaluation (providing they graded on a forced curve), and student evaluation.

As long as we routinely cut the bottom 10% (just like in law firms, accounting firms, and other professional organizations), I don't care on the weighting scheme. But Value Added must be in there, because it is, in my view, the most telling (and frequently damning) statistic we have -- and it has a history to boot.

But the teachers unions will fight this tooth and nail, just like they fight charter schools, their membership will remain silent sheep, and RI will continue to drive businesses and middle class families away, the public sector pension funding gap will worsen, and the RITanic will continue to sink.

I hope the state's teachers don't expect much sympathy when their pension checks start bouncing.

Posted by: John at November 30, 2009 10:14 PM

Merit for teachers should be based on one thing only - student performance. Test scores, graduation rates, maybe even attendance. But that’s it, keep it objective, straightforward, and simple.

I would very strongly disagree that merit should be based in part on any degree of preparation, any sort of peer review, or student review. The former is what we already have and has been a colossal waste of taxpayer money, the later two are just popularity contests and are not worth a damn.

Considering what is at stake, and that an assessment system is already in place which provides this information – though not through all of the grades and in the higher grades not enough in the particulars, it should be possible to assess a year’s worth of learning for each student in each classroom and to then determine a teacher’s worth. Basing a teacher’s performance on anything besides what the students in their classroom are learning makes no sense and would be a complete waste of time and money.

Posted by: Frank at December 1, 2009 7:58 AM

What evidence exists that merit pay improves teacher or student performance? Given the preponderance of evidence against the efficacy of these programs, it's no surprise that you don't have a "tremendous amount of detail."

I've previously cited studies showing it's harmful in education and cited quality improvement professionals, who as a group have railed for decades against the practice in western management. Why waste time and money on something that demonstrably does not work?

Our results consistently indicate that the increased focus on individual teacher performance caused a significant and sizable relative decline in student achievement, as measured by national exams. However, the decline in achievement is less sizable when considering school-level results, suggesting an increasing importance of grade inflation.
Posted by: Russ at December 1, 2009 10:59 AM

Frank, you say attendance. Are you referring to thatvof the teacher orvthe student. Graduation rates would only be effective to measure teachers in high school and even that would only be for seniors. The big problem with test scores is that they mean very little to the students. The younger kids have nothing on the line and a segment of the population does not even take them.

Posted by: Skippy at December 1, 2009 11:05 AM

If you are going to pursue merit pay then make sure it is merit pay and not just another expensive pathetic excuse to pay teachers more than we already do. Make sure that “merit” is not indistinguishable from seniority, preparation, and peer friendships. We have that now and everyone agrees that it is not improving the effectiveness of public education.

If we are going to try merit pay for teachers then make sure it’s based on student outcomes, nothing else. If it’s not working at first then change it and measure student outcomes until you find something that does work. But don’t lose sight of the purpose of education which is student outcomes, not which adult in the school system agrees with who’s methods, or which theoretical model sounds best, or which teacher got which certification – none of that is improving public education, as we have all seen.

For those looking for proof that merit pay will improve student outcomes, I say what other reasonable choices do we have? It is highly likely that many merit pay systems would not work, especially if they are indiscriminate, rewarding nearly all regardless of the results of their efforts. It is doubtless that there are and will be systems of rewarding teachers based on student performance that will improve learning, the trick is to find it ASAP!

You can stick your head in the sand and cross your fingers hoping for public education to improve or you can focus on making real change. What have we got to lose? Merit pay will work when we figure it out, but that can’t happen until you start trying it. At the very least we will be able to tell who the best and who the worst teachers are. Someone explain to me how THAT won’t be helpful.

Russ how can you criticize the possibility of “wasting time and money on something that (supposedly) does not work” when you are advocating continuing with exactly that by sticking with the status quo?

Posted by: Frank at December 1, 2009 12:51 PM

If you are going to pursue merit pay then make sure it is merit pay and not just another expensive pathetic excuse to pay teachers more than we already do. Make sure that “merit” is not indistinguishable from seniority, preparation, and peer friendships. We have that now and everyone agrees that it is not improving the effectiveness of public education.

If we are going to try merit pay for teachers then make sure it’s based on student outcomes, nothing else. If it’s not working at first then change it and measure student outcomes until you find something that does work. But don’t lose sight of the purpose of education which is student outcomes, not which adult in the school system agrees with who’s methods, or which theoretical model sounds best, or which teacher got which certification – none of that is improving public education, as we have all seen.

For those looking for proof that merit pay will improve student outcomes, I say what other reasonable choices do we have? It is highly likely that many merit pay systems would not work, especially if they are indiscriminate, rewarding nearly all regardless of the results of their efforts. It is doubtless that there are and will be systems of rewarding teachers based on student performance that will improve learning, the trick is to find it ASAP!

You can stick your head in the sand and cross your fingers hoping for public education to improve or you can focus on making real change. What have we got to lose? Merit pay will work when we figure it out, but that can’t happen until you start trying it. At the very least we will be able to tell who the best and who the worst teachers are. Someone explain to me how THAT won’t be helpful.

Russ how can you criticize the possibility of “wasting time and money on something that (supposedly) does not work” when you are advocating continuing with exactly that by sticking with the status quo?

Posted by: Frank at December 1, 2009 12:53 PM

Russ,

Just curious. What's your critique of the Value Added methodology? Frankly, it makes a lot of sense to me -- we all agree that socioeconomic factors have the largest impact on student grades. Value added captures the marginal impact of the teacher and, logically, the school environment.

Here's a thought experiment that makes my point. What would be the impact on standardized test scores of switching the entire student body from an inner city school with the student body from East Greenwich or Barrington high school, while keeping the same teachers in place? My bet is that in both cases it would go down (and so too would Value Added), because both sets of teachers have optimized their methods to get the most out of their respective students, given their socioeconomic backgrounds. This is analogous to why CEOs can't easily switch industries and maintain high performance, or athletes switch sports, or cops move from suburban to inner city departments, or doctor switch specialties -- you get the idea.

The point is, teachers must believe that they add unique positive value to the student who come into their schools. So why not based merit pay on that measure? Carrying on from the above comments, let's ditch the peer, principal and student review, and split the merit bonus so that it is 50% due to the teacher's specific value added with his or her students, and 50% based on the school's overall Value Added, to reflect the role that learning climate plays.

What's the argument against this?

Oh, and one more thing Russ. I'm curious what arguments you would make to a CEO who is considering whether to move his or her company to RI? What would your pitch be? Would you acknowledge any problems in the state? And if so, how would you address them?

If we can't make an effective pitch, we can't pay for welfare and public sector pensions, either now or in the future. That's the harsh truth.

Posted by: John at December 1, 2009 6:06 PM

Today we have published test scores that reveal the success of individual schools. These schools are run by principals. We must allow the principals to do their jobs.

Hold principals accountable. Set measurable benchmarks for each school, and refuse to renew the principal's contract if the benchmarks aren't met.

But also give principals the power to hire, fire, and set raises for their teachers and staff. That's true merit pay. And it will bring school communities together around common goals.

Don't let superintendents off the hook either. If the principals they hire don't make the grade and district benchmarks aren't met, then the superintendent's contract won't be renewed either.

Allow schools the power to make change. Give the schools the GLEs and say "make it happen."

In other words, make every school a "charter" school.

Posted by: mikeinri at December 1, 2009 7:44 PM

Mikeinri has is right.

Bottom line, we need accountability with consequences.

Russ says merit pay doesn't work. Non-sense. If that is the case, then why give raises at all?

One thing we know for certain is that the current system is, at best, not optimal, and at worst, not working at all.

Step 1 is to bring back management rights, which the Unions have successfully stripped away.

Step 2 is to allow management to manage, including firing non-performers.

Everyone else in the world is measured, judged and retained / fired based on some measure (whether objective or not). Why should teachers be immune to this?

Test scores are a very good and objective place to start. Design the tests to fit the outcomes that we want to achieve. For example, we'd like our children to be able to read, write and do arithmatic. Test them on that. If they don't show measurable improvement, don't expect a raise.

And don't blame the student population and make all the other usual excuses when the measurement doesn't show improvement.

The local car salesman gets a raise & bonus based on the number of cars he sells. The best salesman in the world had trouble selling cars in this economy. Guess what? The dealership didn't have a good year so the salesman didn't get a raise or a bonus. It wasn't good enough to make excuses that the economy was bad. Very simply, the sales didn't materialize, so no merit increase and no bonus. Real simple. It's called LIFE!

It's time the Union joins the real world. It is also time to stop making excuses for reform. Roadblocks and excuses are a Union core competancy. Time to ignore them and try something new ...warts and all.

Posted by: George Elbow at December 1, 2009 9:54 PM

If you want to follow a "business model" of productivity/performance in education...

If you want to evaluate teachers based on the performance of their students...

Then, let teachers "fire" under-performing students. Not just give them "Fs", but actually fire them. Let the teacher kick the under-performing student out of their classes. That's what good managers do with under-performing employees, right?

If you're not willing to do this, then I think that you're making a mistake by trying to evaluate teachers based on the performance of their students.

Posted by: Thomas Schmeling at December 2, 2009 11:28 PM

Alrighty, then.

I suppose that, as a carpenter, I should disclaim evaluation unless I've a right to "fire" suboptimal tools and materials. Or as a writer unless I can "fire" readers who fail to be persuaded.

Posted by: Justin Katz at December 3, 2009 5:30 AM

Hey Tom Schmeling,

There you go again, carrying the Union's water, singing from the Union play-book, blaming the students for poor performance.

Students are the business model equivalent of raw material. Businesses don't throw out the raw material. They find a way to realize value even from the scrap.

Stop defending the status quo. It ain't working. Try something new (i.e. new to Union run public education, but not new to the real world) such as ACCOUNTABILITY and PERFORMANCE.

Your teachers grade students every quarter, every year. They hold them back if they don't perform up to certain standards.

Yet, they scream bloody murder the minute we begin talking about grading and holding them accountable to high standards ...and they can always count on guys like you to foolishly run to their defense.

Posted by: George Elbow at December 3, 2009 6:59 AM

George- Not long ago, I testified before the Senate Education and House HEW committees. The thrust of my testimony was the need for more frequent and rigorous evaluations of public school teachers in RI. In light of that, and in light of the fact that I have already said that I do support merit pay (if a a reasonable system can be designed), perhaps you might reconsider your presumptions about my views about who I “carry water” for.

My point is that folks who say teachers should be evaluated based on student performance just like “real world” managers are evaluated based on the performance of their workers, tend to forget that situations are not the same.

Justin- An author who can’t “fire” readers who fail to be persuaded, but a publisher can fire writers who fail to sell books. I don’t know much about carpentry, but if a foreman gives poor tools and materials to one set of workers and excellent ones to another, and then evaluates their products on the same metric, I would say that’s unfair, wouldn’t you? Isn’t it possible that the better carpenters will produce the worse product?

Posted by: Thomas Schmeling at December 4, 2009 10:34 AM

George- Not long ago, I testified before the Senate Education and House HEW committees. The thrust of my testimony was the need for more frequent and rigorous evaluations of public school teachers in RI. In light of that, and in light of the fact that I have already said that I do support merit pay (if a a reasonable system can be designed), perhaps you might reconsider your presumptions about my views about who I “carry water” for.

My point is that folks who say teachers should be evaluated based on student performance just like “real world” managers are evaluated based on the performance of their workers, tend to forget that situations are not the same.

Justin- An author who can’t “fire” readers who fail to be persuaded, but a publisher can fire writers who fail to sell books. I don’t know much about carpentry, but if a foreman gives poor tools and materials to one set of workers and excellent ones to another, and then evaluates their products on the same metric, I would say that’s unfair, wouldn’t you? Isn’t it possible that the better carpenters will produce the worse product?

Posted by: Thomas Schmeling at December 4, 2009 10:34 AM

Thomas,
Your point is duly noted.

That being said, we can make excuses all day long why we can't appropriately evaluate teacher performance and match employment and pay to it.

Bottom line, if you support it, let's get it done and stop worrying about getting it 100% perfect, stop providing excuses and roadblocks. Every job in the private sector is differnt, but they all have some sort of evaluation.

Posted by: George Elbow at December 4, 2009 11:55 AM

Thomas,

Did you miss the part where I suggesting putting evaluations in the hands of administrators? Part of the reason for doing so is to enable them to make adjustments for circumstances.

Posted by: Justin Katz at December 4, 2009 9:00 PM

Justin,

Forgive Thomas. He has been hanging around Smith hill too long, absorbing all the Bob Walsh & Co's propaganda, such as "Teacher pay has not kept up with Inflation", or better yet, RI's underperforming schools are the fault of the "the children", with NO culpability from the Union teachers.

Thomas pretends that, when we say Teachers should be graded on student performance, we mean ALL students must be A students.

That is NOT the case. What we want is demonstrated IMPROVEMENT. We want a teacher who can help a D student become a C student, which is an improvement.

The carpenter that builds a structually sound, albeit out of square & somewhat crooked, structure with warped timber is equally competant to the carpenter who builds a structually sound and square / plumb structure with the benefit of using select grade-A straight timber.

Also, Thomas says "real world managers are evaluated based on the performance of their workers". Wrong. In the real world, employees are evaluated on the performance relative to the objective(s) of their job. If the job of the entity is to sell cars, then the number of cars sold is usually the primary measurement, regardless of whether they are selling Chevys or BMWs.

If Thomas was in charge of the Private sector, there would be no employee evaluations. He'd conduct studies to find 100 excuses why people's performance can't be judged ...it was the economy; it was a sunny day; it was a rainy day; the competion had a sale & we didn't; etc.

Thomas too often is about perpetual analysis and excuses. Always easier to "study something" than to "do something".

Thomas, no offense, but even at the best universities with the best scholars, they have a saying that is apropos ..."pencils down".

Posted by: George Elbow at December 4, 2009 9:43 PM

Justin,

Putting evaluation "in the hands of administration" is fine, as long as you can address the completely reasonable concerns of teachers that too much flexibility to "adjust for circumstances" will lead to favoritism and corruption. I'm pretty sure you oppose both of those.

I suspect that you and I agree on principles, but I think you underestimate the very reasonable concerns of teachers (NOT the union) that a) much of student performance is totally out of the hands of teachers and b) no matter how bad seniority promotion is, there is an equally serious concern that "supervisor evaluations" will turn into a system of favoritism.

My suggestion is that you seriously think through the system of incentives that an evaluation scheme creates, and watch carefully for perverse incentives.

I've spent a great deal of time inside RI public schools watching what actually happens "on the ground". Can you say the same?

George,

I really can't fathom your thinking. You "duly noted" my point that, not only do I believe in rigorous teacher evaluations, but have testified before the GA to encourage their adoption. I doubt you have done as much to promote this goal (If you have, please let us all know). Still you think I'm somehow under the union spell and say "If Thomas was in charge of the Private sector, there would be no employee evaluations"

That's complete nonsense.

Posted by: Thomas Schmeling at December 4, 2009 10:39 PM

Justin,

While I'm at it, let me ask this

Do you children attend public schools?

If so, what role have you played? Have you had a leadership role, or any role, in the PTO/PTA?

Posted by: Thomas Schmeling at December 4, 2009 10:56 PM

Thomas,

Are you suggesting that employees in the private sector don't have to be concerned about "favoritism"?

Are you suggesting that teachers should be immune to what the rest of the real world deals with?

Are you so cynical that you can't fathom management empowered with management rights will do the right thing?

When teachers make a living doling out grades every quarter, every year, are you suggesting that those grades are based on favoritism?

You say that "much of student performance is totally out of the hands of teachers". So do tell us again why we are paying them so well??

After nine short years on the job, we are paying teachers ~$70,000+ per year (not to mention the lavish fringe benefits and job security), which is the private sector equivalent of about $106,000 per year. How many private sector professionals are making $106,000 per year after just 9 years on the job, never mind going year after year without any substative and consequential performance evaluation?

Sorry Thomas, there is no other explanation for your ridiculous postion on this issue other than to say you are carrying the the Union's water.

Posted by: George Elbow at December 4, 2009 11:01 PM

Thomas,

If Justin was smart, he'd have his children in private schools ...but not sure he can afford it with the high taxes he must pay to support unaccountable Entitlement-minded protected Union teachers in the Public system.

Your concern about favoritsm works both ways. Those that are appropriately critical of the system have to be concerned about retaliation from Union-hacks that comfortably employ such terrorist tactics as "work-to-rule".

By the way, you really need to stop trying to suggest that you are the only person that has thought about and actively gotten involved in these issues. If you tell us one more time about your "boots on the ground" experiences and your GA testimony, I swear I am going to be sick.

Rest assured, you are NOT the only one that cares and is involved ...we just don't all spend our time trying to convince people of our bonifides.

Posted by: George Elbow at December 4, 2009 11:08 PM

Sorry Thomas, there is no other explanation for your ridiculous postion on this issue other than to say you are carrying the the Union's water.

George,

Sorry Thomas, there is no other explanation for your ridiculous postion on this issue other than to say you are carrying the the Union's water

You're really not paying any attention at all. I hope you'll understand why I won't be responding to your comments anymore. If you don't, others will.

we just don't all spend our time trying to convince people of our bonifides

I don't. But at least I've got some.

Bye, George.

Posted by: thomas Schmeling at December 4, 2009 11:23 PM

Bye Thomas.

Don't worry, I didn't expect you capable of answering my simple straightforward inquiries.

You prefer to pretend things are complex, requiring endless study with no answers.

When faced with simple questions (e.g. You say that "much of student performance is totally out of the hands of teachers". So do tell us again why we are paying them so well?), you become paralyzed and unresponsive.

No problem Thomas. You met expectations. And for that, you'd receive a 2 out of 5 on an evaluation in the private sector, meaning no raise for you this year.

Posted by: George Elbow at December 5, 2009 11:49 AM

Thomas,

To return to your questions to me: One cannot simultaneously involve a subjective authority and have no risk of corruption. Of course, as the unions have taught us, systematic pretensions to objectivity tend toward a broader and more intractable corruption.

The corrective for administrative malfeasance is the inevitability that favoritism and corruption will harm results, and the remedy is for an attentive citizenry to push their representatives to squash the problem.

I'll note, though, that you've made apparent to me something that hadn't occurred to me previously: The "labor" arguments are typically persuasive in their immediate context, and the trick is to keep anybody from considering them all in tandem. You join teachers in their concerns about favoritism, but that is precisely what their union ought to be for. Essentially, you're demanding double protection: First from the stagnating system of senior and cog-like treatment, and then by the union. Well, if the union's there to check dotted Is and crossed Ts, doesn't that mitigate the risks of favoritism? (Unless, of course, the administrative favoritism falls mainly to the benefit of the union leadership... but that's an argument more in my line than yours.)

As for my role in public schools in Tiverton, I think a good deal of it is readily apparent... in public statements and participation. The problem is the entirety of the system that the unions and their facilitators have built, and parent-teacher organizations are part of that system.

Posted by: Justin Katz at December 5, 2009 8:29 PM

Justin,

What I want is a system of annual evaluation that identifies good and hard work with reasonable and fair measurements, and minimizes (I doubt it can be eliminated) arbitrary treatment of teachers by administrators. I find it hard to understand what's so wrong with that.

Did you read Julia Steiny's column today? I've sort of been expecting a post about it. I don't agree with everything she says, but I think she makes some good points.

Posted by: Thomas Schmeling at December 6, 2009 1:41 PM