May 18, 2010
Gist on Central Falls and the Importance of Evaluations
Rick Hess at EdWeek interviewed RI Ed. Commissioner Deborah Gist in light of the recent agreement between Central Falls teachers and Superintendent Frances Gallo. Hess' focus was on the importance of a good evaluation system for making reform work.
Rick Hess: The deal turns critically on the teacher evaluation component that'll be introduced next year (with unsatisfactory teachers targeted for termination). How will we know whether the evaluation is sufficiently tough, or whether it becomes a fig leaf for backing away from more painful measures?As research conducted by The New Teacher Project (TNTP) has shown (outlined in their report, "The Widget Effect" -- PDF and website), the teacher evaluation process is woefully inept nationwide. The TNTP's "Widget Effect" is largely a by-product of the industrial era/collective-bargaining system whereby school districts and unions have come to view and treat teachers as identical widgets in the educational machinery. The operating assumption is that the vast majority of teachers are all equally effective. In the districts that TNTP studied:Deborah Gist: There are a couple of ways that we'll know. One is that the administration has the complete authority to put the evaluation into place. The agreement says the evaluation will be put into place solely by the management. And the Board of Regents passed regulations that define what the evaluations have to look like in this state. The guidelines are good and strong, and everything that we're doing is based on those.
RH: Okay, but suppose that, at this time next year, we see that just five or six of the school's 93 teachers are removed. Would observers be right to be skeptical that the process was toothless?
DG: It's not about removing any particular percentage of teachers. It's hard to know what the proper percentage would look like. But I strongly encourage people to be skeptical. We should be skeptical. I want people to take a hard look at us, and I'm going to do the same with the district and with my staff. But it's not about the percentage of teachers we remove. It's about the quality of the evaluation and about performance. I expect there will be turnover, but how much there is remains to be seen....
RH: What do you say to critics who might ask how you can leave the faculty intact for another year at a school that you've identified as profoundly low-performing?
DG: We don't take this decision lightly. We take it very seriously. But there are some great teachers at the high school and, because teacher evaluation is so poor around the country and in the state, we don't have good evidence as to who should stay and who should not. This deal gives us the opportunity to make those decisions in a more informed way and gives folks the opportunity to be a part of the reform movement. There are examples of groups of teachers coming together to turn their schools around in various communities, and there's no reason to assume it can't happen here. We're going to give teachers that chance. Our expectations are high. We'll be watching carefully. If they're not ready to deliver results, we'll act upon that rapidly.
All teachers are rated good or great - In districts that use binary evaluation ratings (generally "satisfactory" or "unsatisfactory"), more than 99 percent of teachers receive the satisfactory rating. Districts that use a broader range of rating options do little better; in these districts, 94 percent of teachers receive one of the top two ratings and less than 1 percent are rated unsatisfactory.That's simply impossible and unrealistic. While such a flawed system clearly overstates the effectiveness of the average teacher (and being a teacher of "average" effectiveness isn't a net negative, by the way), worse is that they diminish the real effectiveness of those who truly are superior educators. Further, these evaluation systems are not fair to new teachers and teachers who are average or good but still have areas that need improvement.
73 percent of teachers surveyed said their most recent evaluation did not identify any development areas, and only 45 percent of teachers who did have development areas identified said they received useful support to improve....Though it is widely recognized that teachers are least effective in their beginning years, 66 percent of novice teachers received a rating greater than "satisfactory" on their most recent performance evaluation....Despite uniformly positive evaluation ratings, teachers and administrators both recognize ineffective teaching in their schools. In fact, 81 percent of administrators and 58 percent of teachers say there is a tenured teacher in their school who is performing poorly, and 43 percent of teachers say there is a tenured teacher who should be dismissed for poor performance.But instituting an evaluation system won't be easy thanks to the culture that has developed in which, teachers--even novice teachers--expect to be given the highest rating. And why not? They've never been evaluated any differently (everybody wins)!
Our research reflects that there is a strong and logical expectation among teachers that they will receive outstanding performance ratings. While the vast majority of teachers receive the highest rating, those teachers who do not receive it tend to believe that the higher rating was warranted....Even teachers who are just beginning their careers believe they deserve the highest performance ratings and are dissatisfied if they are rated good, not great. This inflated sense of performance is evident in the self-assessment ratings of novice teachers. In a subset of districts where teachers were asked to assess their own instructional performance on a scale of 1 to 10, 69 percent of novice teachers rated their instructional performance an 8 or higher.That's why teacher "buy-in" is so important to effect change. Of course, that doesn't mean that change requires that the current teachers buy in, just that you find teachers who will.In a system where negative or even less than perfect performance ratings are given only rarely, teachers naturally develop an expectation that they will be among the large majority considered top performers. In this context, teachers perceive low or negative ratings not in terms of what they communicate about performance but as a personally-directed insult or attack. The response is understandable in the context of the current system, where so few teachers get critical feedback of any kind. When their evaluation does include criticism, they feel as though they have been singled out while other examples of poor performance go unaddressed.
This creates a culture in which teachers are strongly resistant to receiving an evaluation rating that suggests their practice needs improvement. Schools then find themselves in a vicious cycle; administrators generally do not accurately evaluate poor performance, leading to an expectation of high performance ratings, which, in turn, cause administrators to face stiff cultural resistance when they do issue even marginally negative evaluations. The result is a dysfunctional school community in which performance problems cannot be openly identified or addressed.
There's much more good stuff in "The Widget Effect" report, but the bottom line is that implementing a robust and fair evaluation system is extremely important for moving forward with school reform.
ADDENDUM: Writing elsewhere, Hess is supportive of the Central Falls deal, noting:
The Rhode Island story is a truly encouraging development....this story shows how leaders with backbone can eventually force union leadership to accept a new reality. Yes, Gallo walked back the bold action that won her many education reformers' approval, but good management is about discipline, not bloodlust. The point of school turnarounds is not to count scalps, but to win necessary changes, force out lousy teachers, and reset the board.
http://www.fcps.edu/DHR/employees/evaluations/index.htm
Godd old Fairfax County, Virginia. Certainly not perfect, but a damned sight better than the NOTHING we do now!
Posted by: John at May 18, 2010 1:13 PMKudos for correctly pointing out that performance ratings are horribly ineffective as a way of determining employee performance. You start right, but then you and your NTP study make the flawed and nonsensical assumption that individual evaluations are productive even "extremely important" to implementing reform.
If only we listen to the wisdom of D.C. technocrats, this time the results will be different! One can hope, but predictably this too will fail.
How will people react to the news that they are not in the upper 20 percent? Or top one-third? Or not even above average? Some will accept the news and feel bad about how inferior they are. Some will deny the news and attribute their low rating to a screwed-up rating system. In either case--losers or cynics--what has the company gained?
The way I see it, you advocate replacing "The Widget Effect" with the "the Pygmalion Effect" with its spiral of diminished expectations and results. Until we eventually recognize that it's the process not the people, we're doomed to chose between repeating the same mistakes or exchanging the old ones for new ones.
I might also note the irony that you Tea Party types don't seem to mind this type of inside the Beltway elitism, so long as it's directed at bashing unions.
Posted by: Russ at May 18, 2010 4:12 PMHey Russ,
Out here in the parallel universe you call the private sector and we call the real world, lots and lots of companies, under the brutal pressure of domestic and international competition, get rid of our bottom 10% of performers each year. And you know what? While the process isn't perfect, most of us support it, because we all know that loading up an organization with poor performers is a perfect recipe for bankruptcy, which means loss of income, job, insurance, and lots of other bad things.
Another thing most of us know is that continuous innovation and learning are critical to staying competitive and offering our customers and clients the superior value for their money that keeps them coming back. Without standards, there is no basis for measuring performance, identifying shortfalls, and driving the learning feedback loop. We know that ignorance isn't bliss; rather, it is a recipe for getting a nasty surprise down the road, and losing your job.
Sadly, you and your public sector colleagues don't have a clue about what I'm talking about -- or maybe you do, and that is why you work where you do.
Posted by: John at May 19, 2010 12:53 AMOut here in the parallel universe you call the private sector and we call the real world...
I work in the private sector. So much for the predictable straw man argument.
Another thing most of us know is that continuous innovation [sic] and learning are critical to staying competitive and offering our customers and clients the superior value for their money that keeps them coming back.
Ha! One of the things I do in the private sector is advocate for TQM best practices. The source I'm quoting above was Deming wrote extensively about the "deadly disease" of management by objective and performance reviews.
Sadly, you and your public sector colleagues don't have a clue about what I'm talking about -- or maybe you do, and that is why you work where you do.
Yes, I do know quite a bit about CI, unfortunately I run into many folks like yourself who though nearly totally uninformed on the subject think themselves experts. I'd be quite curious to speak with someone about education reform who could actually speak to the above points versus engaging in ad hominem attacks and spouting cliches.
Posted by: Russ at May 19, 2010 10:17 AMHmmm, messed up a tag there somehow (some much for TQM)...
The source I'm quoting above was a continuous improvement guru.
We know that ignorance isn't bliss; rather, it is a recipe for getting a nasty surprise down the road, and losing your job.
Deming wrote extensively about the "deadly disease" of management by objective and performance reviews.
Posted by: Russ at May 19, 2010 10:24 AMRuss,
You and I clearly have had different experiences in the private sector. Sorry to read that your efforts as a TQM advocate have been so frustrating. I guess efforts to continually find ways to innovate in the product and process domains to stave off competition from you-know-where are just wasted efforts, right?
Fascinated to hear about private sector companies that have neither objectives nor performance reviews, but still manage (I infer) to survive and prosper. You really should call RIEDC and help them recruit these extraordinary organizations to relocate in RI, where the GA and unions would greet them with open arms. They'd also be just perfect places for getting all our welfare clients back to work!
Finally, just a wild guess, but I'll bet those companies with neither objectives nor performance reviews aren't backed by venture capital or other private equity funds.
Posted by: John at May 19, 2010 3:58 PMI've started a venture backed firm. "Just a wild guess," but I'm guessing you can't say the same. For the record, I'm not at all frustrated. I expect that real change is not easy, especially given the tightly held myths that pass for wisdom in certain circles.
I should note that you seem to have mistaken my objection to counterproductive performance reviews with an objection to setting objectives, a pretty clear indication that you didn't actually bother to read what Scholtes had to say (I'm shocked!). Here's his conclusion:
-- Feedback is, by definition, something derived from one part of the system and given to an antecedent part of the system. Performance appraisal is a feed-down activity that is hierarchical by nature, not systemic. And the hierarchy is not ordinarily the best source of system-focused feedback.-- Performance appraisal, particularly when tied to income and promotability, engenders posing and pretense: easy targets, creative accounting, and evaluation that reflects the preconceptions of the supervisor and the manipulations of the person being evaluated. All of this works against the communication of data needed for improvement. A better alternative is to have employees, alone or in natural work groups,
* Identify a key work process.
* Identify the customer(s) of that process.
* Learn what characteristics of the product or service--the output of the key process--are most important to the customer.
* Get feedback from the customer(s) on how well these characteristics are met.
* The next step is that managers must refrain from using this data to evaluate an employee and continue to encourage each employee to develop feedback loops such as this for all the key tasks.
If you can think of anything substantive to say I'll be happy to respond in kind.
Posted by: Russ at May 20, 2010 9:37 AMSo, Marc, your post begs the question of why "implementing a robust and fair evaluation system is extremely important for moving forward with school reform." Can I take it from your lack of response that there is no concrete evidence that reviews have any effect other than the negative effects I've already referenced?
Posted by: Russ at May 20, 2010 1:40 PMNo, Russ, you can take it as I've been busy. The "Widget Effect" report basically proposes a Feedback loop that goes hand in hand with an evaluation system--an integrated approach that may be contrary to your citation of the TQM paradigm, but is hardly some newfangled idea (here and here, for instance). As an engineer versed in Reliablity Centered Maintenance practices and Root Cause Failure Analysis, I understand Feedback loops and how they can work (or sometimes fail). I also conduct employee performance evaluations and know first-hand that the two do indeed cross-polinate. Ie, performance is affected by process or organizational structure and all that.
Certainly, feedback should be generated outside of just employee evaluation, but that is one part of the whole program. Hence, evaluation is one component that leads to feedback, which leads to system/process improvement. That's why I believe an evaluation system is important to reform. You disagree and that's fine.
Posted by: Marc at May 21, 2010 4:04 PM