Carpe Datum

I do understand your dilemma, but you have to remember that unfortunately, what you rate as something completely cut and dry (getting papers back in 2 days) and what students perceive are not necessarily the same. Whenever you ask anyone to rate anything, most people opt for the middle of the road unless something truly extraordinary happens.

For instance, I nearly always give about a 15% tip to waitstaff. Iused to be a waitress, so I appreciate their work a lot. If it's really above the call of duty awesome, then 20 to 25% is not uncommon. If it's horrendously bad, sometimes I'll drop it to 10%. But 90% of the time, 15% is what someone will get from me rounded to the nearest dollar.

Also, I think most students at most universities don't think their teacher evals have any worth other than wastingtheir time... Your school may be different, but that's the reality of a college kid's way of looking at it...

When I was in a math department, evaluations were required by the College, but it was pretty much a ritual in abuse the instructor. Most students would rather chew off a leg than take a math class, and they take out their frustrations with this on the poor bastard that gets, er, has to try to teach them. Absolute numbers are worthless. On the other hand, if you ignore the absolute numbers and look at relative rankings within a subject, those rankings seem to be repeatable.

Everyone vehemently denies this, but the easiest way to get good evals is to teach a course for vapid students (something aimed for elementary education majors) which is insultingly simple, and give everyone A's. To first order, the evaluation a student gives is the evaluation they think they are getting from the course.

As someone who just finished a Graduate program (for Library & Information Science) I have come to view evaluations as completely worthless. We had one adjunct faculty member who couldn't "practice what she preached." We had to turn in contact info for a later assignment (a visit to a library) on time even though the entire state (except for the university we were attending) was shut down due to bad winter weather. The assignment was worth zero points but the professor said she might have docked points if it had been turned in late. Later in the semester she takes three weeks to grade our assignments because "she was busy." She constantly lectured about how we were in grad school now and we should proof read and be be able to follow instructions. Her written assignment instructions were absolute gibberish because she DIDN'T proofread them. Everyone, and I do mean EVERYONE, in the class hated her teaching. Yet, when evaluation time came along most people either gave her straight 3s or refused to fill out an evaluation saying that "if you can't say anything nice don't say anything at all!" A few people did give her straight 1s and wrote that "The Professor" should be removed from the class in response to the appropriate question in the written comments section. Unfortunately, this lady is still teaching this year while much better adjuncts are gone.

Welcome to the new agenda... Accountability.

Somewhere along the way, corporate culture decided that the way to increase productivity was accountability. Marvelous buzz-word. Useless in application by a majority of humanity.

The basic premise is that you want hard and fast data that demonstrates commitment to desired results (e.g. lower call-center hold times). The problem is that without actually thinking about how you want to measure things, the simplest way to achieve low call-center hold times is to pick up a line and then accidentally hang up on the customer allowing you to take the next caller. Amazingly, this does *not* contribute to customer service or satisfaction.

The result? A number of people who went and got MBAs because they weren't thorough or analytical enough to get accounting or science degrees are now making up surveys that fail to capture anything of relevance. Now... If you wanted to actually write a useful survey you could ask questions like "graded work returned before next class = 5, by next class = 4, etc." This would, of course, be analytical but it doesn't really allow any subjectivity and we all know that customer satisfaction is subjective.

There's a whole tangent to be followed about writing a good questionnaire here which could probably turn into a seething political debate about how pollsters ask questions to get the results they want. It's actually a fun subject to play with since you can ask people to take a pair of questionnaires back to back and, by asking the right questions, get them to give contradictory answers.

Actually, it's worse than that. Bizness types view any accountability questions as a way to make sure they look good, not to actually provide feedback that can be used for improvement. This makes for a very distorted evaluation procedure; the purpose of approximately any corporate evaluation procedure is to make someone -- usually the person who created the judging criteria -- look good. It is very rare for that to have much to do with the actual duties of those being evaluated. Consequently the time when one is supposedly working on "goals" and "evaluations" ... typically the month of January ... is a highly stressed and completely worthless time, because the evaluation questions have N*O*T*H*I*N*G to do with a typical employee's actual job. Dilbertism in its most corrosive form.

I dunno, Feff. I think you're missing the origin of our hosts complaint. If the folks at his university wanted to look good they would contrive an eval that would inherently score high. Certainly better than median. This doesn't seem to be the case.

In this situation I would certainly expect this to be a portion of the "merit based" system where they need some sort of arbitrary metric which you are 50% likely to have failed. Given enough such measures, you are likely to have failed at least one and this gives them the necessary "black mark" to hold up in a court should the need ever arise.

There are, of course, people like our host and you and I who actually look at such metrics as a means of evaluating performance and potential for improvement (my boss does the same thing) which can make life better in small portions of the universe. Sadly it is like fighting entropy. The most entertaining thing, however, is pointing out to someone that their metrics are not leading to improvement. (See prior comments about evaluating time a ticket is open but not satisfactory completion of request.)

We get ratings analyzed relative to our peers, and most of us have fairly similar grade distributions, so at least those two variables are controlled. What isn't controlled is the teacher's mood on that day, how the rating period is conducted, when it is conducted, what happened in the previous class, whether there is a test coming up or just finished, what the general mood of the students is, etc.

Basically, the only thing I even vaguely trust are the written comments and then only if they vary from the generic "great!" sorts of comments. Unfortunately, nobody else in any kind of position of authority over me measures my performance in the way that I do, so I have no way of knowing really how much those numbers are trusted.

Oh, no argument, SA. I was bitching about what the bizness world does with the concept of "goals" and "evaluations". You are correct, but only in a rather downstream way, that (from a bizness point of view) evaluations have to be conducted in a way that generates enough bad results so they can be used to dismiss anyone (except, of course, you can't override the crucial criterion, that those evaluations can never produce adverse results about the person/committee who wrote the criteria and evaluation techiniques).

At my ex-employer, FuckEveryoneInTheWorldExceptTheCEOCO, the people who created the evaluation criteria for the peons were (of course) in HR. They have not a friggin clue about what the core business is and what competencies might be necessary for any job in the company (probably including their own); that would involve math, since that's what an insurance company does. HR types are also the kind of people who think the word "logarithm" is a satanic incantation. And, as you point out, they see nothing wrong with any of this.

All I know is that one senior colleague of mine routinely got stunningly good course evals. She also taught courses that were so ridiculously trivial that a full sememster's exposure to them would cause brain damage in anything higher up the evolutionary sequence than a coelenterate, which meant that it was pitched perfectly for the Elementary Education majors who flocked to her courses, and who loved the easy A's.

When I taught the same courses, and demanded real work ... library research papers, a little abstract reasoning, manipulate a few numbers, and so on ... my evals came in about a full point (on a scale of 1 to 5) poorer, which is *huge* for those kinds of things.

Boo to library research papers.

I'd give you a 5 of 5 if you routinely got them back the next class period. That'd be fan-freakin'-tastic, in fact. I've given out 5s for less, even, I believe, if the next assignment was spaced far enough apart and it was difficult enough to grade (presumably). But it really depends on where in the survey that question is posed. For our general, university surveys (non-department specific scannable pieces of crud) that would be pretty early, I think. But if it's not, I'm ready to go.

Best bet is to change how and/or when evals are done because a good eval should take 10-15 minutes at least, conceivably, but there's no way a student wants to spend that much time on something they do view as useless. And, the comments section is last and takes the most effort. Combined with the fact that it's usually done at the end of the class or with other classmates who are waiting to get to class if it's done before class makes it rushed and unimportant to us.

Also, I've disagreed with my own written comments later, on reflection and added classroom experience.

I didn't give my math and science teachers bad evals. I just sucked at both of those subjects and knew it. I'm not arithmetically proficient, but I'm not an idiot or vapid, either. I think there is a lot of contempt for people like me who didn't find math and science so fascinating that we would pour our hearts and souls into every moment of such classes. I'm sorry, but I was busy majoring in something I absolutely adored, which took up a great deal of my time and energy. And on top of that I was working up to 4 jobs at a time to pay for it.

So don't get all arrogant about us "vapid" students. I think I do a hell of a lot of good in my work as a teacher without knowing how to solve a binary equation any more. Let's see you present an indepth harmonic analysis a 4-movement Beethoven symphony and discuss it's relevance to the work of Brahms and Mahler. Then apply that critical thinking skill to a classroom and bring it down to the level of a 14 year-old.

My point is this: I may not have been able to learn critical thinking through mathematics, but at least I had the chance to hone the skill elsewhere.

Vapid. hmph.