151–160 of 244 posts
Earlier quoted context omitted.
I think the problem is when student evaluation have an affect (positive or negative) on the teacher career.
Yes, when that measure becomes a target it may have some unintended consequences. But the device is too useful to be discarded for that reason.
Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/1... Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female profess…
In other words, these are your run-of-the-mill sociology/psychology papers.
There's a much better way to hold educators to account - have other educators sit in on classes. But this is often very strongly resisted, apparently it's 'unprofessional'.
Eventually you get used to it and some even prefer it - I know I really get a lot out of pair programming with someone close to me in skill but with a much different background - but getting over that first hump is tough.
Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/1... Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female profess…
students rated identical teachers Did you actually read critically the papers you cite? [1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is " neither students’ grades nor self-study hours are affected by the instructor’s gender ". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of…
If it was possible to objectively determine teacher quality, student evaluation probably would not exist in the first place. The reason we ask people's opinion is in order to quantify the subjective. Double edged sword, because we tend to conflate quantified & objective.
That's half the theme here. erm.. people's opinions are subjective..
Part of the problem with "student-as-consumer" is that students aren't always the real consumer.
This is the age-old doctor question. You go to the doctor's office because you have a medical problem. Doctor 1 comes in on-time, she's smart, capable, friendly, and spends a good amount of time concerned about you and your situation. Doctor 2 comes in late, she's irascible, pedantic, hurried, and only listens long enough to tell you what to do. She also insults you and your family in various ways and doesn't seem to…
It's not like there can't be an accurate way to measure this. Pre-register patient problems, and see how fast doctors solve problems on average/median. Then it doesn't matter how jovial/amicable the doc is. With enough data you can fish out potential signals (and then test them properly), maybe bedside manner really doesn't matter, maybe being on time is more important. (Because then people spend less time in the cli…
Doctors, like professors, are evaluated by former patients based on a mix of factors.
In your doctor example, doctors don't "solve problems". They see patients. Many times these patients present with conflicting and hard-to-resolve symptoms. I imagine with the internet this has gotten worse. They use a mix of Occam's Razor, deductive reasoning, and social psychology for the patient's long-term health.
You don't want a doctor that finds the answers to complex problems and gives out morphine. You want a doctor that over decades has patients with the best outcomes. In my example, the one doctor may have ticked you off so much you decide never to go back -- and quit smoking just to prove her wrong. That's a win for both you and the doc, although there's no problem being solved.
You can speculate various ways to go at this. Perhaps the doctor is evaluated by other doctors working at that location. Or perhaps it's a matrix of various things. As long as you're just pulling stuff out of the air, you can invent any kind of system you'd like. And tech will deliver it for you.
Human interventions are messy affairs, and thank goodness for that. We are not robots. Your proposal might do a great job of sorting out professors who teach students well enough to take the test, but heck if I see where that is the end of the analysis. That's just the beginning. We're just getting started talking about professors who can lead a class to mostly pass a test. That should be a baseline for any professor.
This is the problem we see in software development. You can come up with all sorts of metrics to stand in place of evaluating whether a team/person is good or not, but whatever you come up with mostly seem to work -- and has very bad effects on those being measured. (With the possible exception of business-value-loop feedback time)
I don't think this is tractable in the way you seem to want it. If I were picking a college or professor, I might look at five or six variables and weigh them in various ways depending on my personal needs and goals at the time -- which might change tomorrow. I might want to go interview them. Meet some current and former students.
There is a severe and onerous "platform danger" in tech. It starts with the conceit that we can build platforms for anything. It ends up with people not-as-smart-as-they-used-to-be having Google tell them where to go for dinner. Or Wally in Dilbert coding himself up a minivan. [0]
I'm not saying the metric definitions and accumulations aren't interesting. They deserve to be considered. I'm saying that by reducing human judgment to single vector, it makes us as a species dumber. That's one of those intuitive things that sound good but are in reality quite bad.
Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/1... Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female profess…
students rated identical teachers Did you actually read critically the papers you cite? [1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is " neither students’ grades nor self-study hours are affected by the instructor’s gender ". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of…
[2] is similarly vague, [...] isn't "contact
with the instructor" significant?
The article doesn't detail how their online courses are structured, but when I've done free udacity/coursera/edx courses, contact with the instructor has been nonexistent.
Seems to me it'd be very difficult to design a test to investigate this that wouldn't have some valid methodological criticisms. Even if you had lecturers, delivering online lectures with fake names, using voice-changing software and online-only office hours, you could still criticise that as being unrepresentative of real college lecturing, and having confounding factors if the courses ran at different schools, years, or times.
A possible solution would be to split the job, to avoid conficts of interest. One teacher teaches. Another one grades. When I was studying in France in "prépa", all students wanted to get a real evaluation of their level, because the really important exam to enter the best universities was organized nation-wide. But, this is hard to organize in private universities where students are also clients. Which parent would…
They might've started doing so to combat the notion that private education is worth less because it just might be easier.
To be honest, I'm not sure how teachers get and keep their jobs, but as students are graded on their studying skills, I feel like teachers should be graded every fews years on their teaching skills. My girlfriend is studying in a distance university, so she basically never meets any of her teachers. This year she has to do practices at a chosen company, and she got a teacher assigned to her who is supposed to "help"…
Everything got recorded in an online system - the training teachers did their reflective writing &c and I read their blog posts (private blogs!) and tried to support. Managers at the University (relevant ones) could access the blogs and help out with any more serious issues. I had direct contact with the mentors as well in case they had issues to discuss.
Probably the hardest teaching job I did in a 30 year career. But almost all of the training teachers are working in local colleges and schools.
The target audience is a crucial part of the feedback loop. Removing the students from the equation sounds counter intuitive. What are the proposed alternative metrics? Who delivers the feedback? And what is the feedback based on, if not on the direct opinions of the people on the receiving end of the service?
One of the main problems with the educational system is that students are evaluated by the same person that teaches them. If the teacher wasn’t the one grading the mean grade of the students would be a decent evaluation of the performance of teacher.
If another professor wrote the exams for a class they didn't sit in, you'd have the students screaming in unison, "this wasn't discussed in class!" Not exactly disagreeing with the idea, but students have been normalized to the idea that, with enough studying and concentration, any exam can be aced. When they feel like they've been cheated by the system they get discouraged, making a harmful learning experience.
If another professor wrote the exams for a class they
didn't sit in, you'd have the students screaming in
unison, "this wasn't discussed in class!"
For example, in electronic engineering I would expect a second-year class in switch mode power supply design at any university to look extremely similar.
Admittedly there may be classes in cutting-edge research, opinion-based subjects or extremely niche topics, where you can't find two academics in the country who could agree on a year's syllabus and set of answers. In my degree I'd say less than 25% of courses were that way, though.