The Dr. Fox Effect: Bias In Student Evaluations Of Professors

Deborah J. Merrit:

Researchers have extensively documented the effect of these [stylistic and non-verbal] signals on student evaluations, often contrasting the dominance of classroom “style” over content. One early and well-known investigation into these classroom dynamics used a charismatic, distinguished-looking, and mellifluous actor to play the role of a scholar named “Dr. Fox.”23 The experimenters created a meaningless lecture on “Mathematical Game Theory as Applied to Physician Education,” and coached Fox to deliver it “with an excessive use of double talk, neologisms, non sequiturs, and contradictory statements.”24 At the same time, the researchers encouraged Fox to adopt a lively demeanor, convey warmth toward his audience, and intersperse his nonsensical comments with humor. “In short,” as one of the investigators summarized, Dr. Fox “gave a very enjoyable lecture in which he offered little or nothing of substance.”25

Fox fooled not just one, but three separate audiences of professional and graduate students.26 Despite the emptiness of his lecture, fifty-five psychiatrists, psychologists, educators, graduate students, and other professionals produced evaluations of Dr. Fox that were overwhelmingly positive.27 In addition to awarding him strong numerical scores, audience members praised him for an “[e]xcellent presentation,” “warm manner,” “[g]ood flow,” “[l]ively examples,” “relaxed manner,” and “[g]ood analysis of subject.”28

Fox’s use of warm, enthusiastic, and lively nonverbal behaviors would have been admirable if it had complemented a substantive presentation. Most faculty use stylistic elements to engage student interest and motivate learning. The disturbing feature of the Dr. Fox study, as the experimenters noted, is that Fox’s nonverbal behaviors so completely masked a meaningless, jargon-filled, and confused presentation. If style can trump substance so easily, even in the minds of a trained, professional audience, then what role do nonverbal behaviors play in more routine student evaluations?

 

Comments

[Ruth]

But don't we see this again and again in life outside academia? Politics, anyone?

[dr x]

I would agee, Ruth.

[Dennis]

"...warm, enthusiastic, and lively nonverbal behaviors."

As long as one is giving a lecture of some substance this would actually count as doing a good job. I try to be warm, enthusiastic and lively in the classroom. Not because I want good student reviews (which are much more closely tied to grades, honestly) but because I want to engage my students, help them see the importance of the topic, and inspire them to learn and to think.

[Trimegistus]

Dr. Fox has been President of the United States since 2009.

[Assistant Village Idiot]

Well, this is appalling, but let me note a difficulty with the experiment. If you take smart people and start them thinking about a subject in a contained setting, they are likely to teach themselves something. Not being jarring, and providing anecdotes that relate at least superficially to the topic, even if they don't have further content, will continue that process.

I am, for example, a daydreamer, who sometimes fades out after the first few minutes of a lecture or a sermon, but learns something anyway.

[Texan99]

AVI, I'm sure you're right, but that's just to say that people can learn in spite of blowhard fakes who aren't teaching them anything. The mark of a good teacher shouldn't be whether he can outright prevent independent learning.

Really, we should all have more self-respect than to admire someone who's supposed to be explaining something and instead is projecting an appealing persona. The persona's fine, who doesn't like an entertaining style? But we ought to know whether we've been presented with a coherent argument or not, and whether we now understand something new or not. It's shameful to be so distracted by the pleasant manner that we're willing to give the professor credit for teaching us something we don't yet know. Or, what's just as bad, to say he's a "good" teacher without any regard for whether he's succeeded in teaching anything. Or to congratulate ourselves on learning when we haven't.

[Ruth]

Right on, Texan99. Additionally, I think all involved should be grateful it wasn't a pyramid scheme presentation, only a harmless academic study.

[TwinkleStars]

What do you think RateAProfessor is all about? I've shadowed a few Dr.Foxes, they usually think they're great instructors. They fall for their own hype. If you get enough kids to say you are "amazing" without looking at what they've really learned, it's easy for a mediocre instructor to skate by on charisma and charm.