Dr. Fox effect

The Dr. Fox effect is a correlation observed between teacher expressiveness, content coverage, student evaluation and student achievement.^[1] This effect also allows insight to other related effects, such as those discussed below, and relationships between student achievement and evaluations of the teacher.

Experiment

The original experiment was conducted at the University of Southern California School of Medicine in 1970 in which two speakers gave lectures to a classroom of MDs and PhDs (psychiatrists and psychologists) on an irrelevant topic. The topic, “Mathematical Game Theory as Applied to Physician Education”, was chosen to eliminate the factor that the students being lectured may know information about the actual subject. Students were divided into two separate classrooms; one classroom would be lectured by an actual scientist and the other by an actor who was given the identity “Dr. Myron L. Fox”, a graduate of Albert Einstein College of Medicine.

In the first half of the study the actor was instructed to teach his material in a more monotoned and inexpressive voice. This lecture was then compared to the control lecture by the scientist. After the lectures, the students were tested on the information they had learned and the students who attended the lecture taught by the scientist learned more about the material, and performed better on the examination.

However, when both "Dr. Myron L. Fox" and the scientist both presented their material in an engaging, expressive, and enthusiastic matter, the students rated Dr. Fox just as highly as the genuine professor. This lack of correlation between content-coverage and ratings under conditions of high expressiveness became known as the Dr. Fox Effect.^[2]

In a critique of student evaluations of teaching, professor of law Deborah Merritt summarized the Dr. Fox Effect as it was observed in the first experiments: "The experimenters created a meaningless lecture on 'Mathematical Game Theory as Applied to Physician Education,' and coached the actor to deliver it 'with an excessive use of double talk, neologisms, non sequiturs, and contradictory statements.' At the same time, the researchers encouraged the actor to adopt a lively demeanor, convey warmth toward his audience, and intersperse his nonsensical comments with humor. ... The actor fooled not just one, but three separate audiences of professional and graduate students. Despite the emptiness of his lecture, fifty-five psychiatrists, psychologists, educators, graduate students, and other professionals produced evaluations of Dr. Fox that were overwhelmingly positive. ... The disturbing feature of the Dr. Fox study, as the experimenters noted, is that Fox’s nonverbal behaviors so completely masked a meaningless, jargon-filled, and confused presentation."^[3]

A 1980 study found that prestige of research could even be increased by confounding writing style, with research competency being positively correlated to reading difficulty.^[4] Anecdotal evidences have since been reported nowadays by researchers. ^[5]

Subsequent research

Study of Air Force Academy

In this study, students were randomly assigned to professors, to eliminate the possibility of good students receiving better professors. These professors were then given all the same syllabi, curriculum levels, and final exams so the difficulty was even across the board. There was also a follow up course given to the students to test the value of deep learning that the students received. These professors taught introductory calculus to a group of over 10,000 students to achieve the proper information.

When the evaluations came through, the professors who were less experienced and less qualified had the best evaluations and best performances on the final exam. However the students who attended the examinations given by the more qualified and experienced professors did best on the follow up examination. This brought speculation that the professors who were more experienced taught more of a broad manner of the material to produce a deeper understanding. This became obvious in the follow-up examinations.

The results showed that the professors who instilled a deeper meaning of the material appeared worse on the initial exam and evaluations, but in the long run received better academic standing. This finding calls into question the validity of student evaluations.^[6]

Eyal Peer and Elisha Babad

Criticisms of the original study include the experiments lack of a control group, the use of a "Yes/No" scale, and lack of measurement evaluating learning among the participants. In a recent study published in the Journal of Educational Psychology, researchers Eyal Peer of Carnegie Mellon University and Elisha Babad of Hebrew University of Jerusalem recreated the original 1973 study taking these criticisms into consideration.^[7]

Through their research, they found that the Dr. Fox effect was still present despite the manipulations they added. Manipulations included the use of a more in-depth questionnaire, removal of the first thirty seconds of video describing the credentials of the lecturer, and a warning at the beginning of the questionnaire to ensure that participants responded honestly.

However, the researchers found that although the students with a charismatic teacher enjoyed the lecture more, they reported that they had not actually learned anything new. This is in contrast to the original study which argued that people attending the lecture actually believed they were learning new material. From their research they were able to reason that an enthusiastic speaker may entertain an audience, but much more is required to be a successful teacher.^[8]

Halo effect

The Halo effect shares similar qualities to those of the Dr. Fox effect in relation to student evaluations of teachers. The halo effect is a cognitive bias in which our overall impression of a person influences how we feel and think about his or her character.^[9] For example, attractive-looking people create a halo effect in which we perceive them as kind, smart or successful, but it may not be true because their attractive appearance clouds our judgment of their performance capabilities. In one study examining halo effect on student evaluation, there was a higher rating for teachers who provided more nonverbal immediacy, but the study also found out that a higher evaluation of teachers was related to a greater halo effect.^[9] In the Dr. Fox effect study, a similar effect was found when Dr. Fox presented the lecture in an expressive manner.

Effects of instructor characteristics on student evaluations

From the experiment of Dr. Fox effect, the expressiveness of a teacher when delivering lecture material can affect student evaluation of the teacher. Other than that, individual differences among teachers such as personality, popularity, lecture fluency, non verbal behavior, and attractiveness can also have an effect on student evaluation of teachers.

Personality

Instructor personality is one factor that has been shown to affect course evaluations.^[10] For example, in one study examining the big five personality dimensions, teachers who were perceived to be more extroverted, open, agreeable, and conscientious were evaluated more favorably, whereas teachers who were perceived to be more neurotic were evaluated less favorably.^[11] Further, Murray and colleagues suggested that the effects of instructor personality on student evaluations of instructor vary across the type of course.^[12] Although some traits, like a teacher’s leadership, consistently predicted course evaluations, other traits varied across type of course. For example, a teacher’s sociability positively predicted course evaluations in introductory psychology courses, but not graduate psychology courses, whereas the ambition of an instructor showed the opposite relationship – predicting evaluations of graduate, but not introductory psychology courses.^[12]

Charisma/popularity

The charisma or popularity of a teacher might also be a factor that contributes to teacher effectiveness and teaching quality. In one study that examined the assessment of charisma as a factor in effective teaching, charismatic teachers tended to receive good student evaluation and were also perceived as being more funny, helpful, encouraging, knowledgeable, sympathetic and other traits that are considered charismatic.^[13] Besides that, the study of Yun-Chen Huang and Shu-Hui Lin shows that different teaching methods can also increase the charisma rating of a teacher.^[13] For example, teachers that are perceived to be more charismatic offer explanations, answer questions by students, vary their teaching methods, and are also interested in and express concern for their students and their learning progress.^[13] Charisma of teachers is becoming more recognized recently with the introduction of an instrument to measure charisma, called "Inventory of Teaching Charisma in the College Classroom" (ICCT).^[13]

Lecture fluency

The fluency of a lecturer when delivering teaching materials can contribute to the effectiveness of student evaluation of a teacher. According to the study of Shana K. Carpenter and colleague that examine lecture fluency on learning perception, a lecture that are considered to be fluent suggested that the teacher stood upright and straight, maintained eye contact, and spoke fluidly without notes whereas a lecture that are considered disfluent include teachers that are slumped and stand with a bent back, looked away, and spoke haltingly with notes.^[14] Furthermore, the study also shows that fluent teachers that are prepared and well organized will receive higher ratings than a disfluent teacher that is unprepared and disorganized in student evaluations of the teacher.^[14] Although the same study of lecture fluency on learning perception successfully show that lecture fluency will increase student rating of a teacher, but further interpretation of result suggest that lecture fluency biases the students’ perceptions of their own learning because lecture fluency did not actually affect the amount of information learned.^[14]

Non-verbal behavior

Non verbal behavior or non verbal communication is a series of wordless behavior that are projected by the speaker to the listener. Common examples of non verbal behavior include eye contact, smile, facial expression, distance between speaker and listener and any other wordless behavior that can communicate information between people. According to Virginia P. Richmond on 'Teacher Nonverbal Immediacy', teachers that show more positive nonverbal behavior increase the immediacy between students and also contribute to higher student rating.^[15] For instance, teachers with higher rating by student were more likely to express non verbal behavior like smiling, walking around, head nodding and touching upper torsos whereas low rating teachers are more likely to touch their head, head shaking rather than nodding, and sitting on a chair.^[15] One study by Ambady and Rosenthal shows that subjects were able to form accurate judgement of impression by just watching short video clips of teachers providing non verbal behavior.^[16] An example from the study, participants were able to form accurate impression of the teachers by just watching 10's, 5's, and 2's length of short clips about teachers walking to classroom. Ambady research provides support that nonverbal behavior has a strong influence on impression formation of teacher.

Attire/appearance

The way that teachers present themselves or the attire they wear when teaching lecture can be a factor on student evaluation of teacher. According to Virginia P. Richmond, the attire of a teacher influence the way that students perceived their teacher.^[15] For example, teachers were perceived to be more competent, organized, prepared and knowledgeable when they dress formally with a coat and tie but they are also presumed to be not receptive to students needs and low in student-teacher interaction.^[15] However, when teachers dress casually with collar-button shirt and jeans, they are perceived to be more friendly, flexible, fair and open but not so competent.^[15] From another study about predicting teacher evaluation from physical appearance, attire and appearance may be a factor influencing rating of student but the effect will somewhat diminished when there are others more important and significant information provided.^[16] For instance, the study found out that unattractive voices dilute the effects of attractive faces of teachers.^[16]

Background of Dr. Fox experiment

Authors

• Dr. Naftulin is associate professor and director of the Division of Continuing Education in Psychiatry, University of Southern California School of Medicine
• Mr. Ware is assistant professor of medical education and health care planning and director of research and evaluation, Southern Illinois University School of Medicine

• Mr. Donnelly is an instructor in psychiatry (psychology) in the USC Division of Continuing Education in Psychiatry^[1]

Actor

The actor portraying Dr. Fox was given one day to prepare for the lecture and was coached by the authors to present the lecture "with an excessive use of double-talk, neologisms, non sequiturs, and contradicting statements.”^[17]

Motivation

There are previous reports and studies that show that the personality of a teacher/lecturer is one of the most important variables when it comes to evaluating the teacher’s effectiveness. Getzels and Jackson, with support from Wallen and Travers, expressed that the personality and patterns of teacher behavior and methods presented represent the main forces for the most significant variable in evaluating the teacher’s effectiveness. Goffman explains that the receptivity of the audience is highly influenced by the person introducing the lecture, the quality of the introduction, and the speaker’s involuntary expressive behavior, which becomes a decisive factor in how the audience responds to the conveying information. One study which looked at the student perceptions of educators from 7th to 12th grade and reported that the students regarded “teacher charisma or popularity” as the most important characteristic when rating teachers.

If the charisma or popularity of the teachers have such an effect on their effectiveness ratings by junior high and high school students, would the ratings be influenced in the same way with a group of highly trained professional educators in a learning situation? So what the authors were looking to see is that if a sufficiently impressive lecture was given, would it result in an experienced group of high end educators participating in a new learning situation feeling satisfied that they had learned from the lecture despite irrelevant, conflicting, and meaningless content conveyed by the lecturer.^[1]

See also

• Albert Mehrabian
• Forer effect
• Sokal affair

References

External links

• The Legendary Dr Fox Lecture - Footage Found!