For this assignment, you will be using the case study “What Can a Fourth-Grader Do With Statistics?” on page 375 of your text. This case study is about a fourth grader’s application of statistics to a real-world question.
Do the following for this assignment:
- Read the case study on page 375.
- Respond to Questions 1–4 on page 376.
- In addition to the questions in the book, discuss what type of results would be needed for Emily to believe that her results were something other than pure chance.
What Can a Fourth-Grader Do with Statistics?
Nine-year-old Emily Rosa was in fourth grade, trying to decide what to do for her school science fair project. She was thinking of doing a project on the colors of M&M candies, when she noticed her mother, a nurse, watching a videotape about a practice called “non-contact therapeutic touch†or “TT.†TT is a popular alternative medical treatment, practiced in many places throughout the world. But no statistically valid test had ever clearly demonstrated whether it actually works. Emily told her mother that she had an idea for testing TT and wanted to make it her science fair project.
Despite the name, TT therapists do not actually touch their patients. Instead, they move their hands a few inches above a patient’s body. Therapeutic touch supporters claim that these hand movements allow trained therapists to feel and manipulate what they call a “human energy field.†By doing these manipulations properly, the therapists can supposedly cure many different ailments and diseases. Emily Rosa’s science fair project sought to find out whether trained TT therapists could really feel a human energy field.
To do her project, Emily recruited 21 TT therapists to participate in a simple experiment. Each therapist sat across a table from Emily, laying his or her arms out flat, palms up. Emily then put up a cardboard partition with cutouts for the therapist’s arms. This prevented Emily and the therapist from seeing each other’s face, but allowed Emily to see the therapist’s hands.
Emily then placed one of her hands a few inches above one of the therapist’s two hands, asking the therapist to identify which hand it was. If the therapist could truly feel Emily’s “human energy field,†then the therapist should have been able to tell whether his or her right or left hand was closest to Emily’s hand. Each trial of the experiment ended with Emily’s recording whether the therapist was right or wrong in identifying the hand.
Emily took several precautions to make sure her experiment would be statistically valid. For example, to ensure that her choices between the two hands were random, Emily used the outcome of a coin toss to determine whether she placed her hand over the therapist’s left or right hand in each case. And to make sure she had enough data to evaluate statistical significance, 14 of the 21 therapists got 10 tries each, while 7 got 20 tries each.
The results were a miserable failure for the TT therapists. Because there were only two possible answers in each trial—left hand or right hand—by pure chance the therapists ought to have been able to guess the correct hand about 50% of the time. In fact, the overall results showed that they got the correct answer only 44% of the time. Moreover, none of the therapists performed better than expected by chance in a statistically significant way. Emily also checked to see whether therapists with more experience did better than those with less experience. They did not. Emily’s conclusion: If there is such a thing as a “human energy field†(which she doubts), the TT therapists can’t feel it. And even if the “human energy field†exists, it’s difficult to imagine how TT therapists could use it for healing if they can’t even detect its presence.
One of the most interesting aspects of this study was that Emily was able to do it at all. Other skeptics of TT had hoped to conduct similar studies in the past, but TT therapists had refused to participate. One famous skeptic, magician James Randi, had even offered a $1 million prize to any TT therapist who could pass a test similar to Emily’s. Only one person accepted Randi’s challenge, and she succeeded in only 11 of 20 trials, about the same as would be expected by chance. So why was Emily able to recruit participants who had avoided more experienced researchers? Apparently, the therapists agreed to participate in Emily’s experiment because they did not feel threatened by a fourth-grader.
The novelty of Emily’s science fair project drew media attention, and it was not long before word reached retired Pennsylvania psychiatrist Stephen Barrett. Dr. Barrett specialized in debunking “quack†therapies, and he convinced Emily and her mother to report her results in a medical research paper. The paper was published in the Journal of the American Medical Association (April 1, 1998) when Emily was 11, making her the youngest-ever author of a paper in that prestigious journal.
QUESTIONS FOR DISCUSSION
1. After Emily’s results were published, many TT supporters claimed that her experiment was invalid because she and her mother were biased against TT. Based on the way her experiment was designed, do you think that her personal bias could have affected her results? Why or why not?
2. Another objection to Emily’s experiment was that it was only single-blind rather than double-blind. That is, the therapist could not see what Emily was doing, but Emily could see what the therapist was doing. Do you think this objection is valid in this case? Can you think of a way that Emily’s experiment might be repeated but be made double-blind?
3. Emily’s experiment was not a direct test of whether TT treatment works, because it did not check to see whether patients actually improved when treated by TT. Suggest a statistically valid way to test whether TT is more effective than a placebo.
4. Based on the results of Emily’s study, skeptics now say that TT is so clearly invalid that it should no longer be used or funded. Do you agree? Why or why not?
(Bennett 375)
Bennett, Jeff, Bill Briggs, Mario Triola. Statistical Reasoning for Everyday Life, 4th Edition. Pearson Learning Solutions, 2012-12-01. VitalBook file.
