Pediatric Associates of the NW Blogs

A middle school science lesson for parents. A study finds…

Jay S. Rosenbloom, MD, PhD
November 09, 2011 04:30PM

Many lessons we learned in school are useful later in life.

One lesson taught to most middle school students is the scientific method, also known as scientific inquiry.  When they learn it, students may see it as just one more concept they have to memorize for a test.  However, it is crucial for adults to use this method when listening to the media coverage about  the findings of “the latest” studies.

Wikipedia succinctly presents the key steps  of this method of study

  1. Define the question
  2. Observe and gather information and resources
  3. Form hypothesis
  4. Perform experiment and collect data
  5. Analyze data
  6. Interpret data and draw conclusions that serve as a starting point for new hypothesis
  7. Publish results
  8. Retest

The key to understanding the importance and relevance of  a particular study lay in the details.  How well the study was done can have a significant impact on the likelihood the conclusion reflects an actual cause and effect relationship.  One example is knowing the size of the sample tested.  Relatively small sample sizes may skew results.

Unfortunately, public health can be impacted by the hype some studies receive.

For example, a report was published looking at 12 autistic children asking how many had their MMR vaccinations in the months prior to noticing the symptoms.  Eight of the 12 families reported that the MMR shot series  was given in the months leading up to their autism.  A study of this size and scope should be interpreted very cautiously.

There are many details that could influence the findings.  How were the children selected to be included in the study?  Had the families already come to conclusions about the cause of autism before participating?  If we ask survivors of heart attacks how many had eaten a banana in the week before, would it be help us understand heart attacks if eight of the 12 answered yes?

This is why we need our scientific method to help us understand media reports of new findings.  Most scientific and medical studies are used to help build a base of knowledge to understand things we observe and hypothesize to be true; they are not intended to be the final answer.

The last step in the method listed above is “retest” (different from “rinse and repeat”). How might we compare a study looking for a link between the MMR  series and autism if it looked at 500,000 individuals instead of 12?  How do we use both studies to find clues as to what the “real” association might be?  Did the author of the paper have any biases that might influence their findings?

Medicine should rarely change recommendations based on a single study.  If it did,  we would be starting, stopping and changing treatments every day. Most of these changes would not lead to improved care, and often it would be worse.

The following are three tweets from about a 12 minute period:

-          “Screen-Watching Children Have Narrow Blood Vessels in Eyes, Study Finds http://nyti.ms/egsCiS

-          “Botox blunts emotional understanding, study finds http://lat.ms/g1VMsO

-          “Tai chi appears to benefit quality of life for patients with chronic heart failure, study finds http://dlvr.it/PpWTm

So before I unplug the TV, start tai chi and give up my botox (not really), I would want to know more about the studies, or at least look for other studies conducted by other research groups found the same thing.  A brief review of the literature I was able to find showed that the tai chi helped but studies have also found yoga, or even listening to soothing music to help chronically ill patients helped equally, as well.