Good Science Bad Science Part II
May 6, 2012
Nutritionist epidemiologist Walter Willett of the Harvard School of Public Health recently released the study “Red Meat Consumption Linked to Increased Risk of Total, Cardiovascular, and Cancer Mortality.” In the press release announcing the conclusions of the study lead author An Pan said, “Our study adds more evidence to the health risks of eating high amounts of red meat, which has been associated with type 2 diabetes, coronary heart disease, stroke, and certain cancers in other studies.”
Before we analyze the study it’s fair to say that nutrition studies on humans are very difficult to do because we don’t clone people and put them in lab cages, and using prisoners or mental hospital patients whose food intake can be carefully controlled and recorded is considered unethical. So the alternative used in this study–people choose what they will eat and record what they ate some time after they ate it–are suspect from the start. Did people record what they did eat or what they should have eaten? Here’s an article on the inaccuracy of self-reporting.
Now let’s look at the study itself. The researchers looked at 22 years of data from 37,698 men from the Health Professionals Follow-up Study and 28 years of data from 83,644 women in the Nurses’ Health Study. All subjects were free of cardiovascular disease and cancer when they entered their respective studies. Diets were assessed through questionnaires every four years.
This is not all bad. Both men and women were in the study. Good science. They were disease free at the outset. GS. There were large numbers of subjects in the study. GS. Data were collected for many years. GS. But questionnaires every four years? VBS.
Next, the researchers interpret their Very Bad Data. They look at the deaths from heart disease and cancer and, amazingly, draw this conclusion: Regular consumption of red meat, particularly processed red meat, was associated with increased mortality risk. ‘Was associated with’—that’s the tricky phrase. Any scientist knows that correlation does not mean causation. A Good Scientist would have looked at other possible explanations. Did the meat eaters who died live unhealthy lifestyles—consuming a lot of carbohydrates, being overweight, drinking, smoking, failing to exercise? VBS.
But they do identify one of the flaws in their own study. “Red meat, especially processed meat, contains ingredients that have been linked to increased risk of…cardiovascular disease and cancer. These include heme iron, saturated fat, sodium, nitrites, and certain carcinogens that are formed during cooking.”
So did the researchers do a follow-up study where they eliminated the sat fat, sodium, and nitrites in subjects’ diets and disallow cooking over high heat that could have contributed to disease? No. Did they distinguish between the quality of meat from a cow that happily lived its entire live on a South Dakota prairie getting exercise and eating its natural food from a cow that spent its last months knee-deep in manure, pumped full of antibiotics and growth hormones, and fed corn that causes acidosis for which the cow was given big doses of baking soda, a cow that was barely alive when slaughtered? No. Can we trust the conclusions of these researchers? You decide.





Probiotics are getting a lot of press these days, and fermentation classes in the Black Hills are attracting sizable crowds. So what are they and why do we care?