Taking Apart Bad Science
Today, an article made headlines around the world that was published by The Lancet: Public Health, a side-shoot of the top academic journal at the moment. The article is open access so you can download and read it if you are scientifically minded. I copy-pasted parts of the study to show you the many errors. I incorporate my thoughts here and there, breaking the quotes and bolded particularly important sections I need to talk about.
“Study design and participants
The Atherosclerosis Risk in Communities (ARIC) study is an ongoing, prospective observational study of cardiovascular risk factors in four US communities (Forsyth County, NC; Jackson, MS; suburbs of Minneapolis, MN; and Washington County, MD), initially consisting of participants aged 45–64 years who were recruited between 1987 and 1989 (Visit 1) Study participants were examined at follow-up visits, with the second visit occurring between 1990 and 1992, the third between 1993 and 1995, the fourth between 1996 and 1998, the fifth between 2011 and 2013, and the sixth between 2016 and 2017…”
Note that this study has six different time frames, each of which is several years apart.
Participants completed an interview that included a 66-item semi-quantitative food frequency questionnaire (FFQ), modified from a 61-item FFQ designed and validated by Willett and colleagues,16 at Visit 1 (1987–89) and Visit 3 (1993–95). Participants reported the frequency with which they consumed particular foods and beverages in nine standard frequency categories (extending from never or less than one time per month, to six or more times per day). Standard portion sizes were provided as a reference for intake estimation, and pictures and food models were shown to the participants by the interviewer at each examination…”
Oh dear… how many glasses of orange juice have you consumed dear reader between January 1, 2012 and December 31, 2015? Just curious if you can recall because I could not… How about the number of servings of butter, chicken, or cakes in that same time period? For that matter, what is your serving size of rice? Is that always the same and similar to what the questionnaire counted as a serving size? How ripe was the banana you ate… a very ripe banana has more sugar. Oh and how long were the bananas you ate? Small, medium or large?
Oh boy… I hope that these participants have detailed records for everything they ate for the 25 years length of this study! Because if not… can the statistical analysis of what information they gave have any meaning?
We did a time varying sensitivity analysis: between baseline ARIC Visit 1 and Visit 3, carbohydrate intake was calculated on the basis of responses from the baseline FFQ. From Visit 3 onwards, the cumulative average of carbohydrate intake was calculated on the basis of the mean of baseline and Visit 3 FFQ responses…”
WOW, hold on now. They collected carbohydrate information from the first and third visit and then they estimated the rest based on these two visits? Do they mean by this that
- The data for years 2,4,5, and 6 didn’t match what they wanted to see?
- The data for years 2,4,5, and 6 didn’t exist?
What kind of a trick might this hide? Not the kind of statistics I would like to consider as VALID STATISTICAL ANALYSIS.
“…We did not update carbohydrate exposures of participants that developed heart disease, diabetes, and stroke before Visit 3, to reduce potential confounding from changes in diet that could arise from the diagnosis of these diseases… The expected residual years of survival were estimated…”
Oh wow! So those who ate a lot of carbohydrates and developed diabetes, stroke, heart disease during the study were excluded? This does not reduce confounding changes but actually increases them. That is because the very thing they are studying is how carbohydrates influence health and longevity, that is no diabetes, no strokes, and no heart disease. By excluding those that actually ended up with them completely changes the outcome to the points the authors are trying to make rather than reflect the reality.
Also, if they presume a change in diet for these participants, why not for the rest? Do you detect any problems here? I do!
37% of kcal from Carbohydrates–from Table 1
The following information was brought to my attention by Dr. Georgia Ede. Mean total energy intake: 1558- 1660 kcal for all participants in the surveys–in table 1 titled “Populations characteristics in the Atherosclerosis Risk in Communities study, by quantile” (page 4). The energy intake from carbohydrates was recorded as 37% to 61%. The 37% is considered to be the low carbohydrate diet. Calculating what 37% of the lowest kcal intake is: (37% * 1558)/4 = 144.115 gr (~144 gr) carbohydrate, where 4 kcal is equal to 1 gr carbohydrate.
In the opinion of the reader: is a diet that contains 144 gr carbohydrate a low carbohydrate diet?
Data Range Manipulation
The table below and the explanation are taken from Dr. Zoë Harcombe’s page analyzing this paper. The carb ranges in the statistical analysis differ in the paper. They are “subjectively selected by the researchers; the number of people that ended up in each range; and the deaths that occurred in that carb range during the 25 year follow-up” notes Dr. Harcombe. “Most covered a 10% range (e.g. 40-50%), but the range chosen for the ‘optimal’ group (50-55%) was just 5% wide. This placed as many as 6,097 people in one group and as few as 315 in another.”
When the data-ranges are different in a statistical analysis that is very specifically tailored to present its outcome in ranges, such range manipulation leads to erroneous outcome.
…High carbohydrate consumption was associated with a significantly higher risk of all-cause mortality compared with moderate carbohydrate consumption…”
Yes, I cherry-picked this sentence because I found it humorous. I think you find it humorous too–either that or cry over it. After spending an entire article showing that low carbohydrate diets are killing us, this sentence is part of their conclusions. And a final meaningless quote–the last sentence of the paper:
“when restricting carbohydrate intake, replacement of carbohydrates with predominantly plant-based fats and proteins could be considered as a long-term approach to promote healthy ageing”
Ahhhh.. they don’t know that plants are carbohydrates!? Interesting.
My Take on This Study
There are 3 types of studies on nutrition:
- Meaningless–meaning it repeats something that was already repeated hundreds of times
This study falls into Bad and Meaningless nutrition studies. It is actually not really science–these researchers simply cracked the same database that others already have and manipulated the data to fit their hypothesis.
I have commented all through the quotes from the study of what was shocking to read and see. What is even more amazing is the last 2 sentences, a quote, in the press release by Jennifer Cockerell, Press Association Health Correspondent:
Dr Ian Johnson, emeritus fellow at the Quadram Institute Bioscience in Norwich, said: ‘The national dietary guidelines for the UK, which are based on the findings of the Scientific Advisory Committee on Nutrition, recommend that carbohydrates should account for 50% of total dietary energy intake. In fact, this figure is close to the average carbohydrate consumption by the UK population observed in dietary surveys. It is gratifying to see from the new study that this level of carbohydrate intake seems to be optimal for longevity.‘”
It is not gratifying but horrible to see that the UK, one of the most diseased countries on the planet today, plagued by type 2 diabetes, obesity, and heart disease, should consider its current general carbohydrate consumption levels to be ideal and finds support in this study for what they are currently doing.
I suppose that if type 2 diabetes, obesity, and other metabolic diseases is what the country wants (and why wouldn’t it want that? Guess who profits from sick people?), then indeed, a 50% carbohydrate diet is ideal.
Comments are welcome as always and are moderated for appropriateness.
Updated on 8/20/2018 by Angela