Dirty Hands, Fake Gardens, & Bad Science

Dirty Hands

When we dig up the soil and get dirty hands, that is a fun thing. Digging and planting seeds to grow new plants that can later be harvested, measured, quantified, and analyzed, makes a lot of great sense; it is satisfying and can be controlled, like a clinical trial of sorts. You know what I mean.

Mining metadata is like digging the soil in many gardens at once, so it doesn’t tell us anything about any one garden and it certainly doesn’t tell us much about what plants grow there and what was harvested. Meta analyses are dataset combinations—meaning the combinations of many people digging in many gardens over many years and then recalling what they saw in those gardens is somewhat pointless, isn’t it?!  Yet that is precisely what metadata analyses are and which are always used by the gardeners in this particular garden I am about to write about.

The Garden

Harvard T.H. Chan School of Public Health

The Gardeners

Marta Guasch-Ferré, PhD, Ambika Satija, PhD, Stacy A. Blondin, PhD, Marie Janiszewski, BFA, Ester Emlen, BS, Lauren E. O’Connor, PhD, Wayne W. Campbell, PhD, Frank B. Hu, MD, PhD, Walter C. Willett, MD, DrPH, Meir J. Stampfer, MD, DrPH—they are the authors of the article I am about to take apart.

Their Challenge

“The study’s lead author, Marta Guasch-Ferré, a research scientist at Harvard’s nutrition department, and her team crunched the numbers by comparing high-meat diets to a combination of other diets. Overall, across all diets, they found no major differences in the factors associated with heart disease.” (see here) ooooops… no difference

How do researchers wiggle their way out of a null finding like this and publish a paper that says the findings are significant and red meat is harmful?

This is a major problem. The thing to do is to reset the problem from a different perspective by changing the data interpretation from what it actually is to something the researchers wished it was, like this:

“Then the researchers took a deeper dive to check the diets individually.” (here)

When comparing the diets head on and nothing was significant, the hypothesis failed. The science I practice is this: hypothesis (a) failed, so now let’s check hypothesis (b) and so forth. What did these researchers do? They said: let’s dig the soil deeper where we may find something that we can say lines up with our hypothesis. What did they find? Actually nothing but here is what they translated that findings to:

“’We find that when red meat was substituted by high-quality plant protein sources including legumes, soy or nuts, that led to more favourable changes in blood lipids and lipoproteins compared to red meat,’ Guasch-Ferré said in an interview.” (here)

We must be clear that there was nothing substituted anywhere. This was not a clinical trial. The merely pulled several data and combined them. They also did something that violates statistical principles of independence assumptions, by taking the data collection of the same people over time when they were switched to different diets and used that as parallel dataset—this assumes independence, which didn’t exist. This activity has a name: data manipulation. Data manipulation is a highly irregular (I am using nice words) activity that is really frowned upon.

Just a reminder about who was doing this data manipulation: the same academicians who wrote the paper that ended up being labeled “low carbs kills”, which is a closed paper from free view but I wrote about it in great detail here and I have also published with the same journal my commentary, which can be found here. These are also the same people who work with this multi-billionaire lady, whose plan is to make the world vegan because she is and she is dying from it, and also the same people who wrote the Eat Lancet Commission. So in sum: these are the people who are causing the greatest harm to the people (in terms of health), who benefit the most (financially, see one here) as a result of pushing the vegan/vegetarian agenda, weight loss agenda, and they have the means of reaching their goal by publishing everywhere and anything so long that it forces people to eat vegetable oils, processed foods, lots of carbs. I call this the rotten fruit.

Perished Perishables

So how did they actually conclude that red meat increases cholesterol? By using the wrong statistics—the use of p-values is strongly discouraged by statisticians—and any hazard ratio under 2 is insignificant (see Austin Bradford Hill’s definition of what can be considered to be significant). See the original meta-analysis published paper here. So they used the wrong statistics and applied the wrong explanation of what they found—on purpose.

“We obtained or calculated the amount of red meat consumed by participants in each intervention diet from the dietary results available in the published journal article or via communications with study authors… Crossover studies were treated as parallel studies, such that each intervention phase of a crossover study was treated as an independent arm of a parallel study [this I noted earlier as a highly irregular manipulation]… Heterogeneity was statistically significant at P≤0.10 [I don’t know any statistics where P ≤ 0.10 is significant]… Intervention durations ranged from 2 to 36 weeks with a mean duration of 8.5 weeks [combining data of such huge time-length differences skews the data]… The amount of red meat consumed ranged from 46.5 to 500 g/d in the red meat interventions and 0 to 266 g/d in the comparison diets [there is a huge overlap in the two groups, making differences impossible to identify]. Minimally-processed red meat was consumed in 24 studies, processed red meat was consumed in 5 studies, and the extent of red meat processing was not reported in 8 studies [so different red meat was consumed, meaning if processed red meat is harmful but not whole food red meat, we cannot distinguish]… Dietary records or food-frequency questionnaires were used in all but 3 studies [dietary food-frequencies are historically unreliable]”¹

From Table 1, I grabbed some examples for the red meat intervention group’s diet:

“≈70% protein from lean beef, pork, veal, eggs and egg substitutes, and partially skimmed milk and milk products; no alcohol; 7-day rotating menu meeting energy needs, some food provided [Gascon 1995]… Habitual diet + red meat, 15 g PUFA margarine, no soy products, fats were provided [Ashton 2000], Usual diet + 8 servings/fortnight of red meat, protein source and other food provided [Grieger 2014]”

From Table 1, I grabbed some examples for the comparison diet—matched with the database used for the red meat intervention diet:

“≈70% protein from lean white fish (cod, sole, pollack, and haddock with <1% fat); calcium (500 mg) and vitamin D (125 IU) supplement; no alcohol; 7-day menu meeting energy needs, some food provided [Gascom 1995]… The tofu diet was designed to replace 90–100% of the animal protein with 290 g of tofu, recommendations to consume 5 g butter, 5 g lard, and 8 mL olive oil, tofu, and fats provided [Ashton 2000]… 8 servings/fortnight of raw, canned, and marinated mixed fish, in addition to usual diet, which may include red meat [Grieger 2014]”

Note how the nutrition consumed is not defined and are quite different in how they appear. Statistically not much can be gained from mixed data like this.

Interestingly, they found:

“With interventions in which only lean red meat was consumed, relative to all comparison diets, red meat yielded greater decreases in total cholesterol (WMD, –0.05 mmol/L; 95% CI, –0.12 to –0.02; P=0.04) and LDL-C (WMD, –0.08 mmol/L; 95% CI, –0.15 to –0.02; P=0.03), but lesser decreases in triglycerides (WMD, 0.10 mmol/L; 95% CI, 0.02–0.18; P=0.04). We observed a trend for red meat to yield greater decreases in total cholesterol and LDL-C when saturated fat intake in the comparison diet was higher than in the red meat group (≥5% difference). No significant differential effects of red meat were observed for total cholesterol or LDL-C when dietary saturated fat intake in the red meat group was higher or similar to that in the comparison diet… The dose-response meta-analyses showed no significant effects of red meat intake… Compared with high-quality plant protein sources, red meat yielded lesser decreases in total cholesterol (WMD, 0.264 mmol/L; 95% CI, 0.144–0.383; P<0.001) and LDL-C (WMD, 0.198 mmol/L; 95% CI, 0.065–0.330; P=0.003). Relative to fish-only comparison diets, red meat resulted in greater decreases in total cholesterol (WMD, –0.109 mmol/L; 95% CI, –0.211 to –0.007; P<0.036), LDL-C (WMD, –0.173 mmol/L; 95% CI, –0.260 to –0.086; P<0.001), and HDL-C (WMD, –0.065 mmol/L; 95% CI, –0.109 to –0.020; P=0.004). In comparison with chicken or poultry diets, red meat showed no significant differential effects on lipid variables. When considering poultry and fish together as the comparison, red meat yielded greater decreases in total cholesterol (WMD, –0.092 mmol/L; 95% CI, –0.177 to –0.008; P=0.032) but lesser decreases in triglyceride concentrations (WMD, 0.224 mmol/L; 95% CI, 0.077–0.371; P=0.003)…” and and and…

OK, stop right here…

Do you know what 0.05 mmol/L total cholesterol decrease is in mg/dL? 1.9335 mg/dL… and this is how much red meat consumption actually lowered total cholesterol relative to comparison diet and this is significant at P=0.04? Are they serious?

The greatest mean change from one cholesterol level to another in any group here is 0.264 mmol/L which is 10.209 mg/dL (verify using this converter). Are they seriously writing a paper about a 10 mg/dL change in cholesterol?

I hope you know that cholesterol values change hourly, so a 10 mg/dL change can happen between two blood tests taken a breath apart. David Feldman has been playing with his cholesterol levels for years, showing how easy it is to get different cholesterol levels every day if you wish. So what’s this about cholesterol change of 1-10 mg/dL?

Ummm… nothing. Below is the table that summarizes their cholesterol findings.

All Foods in the Metadata Analysis

In addition, note that red meat really didn’t have a bad effect on cholesterol. If we decide to add any importance to whatever this paper is about, we must note that red meat seemed to have an inverse relationship, in terms of cholesterol, even with fish—a well-known cholesterol reducing agent—and red meat provided better cholesterol reduction—of course the amount it changed is irrelevant.

Tada! The Fruit of Hard Labor Revealed

The table that shows how much the cholesterol was lowered when consuming a diet rich in vegetables, legumes, nuts and seeds:

veggies, nuts, and veg oils

The changes are not only not impressive but are ridiculous.

In addition, there are many research papers showing that the reduction of cholesterol is not only of no benefit in preventing heart disease but may be harmful. Research shows that cholesterol has a protective role against infections, among other things, and in older people and women in particular, it seems that higher cholesterol leads to longer and healthier lives^2,3.

Publishing Fees

So you wonder perhaps why I wrote all this on my blog and have not sent it as a Letter to the Editor to the journal Circulation. Circulation is a highly irregular journal! They charge publishing fee for all articles—including Letter to the Editor!

When a journal charges publishing fees for a non-open-access article, meaning it will not be readable by anyone freely and openly, I always wonder what is behind the journal and the articles it publishes. The Circulation charges a great sum for publishing a no-access article—and these charges are in addition to having become a journal subscriber to get access to read the article or to pay a fee per article should you wish to read one.

So the journal is making money on both ends, including charging for publishing a Letter to the Editor; scroll down to read “Special Sections” under which you will find “Note that as with all other article types, Letters to the Editor which are accepted for publication will incur an article publication charge”! This is highly unusual and irregular for several reasons.

It discourages the critique of any badly done study—would you pay $70 to have 500 words max published in a critique of an article that no one will be able to read without having a membership or pay for the article? And that’s the other problem: no one can read the critique so it will be swept under the carpet.

Here is the fee structure of the Circulation:

• Cost to Authors of Non Open Access Articles
• Fee per printed black and white page: $70;
• Fee per online-only published page: $35;
• Fee per printed color page: $723;
• Supplemental fee for exceeding the word limit: Authors of papers exceeding our 7,500 word limit will be charged for the overage. Minimum fee for papers exceeding this maximum will be $425. In addition, authors will be charged an additional $425 for each additional 1,000 words over 7,500. The usual $70 page charge will also apply. Word count will be calculated by the editorial office, using the Microsoft Word tool. Title page, abstract, references, tables and figures legends are not included in the total word count.
• Reprint expenses: Price lists are sent with author’s proofs.
• Excessive author alterations to typeset page: $50 per page.
• Correction (erratum): $100 per page if error results from an author’s oversight.

Would you submit a letter to the editor? Me neither… now you understand why I wrote it on my blog.

Additional Sources

1          Guasch-Ferré, M. et al. Meta-Analysis of Randomized Controlled Trials of Red Meat Consumption in Comparison With Various Comparison Diets on Cardiovascular Risk Factors. Circulation 139, 1828-1845, doi:doi:10.1161/CIRCULATIONAHA.118.035225 (2019).
2          Petursson, H., Sigurdsson, J. A., Bengtsson, C., Nilsen, T. I. L. & Getz, L. Is the use of cholesterol in mortality risk algorithms in clinical guidelines valid? Ten years prospective data from the Norwegian HUNT 2 study. Journal of Evaluation in Clinical Practice 18, 159-168, doi:10.1111/j.1365-2753.2011.01767.x (2012).
3          Ravnskov, U. et al. Lack of an association or an inverse association between low-density-lipoprotein cholesterol and mortality in the elderly: a systematic review. BMJ Open 6, doi:10.1136/bmjopen-2015-010401 (2016).

Comments are welcome, as always, and are monitored for appropriateness.

Angela