The China Study vs the China study

..man, proud man,

Dress’d in a little brief authority,
Most ignorant of what he’s most assur’d…

From Measure for Measure by Wm Shakespeare

The web has been alive with commentary the past few
weeks since Denise Minger lobbed her first cannonball
of a critique across the bow of The China Study, the
vessel T. Colin Campbell, Ph.D. rode to fame and
bestsellerdom. Seems like everyone is now jumping into
the fray and gunning for poor Dr. Campbell, who early
on in the fracas made a few halfhearted attempts to fight
back but has now fled the scene. I’ve been laying low
watching it all play out, and so now figured it’s about
time I add my two cents worth to the debate. But first a
little history.

I met Dr. Campbell about ten years ago (five or so years

before the publication of the popular book The China
Study) when we both spoke at the same conference. He
was a nice enough man who spoke about the work he
and his team had done in China gathering the data
published in the massive 894 page monograph Diet, Life-style and Mortality in China (pictured above left).
As Dr. Campbell presented his data ‘demonstrating’ the superiority of a plant-based diet and demonizing
protein of animal origin, I didn’t think much about it because the data was all in the form of observational
studies, which, as all readers of this blog should know by now, despite often showing correlation don’t
prove causation. My lecture, which followed Dr. Campbell’s, was, as you might imagine, a lecture of a
different sort. Then we both sat on a panel after our talks and fielded questions. And were both cordial to
one another.

A few years ago, I became vaguely aware that Dr. Campbell had written a popular book on his work in
China titled, appropriately enough, The China Study. I assumed it pretty much mirrored his presentation I
had watched, so didn’t rush out and grab a copy. Over the past few years a number of people have asked
about The China Study through the comments section of this blog, and I’ve typically answered that the data
are all observational and so not really meaningful in terms of causation.

(Note: Throughout this post whenever I refer to the popular book Dr. Campbell wrote, I’ll call it by it’s title
The China Study, and when I refer to the large study Dr. Campbell was involved with in China and was the
basis for the monograph Diet, Life-style and Mortality in China, I’ll call it the China study.)

About a year ago, I wrote a guest post for Tim Ferriss’s The Four Hour Workweek blog. It actually wasn’t
a guest post as much as it was an excerpt of a chapter from our book The Six-Week Cure for the Middle-aged
Middle extolling the virtues of saturated fat. It was a popular post that has garnered to date 520 plus
comments, many of them fairly spirited. I agreed to answer a number of the comments and did so. I
noticed as I sifted through them that a handful were absolutely fawning of Dr. Campbell and The China
Study. Here is a sampling:

The number one study of diet and disease is the China Study. All other data points are slivers compared
to the volume of data and statistical correlations that came from the China Study.
 Have you read The China Study? Dr. Campbell points out repeatedly that none of the weight loss
studies such as Atkins or South Beach diet follow any type of peer reviewed scientific method.

Tim…and to think I was such a big fan of yours. This is by far the weakest (and least cited) argument I
have ever read on diet–especially increasing saturated fats. Half knowledge is a scary thing in the hands
of influential people. Maybe it’s another genius marketing ploy (like the myth riddled protein Atkin’s
diet)–people love to feel good about their personal yet poor decision making–and diet is very personal.
Check out researchers that actually meant to study nutrition–like Dr. T. Colin Campbell’s The China
Study comes to mind.

It was pretty apparent that the disease of non-critical thinking was at epidemic proportions.

After reading a number of these, I decided I had better take a look to see what Dr. Campbell had going on
that had attracted such devotees. I pulled up his book on Amazon and read through a few comments, most
of which were even more nauseatingly gushing than the above. I ordered a copy of The China Study.

I knew that both Anthony Colpo and Chris Masterjohn had done their own critiques of the original data, so
I figured, what the hell, I’ll take a look at the ‘real’ China study (as opposed to the popular book of that
name) and do one too. And I’ll critique the popular book, which I figured was a rehash of the China
project, while I’m at it.

I tracked down a copy of the 894 page book in a bookstore in the UK and forked over $240 to purchase it
and have it shipped. As I was awaiting its arrival I told Gary Taubes what I had done, and he replied that
he had done the same thing himself a few years earlier. And that I could have borrowed his. And, even
worse, that most of the data was available online for free.

When the book arrived, I was amazed at the size of it. Not only was it the 894 pages as advertised, it was
in a large format. Much larger than a volume of the Enclyclopaedia Britannica. It wasn’t at all what I
thought it would be.

Here are a couple of photographs shamelessly using our own book to show the size of this behemoth

Of the 894 pages, the first 82 are a study overview, description of methodology and author commentary. It
is written in the form of a scientific paper with half the page in English and half in Chinese (which,
presumably, is a translation of the English half). The remainder of the 894 pages are raw data and
correlations. Page after page after page of correlations. I didn’t bother counting them, but Dr. Campbell
says there are 367 variables, each of which is compared with every other variable. I don’t doubt him. This
study was a massive undertaking, requiring thousands upon thousands of man hours and God only knows
how much money. No one can possibly accuse the team members of not giving it their all.
Here is one page of correlations. This one between stearic acid and all the other variables studied.

But in the end it is still only an observational study. And even though – again, according to Dr. Campbell
– there are over 8000 statistically significant correlations, correlations are not causation. Any scientist worth
his/her salt will tell you that all you can do with data from observational studies is use them to form
hypotheses that can be rigorously tested in randomized, controlled trials. Then and only then (assuming
the study results show it) can you even begin to talk about causation. The whole enterprise, costly and time
consuming though it was, was described perfectly by Shakespeare in the words of MacBeth:

…it is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.

Once I saw that the original China study was nothing but a huge number of correlations, I quickly lost
interest. What is the point in going through the brain damage of ferreting around in these to see if Dr.
Campbell interpreted them correctly when he tries to make his case that a plant-based diet is optimal. It
doesn’t really matter whether he interprets them correctly or not, they are only correlations. Repeat after
me one last time: Correlation is not causation, correlation is not causation, correlation is not causation…

I wondered why Dr. Campbell and his group didn’t spend a fraction of the time and money they spent on
this behemoth of a spreadsheet full of correlations on a real study that could provide hard evidence. Why
not randomize subjects into two groups and provide one a plant-based diet and the other a meat-based diet
or something similar. Lock them down as Ancel Keys did if they had to. Surely the money spent on the
China study could’ve covered that. Get some real data. I discovered later that I wasn’t the only one who
wondered that. Even some of Dr. Campbell’s own colleagues abandoned him to this study and told him it
would be worthless. More about this later.

So enough for me. I stuck my copy of the $240 book of correlations in my library and forgot about it. Until
Denise Minger’s critique hit the net.

Upon reading her blog post, my first reaction was This is great; someone took the time to do what I was
going to do. I figured Dr. Campbell had cherry picked his correlations to make the case he wanted to
make, and I had seen Colpo and Masterjohn catch him on it. Ms. Minger went even further and really
caught Dr. Campbell with his pants down, correlation-misinterpretation speaking. I continued to read with
mounting glee Ms Minger’s successive critiques and a few other bloggers who had critiques of their own.
(Believe me, there is no dearth of material here for people to attack without any two attacking the same
data twice.)

After this went on for a while, I had my second reaction to the whole affair.

Which was that I had fallen victim to the confirmation bias. My bias was that Dr. Campbell was wrong, so
I was more than happy to uncritically accept evidence confirming his error without lifting a finger to
double check said evidence myself. I knew that if a blogger somewhere had come out with a long post
describing an analysis of the China study demonstrating the validity of all of Dr. Campbell’s notions of the
superiority of the plant-based diet, I would’ve been all over it looking for analytical errors. But since Ms.
Minger’s work accorded with my own beliefs, my confirmation bias ensured that I accepted it at face value.

Once the fact that I had succumbed to my confirmation bias settled in around me, I became suffused with
angst. I had tweeted and retweeted Ms. Minger’s analysis a number of times, giving the impression that I
had at least minimally checked it out and had approved it. I had emailed it to a number of people, many
of whom, I’m sure, had forwarded it on. I’m sure I played a fairly large role in the rapid dissemination of
the anti Campbell/China study info.

(It didn’t really make me feel better to know I wasn’t alone in falling into the confirmation bias quicksand.
Take a look at this post from Richard Nikoley’s Free the Animal blog. I doubt that all these people checked
Ms. Minger’s calculations before posting.)

My angst wasn’t because I had possibly fed the flames of a misinformation wildfire – I wasn’t particularly
worried about that because mountains of other data (including first hand data from my own clinical
practice) have persuaded me that Dr. Campbell is dead wrong in his ideas about the superiority of a plant-
based diet. No, my angst arose for two other reasons: first, because I was distressed that I so easily fell
prey to the confirmation bias, and, second, because I felt I needed to go through all the calculations myself
to make sure Ms. Minger and others whose work I had circulated were truly correct in their analyses.

As I was wallowing in self pity over all this, I didn’t realize that salvation was at hand. And that my savior
was none other than Dr. T. Colin Campbell himself.

Yep, his first response to Denise Minger’s critique of his work appeared on the Tynan.net website and
rescued me from my pit of self-loathing. In it, Dr. Campbell wrote:

But she suffers one major flaw that seeps into her entire analysis by focusing on the selection of
univariate correlations to make her arguments (univariate correlations in a study like this means, for
example, comparing 2 variables–like dietary fat and breast cancer–within a very large database where
there will undoubtedly be many factors that could incorrectly negate or enhance a possible correlation).
She acknowledges this problem in several places but still turns around and displays data sets of
univariate correlations.

In other words, the China study is an observational study comparing one variable to another (univariate
correlations) and, as such, meaningless. And this from the man’s own pen.

Since these observational studies are meaningless in terms of causality, it doesn’t really matter how one
slices and dices the data because meaningless correlations by any other names are still just as meaningless.
All this falderal over whether or not Dr. Campbell had his interpretations right was tantamount to the
medieval theological argument over how many angels could stand on the head of a pin. And my
participation certainly wasn’t required.
I’d known this all before, of course, but somehow had lost my focus on it.

I was ready to wash my hands of the whole affair when I came across another statement Dr. Campbell
made in his response to Ms. Minger’s critique. Writing of her, he said:

One further flaw…is her assumption that it was the China project itself, almost standing alone, that
determined my conclusions for the book (it was only one chapter!). She, and others like her, ignore much
of the rest of the book.

Only one chapter? As I mentioned above, I always figured The China Study was simply Dr. Campbell’s tale
of the China study and the conclusions he had drawn from it. Now he says that only one chapter is about
the China study, leaving me to conclude that the rest must be about something else. I found the book,
which I hadn’t yet taken from the pack it came in from Amazon, opened it and started reading.

Wow!

In 1976 author Mary McCarthy famously said live on the Dick Cavett show of her rival Lillian Hellman:

Every word she writes is a lie, including ‘and’ and ‘the’.**

I feel much the same way about The China Study. Except it’s not
really a lie, it’s an obfuscation.

In fact, in my studied opinion, The China Study is a masterpiece

of obfuscation.

It is obfuscatory in so many ways it could truly qualify as a

work of obfuscatory genius. It would be difficult for a mere
mortal to pen so much confusion, ambiguity, distortion and
misunderstanding in what is basically a book-length argument
for a personal opinion masquerading as hard science.

Let me take just one tiny section of the book, one that is in no
way atypical, and show you what I mean.

In Chapter 3 titled Turning Off Cancer, Dr. Campbell is starting

to hit his stride in his anti animal protein jihad. He has
described the three stages of cancer – initiation, promotion and
progression – and is setting the stage for his description of his
laboratory work implicating animal protein in all three stages.

Here is his setup paragraph starting on page 50:

At the start of our research, the stages of cancer formation were known only in vague outline. But we
knew enough about these stages of cancer to be able to structure our research more intelligently. We
had no shortage of questions. Could we confirm the findings from India that a low-protein diet represses
tumor formation? More importantly, why does protein affect the cancer process? What are the
mechanisms; that is, how does protein work? With plenty of questions to be answered, we went about
our experimental studies meticulously and in depth in order to obtain results that would withstand the
harshest of scrutiny.

The “findings from India that a low-protein diet represses tumor formation” were the results of a rodent
study published in the Archives of Pathology in 1968 that Dr. Campbell wrote about 14 pages earlier in the
book. He mysteriously refers to the Archives of Pathology as an obscure journal when it is anything but. It
was published then by the American Medical Association and still is today under the new name Archives of
Pathology & Laboratory Medicine. But the notion of the paper initiating his quest being discovered by Dr.
Campbell in an “obscure medical journal” fosters the impression of him as a leave-no-stone-unturned kind
of guy. Even the little throw away but incorrect phrase “obscure medical journal” is part of the greater
picture of obfuscation that maintains throughout the book.

The study from India showed that rats given aflatoxin along with a high-protein diet got liver cancer while
rats given the same amount of aflatoxin while consuming a low-protein diet didn’t. Aflatoxin is a
substance released from a fungus often found in peanuts, corn, other grains and even hay. It is converted in
the liver to a much more toxic compound and is often used in laboratory experiments with animals to
induce cancer and other problems.

Moving on, here is what Dr. Campbell has to say about protein and cancer initiation:

[I] How does protein intake affect cancer initiation? Our first test was to see whether protein intake
affected the enzyme principally responsible for aflatoxin metabolism, the mixed function oxidase (MFO).
This enzyme is very complex because it also metabolizes pharmaceuticals and other chemicals, friend or
foe to the body. Paradoxically this enzyme both detoxifies and activates aflatoxin. It is an extraordinary
transformation substance.

[II] At the time we started our research, we hypothesized that the protein we consume alters tumor
growth by changing how aflatoxin is detoxified by enzymes present in the liver.

[III] We initially determined whether the amount of protein that we eat could change this enzyme
activity. After a series of experiments, the answer was clear (Chart 3.2). Enzyme activity could be
easily modified simply be changing the level of protein intake.

[IV] Decreasing protein intake like that done in the original research in India (20% to 5%) not only
greatly decreased enzyme activity, but did so very quickly. What does this mean? Decreasing enzyme
activity via low-protein diets implied that less aflatoxin was being transformed into the dangerous
aflatoxin metabolite that had the potential to bind and to mutate DNA.

These four little paragraphs and accompanying chart take up less than a page in space, and are tiny
glittering gems of obfuscation. Let’s deconstruct.

First, take a look at how subtly these four paragraphs are written, especially II. Note how he writes “the
protein we consume”? I’m sure many people took these paragraphs to mean that the studies were done on
humans. That’s almost the implication. Reread them to see if they indicate anywhere that the author is
talking about rat studies.

As Dr. Campbell progresses through this chapter, he does ultimately tell the reader he is talking about rat
studies and not human studies, but he doesn’t mention the word rat for another two pages after the above
paragraphs. By this time it’s probably implanted in the minds of many readers that he’s talking about
human studies.

He describes experiments showing that rats getting diets high in casein (a milk/animal protein) develop
more cancer at the same dose of aflatoxin than do rats getting a lower-casein diet. The implication: animal
protein causes cancer.

Dr. Campbell then gave his rats diets of varying amounts of plant protein (wheat gluten) and found that
they did not get cancer after exposure to aflatoxin irrespective of protein dose. Same thing happened with
soy. Implication: plant protein protects against cancer.

If you’re worried about cancer – and who isn’t – you’re now starting to look at animal protein a little
differently. Which is what Dr. Campbell wants. But he hasn’t told you the complete story.

As I’ve written often in these pages, rodents aren’t just furry little humans. They are a distinct species
separate and apart from humans. The rodents usually used in lab experiments are Sprague-Dawley rats,
and inbred strain that has a tendency to develop cancer easily. (See Abelson, PH. (1992) Diet and Cancer in
Humans and Rodents, Science 255(5041); Jan 10: 141) In fact, these rats can develop cancer just from a
change in diet. I ran quick checks on a bunch of the studies referenced in The China Study, and all checked
used Sprague-Dawley rats.

And think about this. If you were to visit a farm and search for rodents, where do you think you would be
most likely to find them? In the grain or in the milking area? Like Dr. Campbell, I grew up in a rural area
and spent a lot of time on a farm. Rats and mice are in the hay and in the grain. You have a helluva time
keeping them out of the animal feed, which they eat, too. Grain and hay are common places for growth of
the fungus that produces aflatoxin. Since rodents spend most of their days in this stuff (grains), and since
they eat it as well, I would bet that most have adapted over the generations to the combination of plant
protein and aflatoxin. If this did them in regularly, there wouldn’t be the rodent problem on farms that
there is. So, in my opinion, making a huge issue of the fact that rats didn’t get cancer after dosing with
aflatoxin irrespective of how much plant protein they ate is pretty disingenuous.

Most disingenuous of all in the above four paragraphs and chart is the lack of full disclosure in these
paragraphs of the very study Chart 3.2 is made from. Let me explain.

Certain enzymes in the liver convert aflatoxin into a more toxic substance that Dr. Campbell claims can
initiate the formation of cancer. He demonstrates in rat studies that giving the rats a lower protein diet
decreases the activity of this enzyme, meaning that the lower the protein intake, the less conversion of the
aflatoxin into the really nasty stuff. Chart 3.2 above and on page 52 of his book shows this graphically.

When I pulled the study from which this chart was adapted (Mgbodile MUK and Campbell TC. (1972)
Effect of protein deprivation of male weanling rats on the kinetics of hepatic microsomal enzyme activity, J
Nutr, 102: 53-60.) and read it, I found a little disclaimer Dr. Campbell didn’t bother to mention in The China
Study. You can read the last paragraph of the study (highlighted in yellow) below:
Nice, eh? He hits the nail on the head. Protein utilization may be influenced by what is eaten along with
the protein. Sucrose (table sugar) was eaten along with the protein used in this experiment. In other
experiments corn starch was used instead of sugar and the effect of the protein on the enzyme was
diminished, meaning that the protein along with starch did not have nearly the same effect as protein with
the sugar. Who knows whether or not it’s even the protein that has the effect and not the sugar? It can’t
be shown from this study. That caveat certainly didn’t make in into The China Study.

See what I mean about a masterpiece of obfuscation?

I could go on and on, but I’ll quit after I give you just a couple more examples.

On page 107 of The China Study Dr. Campbell writes:

At the end of the day, the strength and consistency of the majority of the evidence is enough to draw
valid conclusions. Namely, whole plant-based foods are beneficial, and animal-based foods are not.

Then one inch below (literally) he writes the following:

The China Study was an important milestone in my thinking. Standing alone, it does not prove that
diet causes disease. [Italics in the original]

So, the China study produces valid conclusions as to causality, i.e., “whole plant-based foods are beneficial,
and animal-based foods are not.” Yet the China study “does not prove that diet causes disease.” Say
what?

Don’t believe me, take a look at a scan of my copy:

On page 73 Dr. Campbell dons the mantle of prestige conferred by one of America’s most august
newspapers. Writes he referring to the China study:

We had a study that was unmatched in terms of it’s comprehensiveness, quality and uniqueness. We
had what the New York Times termed “the Grand Prix of epidemiology.”

A quick search of that phrase in the online version of the NY Times reveals that it came from an opinion
piece by none other than Jane Brody, a kindred spirit to Dr. Campbell. Brody, a lipophobe of the deepest
hue, has written a number of low-fat cookbooks and is a believer in the plant-based diet. So she hardly
qualifies as an unbiased commenter.

And speaking of the so-called plant-based diet, when Dr. Campbell responded to Ms. Minger’s critique, he
took her to task for mentioning the words ‘vegan’ and ‘vegetarian’ as it applied to his work.

One final note: she repeatedly uses the ‘V’ words (vegan, vegetarian) in a way that disingenuously
suggests that this was my main motive. I am not aware that I used either of these words in the book, not
once. I wanted to focus on the science, not on these ideologies.

Just for grins, I turned to the index of The China Study to see if ‘vegan’ or ‘vegetarian’ were indexed. Here’s
what I found on page 417:

vegetarianism or veganism. See plant-based diet

When I flipped over to ‘plant-based diet’ on page 414, I found a long grocery list of references.

Even in his online response to his opponents, Dr. Campbell apparently can’t resist obfuscating.

Okay, just one more, then I’ve got to draw this to a close. Let’s go back to the bottom of page 52, the page
the paragraphs above and Chart 3.2 appear on. Dr. Campbell shows in Chart 3.2 how protein is involved
in stimulating the liver to convert aflatoxin to the toxic product that he implies is involved in cancer
initiation. He then reports how he wanted to see if animal-based protein was involved in the other phases
of the cancer progression cascade. So he and his grad students started to look. He writes:

As time passed, we were to learn something quite remarkable. Almost every time we searched for a way,
or mechanism, by which protein works to produce its effects [on cancer formation and progression], we
found one!

That, my friends, is almost the dictionary definition of the confirmation bias summed up in one sentence.

This tiny bit of the book that I’ve chosen to lay bare is truly the tip of the iceberg. I could go on and on
and on, but I’m sure you get the picture.

Before I finish, I want to get back to something I mentioned earlier about how one of Dr. Campbell’s own
colleagues bailed out from the China study because he recognized it for what it was: a giant observational
study that was meaningless. Here is how Dr. Campbell describes it on page 105-106:

When we first started this project we encountered significant resistance from some people. One of my
colleagues at Cornell, who had been involved in the early planning of the China Study, got quite heated
in one of our meetings. I had put forth the idea of investigating how lots of dietary factors, some known
but many unknown, work together to cause disease. Thus we had to measure lots of factors, regardless
of whether or not they were justified by prior research. If that was what we intended to do, he said he
wanted nothing to do with such a “shotgun” approach. [i.e., a big, meaningless observational study]

This colleague was expressing a view that was more in line with mainstream scientific thought than
with my idea [i.e., a randomized, controlled trial that might demonstrate causality would be a better use
of the funds.] He and like-minded colleagues think that science is best done when investigating single –
mostly known – factors in isolation. [He and like-minded colleagues are correct.] An array of largely
unspecified factors doesn’t show anything, they say. [They are right.] It’s okay to measure the specific
effect of, say, selenium on breast cancer, but it’s not okay to measure multiple nutritional conditions in
the same study, in the hope of identifying important dietary patterns.

I prefer the broader picture, for we are investigating the incredible complexities and subtleties of nature
 itself…

So I say we need more, not less, of the “shotgun approach.” We need more thought about overall dietary
patterns and whole foods. Does this mean that I think the shotgun approach is the only way to do
research? Of course not. Do I think that the China Study findings constitute absolute scientific proof?
Of course not. Does it provide enough information to inform some practical decision-making? [No.]
Absolutely.

Dr. Campbell uses an impassioned written speech to persuade the scientifically untrained that the China
study carries vastly more scientific value than it actually does. Once again, it’s a large observational study,
but an observational study nonetheless. And as such, it is useful only in developing hypotheses to be
tested with randomized, controlled trials. The entire 894 page study proves not a shred of causality.

What saddens me about all this is that hundreds of thousands (probably millions) of people who can’t (or
won’t) read critically have fallen for the premise of The China Study without even thinking about it.
Believing that the entire book is based on the greatest and most important nutritional study ever
completed. What happened to the ability to read critically? Has it vanished from the populace? Based on
the comments on The China Study on Amazon it would seem so.

In my opinion, there really isn’t much of substance in the entire 400 plus page book. But I encourage you
to buy it and read it to test your own critical reading skills. If you don’t want to test your critical reading
skills, you’ll at least enjoy coming across some real howlers such as this one believed only by the
vegetarian/vegan zealots out there (oh, sorry, plant-based diet followers):

As you will see in this book, there is a mountain of scientific evidence to show that the healthiest diet
you can possibly consume is a high-carbohydrate diet. [italics in the original]

I wonder if Gary Taubes, who wrote a vastly more scientific book, would agree?

Lest you think I’m being too hard on poor Dr. Campbell, let me tell you a few things. First, as I mentioned
earlier, the few sections of The China Study I dissected are just a tiny fraction of the whole. I could go on
and on. Second, Dr. Campbell mentions Protein Power by name on page 19 and labels it a modern protein
fad diet that “continue[s] to inflict a great variety of dangerous health disorders.” Third, he is absolutely
and unnecessarily brutal in his treatment of Dr. Robert Atkins. He has an entire section on Dr. Atkins
starting on page 95 that runs for almost three pages. After quoting from one of Dr. Atkins’ books, he writes
the following about the deceased diet doctor:

There are snake oil salesman, who have no professional research, professional training or professional
publications in the field of nutrition, and there are scientists, who have formal training, have conducted
research and have reported on their findings in professional forums. Perhaps it is a testament to the
poser of modern marketing savvy that an obese man with heart disease and high blood pressure [here he
inserts a citation for an article discussing Dr. Atkins’ death] became one of the richest snake oil
salesmen ever to live, selling a diet that promises to help you lose weight, to keep your heart healthy and
to normalize your blood pressure.

A way below-the-belt commentary when you consider that Dr.Atkins was a trained cardiologist who took
care of thousands of real, live patients throughout his career – he wasn’t, like Dr. Campbell, a bench
scientist doing rat studies in a lab. Bob Atkins and I have had our differences, but were he still alive, I
would vastly prefer to put my own care in his hands than I would those of Dr. Campbell, who has never
treated a patient in his life.

You may ask if I took anything of value from my reading of this book? I did. On page 107 Dr. Campbell
writes the following:
 The results of this study…convinced me to turn my dietary lifestyle around. I stopped eating meat
fifteen years ago, and I stopped eating almost all animal-based foods, including dairy, within the past six
to eight years, except on very rare occasions, MY cholesterol has dropped, even as I’ve aged; I am more
physically fit now than when I was twenty-five; and I am forty-five pounds lighter now than was when
I was thirty years old. I am now at an ideal weight for my height.

I have no reason to doubt Dr. Campbell’s own medical and dietary history (except maybe for the part about
being more physically fit than he was at age 25 – that’s a tough act for someone who is 73), so I’ll assume
it’s all true. As I recall, he had a trim physique when I met him 10 years ago, which, assuming nothing has
changed, is probably the same. And I’m going to take Dr. Campbell at his word about what he eats.

Granted, I’m younger than Dr. Campbell, but I follow almost the opposite diet as he does yet I, too, have
low cholesterol, very low blood pressure and am ideal weight for my height. What this all tells me is how
wonderfully adaptive the human species is where diet is concerned. It’s no wonder we took over the earth.

** Lillian Hellmann was predictably furious over McCarthy’s comment and adopted the typical American
response: she sued. In one of those turns in which the law of unintended consequences jumps up and bites
one, many of her untruths came to light in the courtroom as McCarthy was forced to defend her statement.
Hellmann disengaged by dying during the proceedings.

First I would like the heartily thank everyone who took the time to send me a comment on how to make
this blog better both functionally and in content. I read every single suggestion, and appreciated every
one. I’ll try to incorporate as many of the functional changes as I can within the design framework I have
and within the limits of my pocketbook. To demonstrate my profound gratitude for all the blog topic
selections, I’m going to put up a post that absolutely no one asked for. But only because I’ve had it rattling
around in my brain for the past week.

One view of the value of epidemiology

A day almost never passes without someone sending a comment my way about some recent study, plucked
by the media from the hundreds published that same day, showing that low-carb diets cause brain fog or
decreased longevity or cancer of some type or any number of conditions any of us would rather not have.
These comments always end with the plaintive request, is there any truth to this?

My answer follows: This data comes from an observational study, and, as such, can’t possibly indicate
causality.
Since I get these comments so often and answer them the same equally often, I figured it was about time to
write a post on what an observational study really is so that I can link to it when I give my standard reply.

I can then add this post to the ones on the glycemic index and relative risk, both of which serve the same
purpose. I can simply link instead of explaining what these terms mean each time I have to use them.

Observational studies – also called prospective or cohort studies and sometimes even epidemiological
studies – are the kind most often reported in the media simply because there are so many of them. These
are the studies in which researchers look for disease disparities between large populations of people with
different diets, lifestyles, medications, incomes, etc. If disease disparities are found to exist between groups,
then researchers try to make the case that the difference in diet, lifestyle, medication, etc. is the driving
force behind the disparity.

We’ve all seen these studies by the score. We read that a large study population of people is separated into
two groups based on blood levels of vitamin C. One group of subjects has high blood levels, the other
group has lower blood levels. And since every one seems to believe that vitamin C protects against the
common cold, the researchers decide to monitor these two groups for a year and find that the group with
the highest blood levels of vitamin C has the fewest colds. These findings are rushed into publication, and
soon we read everywhere that vitamin C prevents the common cold. It all seems so reasonable and so
scientific, but the truth is that these studies don’t mean squat. And the researchers who do them know it,
or at least should know it. The fact that they do know is evident in the weasel words they use in
describing their findings. You’ll read that these data ’suggest’ or that they ‘imply’ or that this ‘may cause’
that. The non-technically trained public, however, read these to say that vitamin C prevents the common
cold. And usually the media helps to sway opinion by slanting the story in the same direction.

But, you may ask, why aren’t these studies sound? If the one group with the greater blood levels of
vitamin C had significantly fewer colds, why is it such a stretch to say that vitamin C prevents colds?

I can explain by way of a game I used to play with myself as a child. I’ve never been one to sleep much
even when I was a kid. I always stayed up late and I always woke up early. My brain never seemed to
slow down. I was always ruminating on something. My way of trying to get to sleep was to try to think
of everything that could be thought of. My mind would race, and I would think of my brothers sleeping in
the room with me, their beds, my bed, the closet, the tree outside, my dad’s car, the rug on the floor, the
moon, and on and on and on. As I thought faster and faster, continuing to compile things that could be
thought of, I would finally hit a quitting point. Then I would try to figure if there was anything I hadn’t
thought of. Of course, immediately I would think of something. I hadn’t thought of the pigs on my
grandfather’s farm. Or I hadn’t thought of the fire hydrant out front. Or my father’s shoes. Or whatever.
Then I would start the game again, this time, of course, starting with the pigs on my grandfather’s farm
and going from there. I would always fall asleep before I had ever thought of everything there was to
think of.

Researchers doing observational studies have much the same problem. They try to think of all the
differences between two large populations of subjects so that they can statistically negate them so that only
the observation in question – the vitamin C level in the example above – is different between the groups.
Problem is they can never possibly think of all the differences between the groups. As a consequence, they
never have a perfect study with exactly the same number, sex, age, lifestyle, etc. on both sides with the only
difference being the study parameter. And so they don’t really ever prove anything. In fact, we would all
probably be a lot better off if all the researchers doing observational studies had followed my lead and
fallen asleep mid study.

But I’m being too harsh. These studies do have some value. Their value is in generating hypotheses.

The observational study demonstrates a correlation. In our example above, the correlation is that higher
vitamin C levels correlate (in this particular study) with lower rates of colds. So, from this data, we could
hypothesize that vitamin C prevents the common cold. But at this stage that would be just an hypothesis –
not a fact.

Once we have the hypothesis, we can then do a randomize, placebo-controlled trial. We can recruit
subjects, randomize them into two groups that are as equal as possible, especially as vitamin C levels are
concerned. Then we give one group of subjects vitamin C and the other a placebo and watch them for a
year. At the end of the year (or whatever the study period is), we break the codes, see who is on vitamin C
and who is on placebo. We already know how many got colds, so now we compare that to vitamin C
intake. We may find that those who took the vitamin C got significantly fewer colds, so we can say that
our study demonstrates that vitamin C prevents the common cold. If this same study is repeated a number
of times with the same outcome, then it can be said to be proven that vitamin C prevents colds. (This study
is, of course, hypothetical.)

But these studies are randomized trials, not observational studies. Observational studies only show
correlation, not causation, a fact that everyone doing research and reading about research should have
tattooed on their foreheads.

CORRELATION IS NOT CAUSATION

More often than not observational studies are chock full of all kinds of technical-looking graphs, charts and
tables. Many even have complicated equations. And long statistical analyses of the data derived. They are
like zombies, however. They give the appearance of scientific life, but they are really scientifically dead.
Irrespective of how many scientific baubles are strewn through them, they are nothing but observational
studies, worthwhile only as generators of hypotheses. They demonstrate only correlation, not causation.

If you want to bear with me, I’ll show you a bizarre observational study that was actually performed that
demonstrates everything you need to know about observational studies.

The study was published in 2003 in the prestigious American Journal of Epidemiology. The title of the study
is Shaving, Coronary Heart Disease, and Stroke. (Click here for free full text) This study purports to show
that the frequency of shaving correlates with risk for developing heart disease, with those men shaving less
having a greater risk.

Here’s the finding that initiated this study.

A case-control study comparing the frequency of shaving in 21 men under 43 years of age who had
suffered a myocardial infarction and 21 controls found that nine of the cases but none of the controls
shaved only every 2 or 3 days.

Someone noticed that about half of the men in a small group of subjects who had a heart attack shaved
once every two or three days. Another group of men of similar age who hadn’t had a heart attack were
designated as controls. Upon questioning it was discovered that all of the men in the control group shaved
every day. Thus the first hypothesis was born: Infrequent shaving correlates with heart attack.

The researchers had access to a large population of subjects from another ongoing study called the
Caerphilly Study. Researchers recruited 2,513 men aged 45-59 from this study and gave them
comprehensive medical workups including extensive laboratory testing.

Men were asked about their frequency of shaving by a medical interviewer during phase I. Responses
were classified into categories ranging from twice daily to once daily, every other day, or less frequently.
The 34 men with beards were not classified. These categories were dichotomized into once or twice per
day and less frequently.
The men in the study were followed for the next 20 years with follow-up exams periodically to monitor for
history of chest pain, heart attack and/or stroke.

Of the 521 men who shaved less frequently than daily, 45.1 percent died during the follow-up period, as
compared with 31.3 percent of men who shaved at least daily.

When the data were further refined it was determined that

The age-adjusted hazard ratios demonstrate increased risks of all-cause, cardiovascular disease, and non-
cardiovascular-disease mortality and all stroke events among men who shaved less frequently.

So there you have it. Proof that shaving daily prevents heart disease. Or is it?

The researchers doing this study aren’t so stupid that they really think that the act of shaving itself has
anything to do with a man’s risk for developing heart disease. In fact, they went to great lengths to show
that shaving was merely a marker for other things going on that may well have something to do with risk
for developing heart disease or increased all-cause mortality.

The one fifth (n = 521, 21.4%) of men who shaved less frequently than daily were shorter, were less
likely to be married, had a lower frequency of orgasm, and were more likely to smoke, to have angina,
and to work in manual occupations than other men.

And these are just the differences the researchers found. Had they looked harder, I’m sure they would
have found more, just like I did when I played my ‘think of everything that can be thought about’ game
with myself as a kid.

But if these researchers had really believed that the data showed that the lack of frequent shaving itself
may have been the driving force behind the development of heart disease, they may have designed a
randomized clinical trial to show causality. They could have recruited men without heart disease,
randomized them into two groups, instructed the men in one group to shave daily and the men in the
other to shave every third day. Then after 20 years the researchers could tell whether or not shaving
protects against heart disease.

But the idea that shaving itself has anything to do with heart disease is so ludicrous that no one would ever
do such a study. We can all see that. It’s a ridiculous idea. It should be obvious that the shaving or lack
thereof has nothing to do with heart disease or early death; the lack of shaving is merely a marker for all
the other conditions that are risk factors for heart disease, i.e., small stature, unmarried, smoking, lower
socioeconomic class, etc. It’s all so easy to see.

But let’s just suppose that we take this same study and substitute the term ‘elevated cholesterol’ for
‘infrequent shaving.’ Now what do we see? Let’s change one of the quotes from above to reflect this
change. What then?

Of the 521 men who had elevated cholesterol, 45.1 percent died during the follow-up period, as
compared with 31.3 percent of men who had low or normal cholesterol.

We nod our heads sagely. Suddenly we have a study that seems to make sense. But – and this is
important – it doesn’t make any more sense than the shaving study. Both are observational studies. We are
programmed to think cholesterol is bad and causes heart disease, so this second study appears reasonable
to us. It triggers our confirmation bias. We don’t believe for a second that shaving has anything to do with
heart disease, so we can easily dismiss those findings. But we are more than ready to believe that the
elevated cholesterol caused those men who had it to have heart attacks. But the reality is that both studies
are exactly the same – and neither proves anything.

If you’re interested in a longer, more in-depth article on observational studies, take a look at Gary Taubes
long piece in the New York Times a few years ago. I’ve tried to take a little different slant than he did so
that my post and his article would cover all the bases.

Cartoon above from: Smith, G. D. et al. Int. J. Epidemiol. 2001 30:1-11