[Added note, Dec 14, 2016] For reporters or anyone else focusing on Dr. Mark Hefland’s comments. It’s important to put his comments in the larger context, which is that overall, The BMJ came out strongly in favor of the article, as reflected not only in the journal’s press release but also, quite powerfully, in comments by Editor-in-Chief Fiona Godlee. Moreover, the one other reviewer, Dr. Bero, was appropriately neutral in her comments and addressed only the facts. Dr. Hefland was the outlier. He made a number of extraneous, critical remarks, without substantiation, reflecting his own opinions. I believe that focusing on his opinions alone, as some critics of mine have done, is neither fair nor balanced coverage of this issue.]
Response to Dr. Hefland
[Posted 12/2, along with all my comments published by The BMJ. Minor edits on 12/14/16]
In his review of my article Dr. Mark Hefland finds that my assertions are within the realm of legitimate scientific discussion and concludes that the piece does not merit retraction. He writes:
“It is entirely legitimate to raise and debate the overarching, central claim that the DGAC did not review the science impartially and consequently drew the wrong conclusions from it. In raising the question of bias, it is also reasonable to contrast the panel’s endorsement of a vegetarian diet despite weak evidence with its strong stand against saturated fats despite conflicting evidence. Previous versions of the Dietary Guidelines have been criticized for relying too heavily on observational studies and on expert consensus.(4) The DGAC’s conclusions are largely unchanged from the previous Dietary Guidelines, raising the question of whether their modified process has the same problems.(5)of me and the article. ”
Hefland also writes that he “found nothing to contradict Teicholz’ central concern that the DGAC’ processes to protect against bias are inadequate.”
Yet Hefland makes a number of new, unsubstantiated criticisms of the piece. These are entirely extraneous–outside the scope of what he was asked to do by The BMJ. Moreover, they are all negative, suggesting a bias on his part against me or this article.. It’s notable that Hefand is the director of a center that is both funded by, and solely in service to, the US Department of HHS, which was the lead government agency for the 2015 DGAs, the very document critiqued in this article. I believe this represents a rather large potential conflict of interest and could very well be why Hefland takes such a critical tack.
Two important points:
- None of Hefland’s new accusations are substantiated by any factual information
- On most of the points where Hefland argued for corrections to the piece, he failed to make a persuasive case. The BMJ did not agree with him. In other words, his arguments against the article were found to be in error.
In sum, Hefland makes unsubstantiated and erroneous arguments against this article, which point to significant bias on his part. Whether this is due to the fact that he is a contractor to HHS is unknown.
Below I respond to each of his arguments sequentially, using the alpha-numeric system he uses in his comment for reference. Overall, the only finding by Hefland that has been upheld by The BMJ is that for two reviews, I did, indeed, overlook pre-defined methods for the literature searches. Yet even this finding does not change my article’s assertion that the reviews in question was still unsystematic in a number of ways.
E. Hefland claims that my article has “major deficiencies” and that my arguments are “contrived and superficial,” which he substantiates by saying that (1) I failed to acknowledge that the DGAC stated, in advance, the types of evidence it would use, and (2) that I should not have critiqued the DGAC’s use of outside review papers.
My reply: Hefland misunderstands the claims of my article on both these points. (1) I do, in fact, acknowledge, in my Rapid Response, that the DGAC provided a general methodology section in which it describes the general types of evidence used. However, the mere fact of stating methods in advance does not imply that these methods were systematic. A systematic review is defined by many more factors than simply whether general categories of evidence have been pre-defined. The finding in my article, as confirmed by both outside reviewers is that much of the expert report did not, in fact, employ a rigorously systematic approach. (2) My article does not make any claim that the DGAC should not have used outside review papers; the article objects only to the use of outside review papers by the American College of Cardiology and the American Heart Association, since these are private associations supported by industry and might have introduced a potential conflict of interest.
Thus, I believe that Hefland’s negative judgments about my article are not justified by the facts he presents.
F. Hefland documents that I self-published a thorough critique of the 2015 DGAC report and that this was funded by the Laura and John Arnold Foundation.
My reply: My BMJ piece was indeed based on a portion of the findings in my self-published report, which I largely completed in the spring of 2015. I felt compelled to do this research because, like many people who follow nutrition science, I could not understand how the expert report could reach the conclusions that it did; On many topics, the report seemed to ignore either the best or most recent science, and I sought to understand why. The last decade-plus of my life has been devoted to trying to ascertain the truth about nutrition science and policy. Foundations often fund work by journalists on topics of public interest, so there is nothing untoward in this arrangement.
H. Hefland states my work is “poorly researched” and “poorly documented” because I did not cite interviews with people whom he thought should have been consulted.
My reply: Hefland might not understand the work of journalism, wherein much research is conducted without necessarily citing every interview or source. In fact, I did interview members of the federal government, other experts, and former members of the DGAC for this article; Hefland has no factual basis for asserting otherwise (and he never contacted me to ask). Most of the subjects I interviewed did not want to be quoted. I also contacted the chair of the DGAC, Barbara Millen, and gave her the opportunity to respond to every claim in my article. Her responses are quoted in the piece. I did not watch all the video proceedings of the DGAC, because any relevant material should have been included in the 471-page expert report, which I read in its entirety. Moreover, The BMJ published 3 tables that I compiled, comprising hundreds of facts–altogether an enormous amount of supporting evidence; perhaps Hefland missed these.
J.through U. Hefland asserts that there are 4 errors in the piece. In 3 cases, The BMJ agreed with my arguments and downgraded Hefland’s proposed corrections to “clarifications.” On the 4th issue, regarding the questions of systematic methodology, The BMJ agreed that corrections were needed due to the fact that for two reviews, I overlooked pre-defined methods for the literature searches.
V. Hefland returns to the same points he made in E. and H. without any greater documentation for his assertions.
W. Hefland objects to the classification of the article as an “investigation.” How the article was classified was entirely up to The BMJ.
X. Hefland objects to my critiquing the science behind the Guidelines in other settings. I see no inconsistency in making the same science-based arguments, solely in the interest of the public health, to different audiences. My research on the 2015 DGA expert report was conducted in spring of 2015 (see background here), with no idea that it would become a BMJ article nor that I might eventually have the opportunity to present this and other research to policy makers in D.C.; to call it a “lobbying document” is thus demonstrably false and possibly defamatory. The arguments that I have made are consistent with aims of any good journalist: to document ways in which a policy is not sustained by the evidence. This is important work, especially when there is evidence that this policy is significant causing harm to the public.
Y. Hefland repeats the same claims made in E. without any greater documentation for his assertions.
Z. Despite all of the above, Hefland affirms one of the article’s chief claims: “I found nothing to contradict Teicholz’ central concern that the DGAC’ processes to protect against bias are inadequate.”
Click here for my complete comment on The BMJ review.
Note: Dr. Hefland is the Director of a center that is part of AHRQ, the very US-HHS office in charge of the Dietary Guidelines for Americans.
