http://arstechnica.co.uk/science/2015/09/the-lancet-attacks-uk-health-agencys-claim-that-e-cigarettes-are-95-less-harmful-than-tobacco/
Health agency’s report on e-cigarettes used shaky evidence, ignored conflicts of interest.
An editorial in the current issue of The Lancet criticises Public Health England (PHE) for using weak evidence in its recent review of evidence on e-cigarettes, and a press release that followed.
In particular, The Lancet takes aim at the claim that e-cigarettes are around 95 percent less harmful than tobacco. It argues that the evidence for this statistic is weak, and that it originates with researchers who have relevant conflicts of interest.
Weighing up the potential benefits and harms of e-cigarettes has become an important public health question. The PHE report, authored by addiction researchers at King’s College London and Queen Mary University of London, makes the case that e-cigarettes are substantially less harmful than cigarettes, and seem to help smokers to quit more successfully.
This means, the report argues, that e-cigarettes could be very helpful in reducing smoking in the UK, lowering smoking-related death and disease while bringing down public health costs. However, according to the report, public perception of e-cigarettes is an obstacle. Surveys suggest that people are increasingly nervous about potential harm from e-cigarettes, with many people reporting that they think of them as “just as harmful” as cigarettes.
“There is a need to publicise the current best estimate that using EC is around 95 percent safer than smoking,” the report suggests. And publicising is what PHE did: a press release went out entitled “E-cigarettes around 95 percent less harmful than tobacco.” It was picked up widely in the media, with some reports suggesting that the figure was established directly by the PHE review.
The horse’s mouth
It wasn’t. The report cites two references for the figure: a paper in the journal European Addiction Research, and a review (PDF) of e-cigarette evidence presented to the UK’s All Party Parliamentary Group on Smoking and Health. The review in turn cites the European Addiction Research paper as its source for the statistic, so it’s safe to assume that this paper is where it originated.
The paper, authored by a group of scientists led by addiction researcher David Nutt, was the result of a collaboration exploring the harms of nicotine products. To estimate how harmful tobacco and other nicotine products are, the group took into account harm to the user (such as disease and addiction) and harms to non-users (like disease and environmental degradation).
The scores of various products were calculated, with cigarettes notching up a score of 99.6 out of a possible hundred, and e-cigarettes scoring only 4. However, Nutt and his co-authors note a number of caveats that encourage caution with these results. Most importantly, they note that there just isn’t much evidence available.
They go so far as to say that there is a “lack of hard evidence for the harms of most of the products on most of the criteria.” In other words, they were estimating. Estimating is a useful starting point, but it’s shaky ground for a litany of headlines. It’s even shakier ground when you consider that the paper declared relevant conflicts of interest for three of its 11 authors.
Of course, conflicts of interest don’t necessarily mean that the research is shady: that’s why they’re often declared right out in the open, on the paper itself. But the conflicts of interest combined with the weaknesses in the estimates of harm should invite caution about the conclusions, The Lancet argues: “the reliance by PHE on work that the authors themselves accept as methodologically weak raises serious questions not only about the conclusions of the PHE report, but also about the quality of the agency’s peer review process.”
Evidence and public perception
The authors of the PHE report responded to the editorial, arguing that its criticisms were based on “perceived flaws in one of the 185 references we used, ignoring the rest of our 111 page document.” There was a great deal more in the report, they write, which delved into evidence on how good e-cigarettes are at helping people quit, whether they’re likely to encourage people to start smoking, and other relevant questions.
The report’s authors also offer different grounds for the contested statistic: that the compounds we know to be harmful in cigarettes are either absent in e-cigarettes, or present at levels below 5 percent of their cigarette doses. Meanwhile, nothing that’s in e-cigarettes (but not cigarettes) has been shown to be harmful (yet). There’s no source cited for these calculations, and it’s not clear why the authors are claiming a different source for the statistic from the one they cited in their original report.
The public misinformation about the harms of e-cigarettes, combined with the potential benefit they hold for helping people to quit, justifies the publicity surrounding the “95 percent” statistic, the authors argue. They’re backed up by PHE directors in a further response to the editorial, arguing that other important findings, like the growing perception of harm from e-cigarettes, have been neglected in the critique.
The debate here really separates into two separate issues. The first is about the actual evidence for harms and benefits of e-cigarettes, which is still a very hazy picture. It’s still too early to know what the long-term effects might be, and the evidence for how helpful they are for quitters is mixed—even according to the review cited by the PHE report. Moreover, a recent Journal of the American Medical Association paper reported that we might be optimistic in thinking that e-cigarettes don’t get people started on smoking: they found that teenagers who had smoked e-cigarettes were more likely to start smoking tobacco.
The second multifaceted issue, which is less easily resolved by lots of long-term research, is about public information, over-hyped and uncritical reporting in the media, and the over-hyped press releases underlying that naive media coverage. Minds can be slow to change on matters of public health, and the ethics of reporting shaky statistics that could shape health perceptions for years to come are worthy of The Lancet’s scrutiny.