To the End of Reckoning

It is not a shining moment for science or for law. 

Indeed, the story reads more like a gripping thriller — one that the reader finally sets down by the side of the bed at 3 AM thinking that it was an exciting enough tale to forgo sleep even if it was totally unbelievable.  Imagine the plot summary.  A physician, receiving handsome payments from an attorney who wants to bring a class-action suit against a vaccine manufacturer, invents a fictional disorder and fabricates data to establish a fraudulent link with the vaccine.  The stuff of Ian Fleming or John Le Carre. 

In 1998, Andrew Wakefield and twelve other others published a paper in Lancet, a prestigious medical journal, implying a link between the measles, mumps, and rubella (“MMR”) vaccine and a new syndrome of autism and bowel disease.  Many scientists were skeptical, pointing to any number of scientific weaknesses in what was reported.  But large segments of the public relied on the paper when its findings were reported in the general press.  Vaccination rates plummeted as parents determined that the risk of autism in their children was too great.  The inoculation rate dropped most dramatically in Britain, from about 92% to below 80%, but many children in other parts of the world also failed to receive the MMR vaccine as a direct consequence of the Wakefield paper.  From having been declared “under control” in the mid 1990’s in Britain, measles was declared “endemic” in Britain only ten years after publication.  The U.S. Centers for Disease Control notes that more cases of measles were reported in the United States in 2008 than in any year since before publication, with more that 90% of those having not been vaccinated (or having a vaccination status that was unknown). 

A series of articles being published in the British Medical Journal  began last week, exposing a fraud surrounding the paper that author Brian Deer likens to the Piltdown Man scandal, an elaborate hoax that disrupted the natural course of paleontology research and that took decades to uncover.  I briefly discussed Piltdown man some months ago here.   Deer describes how Wakefield was retained by attorney Richard Barr two years before the paper was published, and paid more than £435,000, six times his annual salary as a physician.  The children in the “study” that he conducted on MMR were targeted and preselected to have desired symptoms, he reinterpreted clinical records to suit his contrived syndrome, and “chiseled” histories to reach unsupported clinical diagnoses.  While admonishing many who failed to exercise sufficient diligence — coauthors, fellow scientists, hospital managers, journal editors — Deer reports that the evidence shows it was Wakefield alone who perpetrated the scandal and that not even the attorney who paid him knew what he was doing. 

When Deer first reported on irregularities surrounding the study in 2004, ten of Wakefield’s coauthors retracted the interpretation.  In January 2010, a five-member tribunal of the British General Medical Council found dozens of allegations of misconduct proved, including four counts of dishonesty and twelve involving the abuse of developmentally challenged children.  A copy of the results of the Fitness to Practise Hearing, which was the longest ever conducted by the Council, can be found here.  

The consequences of the misconduct are truly staggering.  Epidemiological studies were conducted at great public expense, unable to confirm any link between autism and the MMR vaccine.  Research funds and personnel were diverted from more legitimate avenues to understand the actual causes of autism and to help those affected by it.  Many children who might otherwise have been vaccinated suffered an illness that we have the technology to prevent and may be among the small number who died as a result of having contracted measles.  At the moment, the effect on mumps contraction remains unclear because its peak prevalence is in older adolescents. 

In its editorial, the British Medical Journal pulls no punches.  It asks whether it is possible that Wakefield was wrong but not dishonest.  After all, scientists are allowed to be wrong.  Indeed, it is a strength of the scientific method that all honest ideas should be considered so that they can be scrutinized and dismissed if they are incorrect. 

The answer it gives is a simple one:  “No.” 

Not a single one of the case studies was reported accurately.  The pattern of the misreporting shows an intent to mislead.  Already actions are being taken to examine Wakefield’s other publications, mindful of the experience that misconduct is rarely an isolated event. 

When I write about various issues on this blog, one of the things that I am reminded of repeatedly are the similar ethical requirements that attorneys and scientists are expected to adhere to.  Those ethical requirements exist because of the importance of the work that they do, and the trust that is necessarily placed in them by the public. 

Attorneys suffer a great deal from distrust by the public, and I have always thought that that is one of the prices to be paid for the system.  Attorneys deal with controversies between parties that have opposing interests and operate in a system that requires that the best arguments be put forward on behalf of their clients, most especially for those clients who seem unlikeable and potentially subject to victimization if the state or other opposing party is not held to the strictest standards of proof. 

Scientists have generally enjoyed a more favorable reputation by the public, but misconduct of this scale has repercussions that extend far beyond even vaccines and autism.  Such acts of misconduct erode public confidence in the legitimate conclusions of science, putting members of the public in a circumstance where they do not know who or what to believe.  They are inconsistent with the openness that is the very bedrock of science, which seeks to communicate not only what we do know but also what we don’t.  Because if people are to make decisions in their lives based on the results that science achieves, they deserve to know both.

Rooftop of the World

“Glaciers in the Himalaya are receding faster than in any other part of the world and, if the present rate continues, the likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the Earth keeps warming at the current rate. Its total area will likely shrink from the present 500,000 to 100,000 km2 by the year 2035.”

The quotation is from the Fourth Assessment Report issued by the Intergovernmental Panel on Climate Change (“IPCC”) in 2007. It’s a sobering statement about the impact and pace of global climate change — an astonishing vanishing of the glaciers of the Himalaya in only 25 years.

It is also wrong.

The categorical nature of the statement has caused much criticism and embarrassment for the IPCC, which has significant impact on the formation of laws to implement climate policies by nations around the world. It does, however, provide a fitting example of the role of scientific peer review and the sweeping impact that errors in that process can potentially have.

The IPCC was formed in 1988 by the World Meteorological Organization and the United Nations Environment Programme. Its principal role is to assess the scientific information available about climate change and to issue assessment reports that are used by governments in developing laws and policy. Together with former Vice President Al Gore, it was awarded the Nobel Peace Prize in 2007 for “efforts to build up and disseminate greater knowledge about man-made climate change, and to lay the foundations for the measures that are needed to counteract such change.” There is no doubt that when its credibility is tainted by bold statements that turn out to be false that it fails, at least in part, in achieving that goal.

The incorrect statement apparently has its origins in a statement made by Syed Iqbal Hasnain, an Indian scientist, in 1999. He claims he was misquoted in an interview with the science magazine New Scientist: “I had simply told the New Scientist in an interview that the mass of the glaciers will decline in 40 years…. The date (2035) was their invention.” He acknowledges that his statement itself was based on data published in the 1970’s.

When the IPCC was preparing its Assessment Report, it was subject to peer review, which is the process scientists use to evaluate statements before publication. Peer review is always important, but it is especially so when pronouncements are being made by a body with the influence that the IPCC has. That process failed in this case. But what is notable is that the statement about the Himalaya was noted and questioned by at least some of the peer reviewers — one of whom pointed out a glaring inconsistency: “100,000? You just said it will disappear.”

Another of the peer reviewers noted that the statement seemed to be at odds with other research suggesting that glaciers in the Himalaya are actually expanding, pointing out specific references that should be consulted in assessing its validity.

The end result is that the report failed to account for the criticisms of the peer reviewers and instead retained a statement that was not itself validated by peer review. In the time since the error has been pointed out, there have been numerous allegations whose veracity is hard to assess: that the statement was included deliberately, even knowing it was wrong, to prompt action by governments; that Hasnain is being untruthful when he claims not to have mentioned a date; that those involved with publication of the statement have financial interests that would be enhanced by concerns the Himalayan glaciers are rapidly disappearing; and others. Any of these would be cause for additional concern if substantiated, but my focus today is more narrow: The peer-review system worked up to a point, with peer reviewers identifying the weaknesses in the statements. But it takes attention and diligence on the part of all involved with that process for it truly to function the way it is intended.

Last week, the InterAcademy Council (“IAC”), a multinational organization of scientific academies, issued its Review of the IPCC. While it noted an “essential” need for “some fundamental changes to the process and the management structure,” its tenor was generally positive about the contributions the IPCC has made: “[T]he IPCC assessment process has been successful overall and has served society well. The commitment of many thousands of the world’s leading scientists and other experts to the assessment process and to the communication of the nature of our understanding of the changing climate, its impacts, and possible adaptation and mitigation strategies is a considerable achievement in its own right.” The full report can be read here (a discussion of the Himalayan glaciers statement begins on page 23).

Coincidentally, the U.S. Geological Survey released its image atlas on “The Glaciers of Asia” only about a week before the IAC’s report, providing evidence that the Himalayan glaciers are, in fact, receding, although there is still conflicting data about what is actually going on there. That atlas can be seen here.

There is no question that the credibility of the IPCC has been damaged and deservedly so. Himalayan glaciers are vital. Even Al Gore noted in his movie that the Himalayan ice sheet feeds seven of the world’s major river systems, providing water to some 40% of the world’s population. Statements about its demise should not be made lightly.

But at the same time, it is important not to lose an appropriate sense of context. Global climate change is incredibly complex to understand and the volume of information considered by the IPCC in issuing its assessments is large. Nations very much need a body to aid in the digestion of the scientific information and to provide carefully considered evaluations of the implications of that scientific information.

Mistakes can and do occur during peer-review processes, and it is very much my hope that those who see something more sinister than a simple error are mistaken. Long-time editor of the Applied Journal of Physics Steven J. Rothman, a man I knew and deeply respected, once provided a highly apt assessment of the system of peer review by paraphrasing Winston Churchill’s quip about democracy as a form of government: “It is the worst … except for all the others that have been tried.”

Fraud and Falsehood Dread Examination But Truth Invites It

My title today is a quotation by the English poet Samuel Johnson, who is perhaps most famous for his 1755 Dictionary of the English Language, which set the standard for modern lexicography. His expression of the role of examination is one that rings loudly for scientists, who have a history and culture of presenting their results openly through publication so that they may be examined by others. This is one of the great strengths of science and it is not unfair to say that it has been responsible for the deep respect that scientists generally enjoy for the integrity of their work and their tremendous advancements in knowledge.

But scientists are human. They are subject to the same kinds of pressures that are put on people in all professions to achieve professional respect and accomplishment. And sometimes the temptations to engage in questionable scientific practices are too great to resist.

Consider one of the most notorious recent cases of scientific misconduct. The German physicist Jan Hendrik Schön was a rising star. At the age of 27, he was granted a Ph.D. from the University of Konstanz and hired by Bell Labs, at which the work for no fewer than seven Nobel prizes in physics has been performed. He published numerous papers in highly prestigious journals reporting astonishing breakthroughs in nanotechnology, notably the ability to produce a molecular-scale transistor. He won the Otto-Klung-Weberbank and Braunschweig Prizes in 2001 and the Outstanding Young Investigator Award presented by the Materials Research Society in 2002.

But when physicists examined his publications, they started to notice anomalies. Identical noise data for experiments carried out at very different temperatures. Schön said it was an accident. Then it was noticed again. And again. And again. Schön said he had destroyed his data because of space limitations on his hard drive and that he kept no laboratory notebooks, difficult rationalizations to accept and undoubtedly poor research practices. Physicists were unable to study his data directly and were unable to reproduce his results independently. Bell Labs launched an investigation and ultimately detailed evidence for 16 examples of scientific misconduct by Schön. His coauthors — prominent scientists who were exonerated of any misconduct themselves — were embarrassed and had their own reputations for scientific integrity called into question.

In the end, most of Schön’s papers were to be retracted by the journals that had published them, with the unanimous agreement of his living coauthors (one coauthor was deceased at the time of the retractions). He left Bell Labs in disgrace. The University of Konstanz revoked his Ph.D.. The Deutsche Forschungsgemeinschaft — the German Research Foundation — withdrew his right to vote in the Foundation’s elections and banned him from serving on its committees. He is not permitted to review proposals for the Foundation nor to apply for any of its funds.

While many like to point to the Schön case as a negative example, it is better seen as a positive example of how the system of scientific review operates to uncover and expose misconduct. The scientific community is rightly concerned about misconduct. Some recent studies have found that about 2% of scientists admit to having fabricated data and as many as 35% have admitted to other questionable research practices, ranging from serial publication of research in multiple journals or abusing anonymity as a peer reviewer to suppressing data inconsistent with their theories or violating articulated ethical standards in research on humans and animals. And there is evidence that the rate of scientific misconduct is rising.

Last week, the Second World Conference on Research Integrity was held in Singapore, in which scientists in different disciplines from nations around the world gathered to develop a global ethics code that it is hoped the scientific societies of individual nations will adopt. This is a positive step.

The final result of the ethical guidelines that the conference proposes is not yet available, but a review of the draft shows that it follows the basic structure of a legal constitution. That is, the document avoids addressing detailed minutiae of specific practices that are considered to be improper, but instead sets forth higher-level concepts. It is accordingly, like most legal constitutions, a short document.

For example, the draft statement on Data Management says only that “Researchers should keep a clear, accurate record of all research in a way that will allow others to verify and replicate the work they have done.” It is simple and to the point. It does not dwell on how records should be kept or the form of the data to be kept, nor the specific ways in which it should be made available to others. These are details that might genuinely differ among different scientific disciplines and in different kinds of research environments, and the simplicity of the statement gives the underlying principle strength.

The entire draft statement can be read here.

Is a global ethics code for scientists a good thing? I believe it is. Another of the strengths of science has been that it is international in character; for the most part, scientists willingly share the results of their research openly and widely without regard for national borders because the state of knowledge will advance more quickly that way. There are, to be sure, circumstances where governments impose secrecy restrictions and modern concerns about securing intellectual-property rights at times interfere with that posture, but a posture of openness is a general part of the culture of science.

The system of review that science uses worked effectively in the Schön case to expose the fraud and to take corrective actions with the research record. But the system can be improved and promulgation of a global code can serve as an important step in that improvement — by providing a clear statement agreed to by scientists all over the world that will educate students and others of practices that are acceptable in conducting research and in publishing the results of that research.

The Missing Link

Eoanthropus dawsoni.

This is one of the most famous hominids ever discovered. But certainly for the wrong reasons.

It was 1912. Very few hominid fossils had so far been found. Neanderthal Man in 1856; Cro Magnon Man in 1869; Java Man in 1890; Peking Man in 1903; Heidelberg Man in 1908. Each of these discoveries had added a small piece to the great puzzle of modern man’s origins, but there was still no clear species that represented a clear link in the evolution between ape and man.

In a way, that changed with the 1912 discovery of the fossil remains of a hominid — identified as eoanthropus dawsoni after the man who discovered them — that clearly showed a mix of features between ape and man. Found in quarries in Sussex, England, the skull was similar to that of modern man, but would hold only a brain about two-thirds the size of a modern brain. And the jawbone was decidedly more apelike. The combination supported the theory that the evolution between ape and man would begin with the brain so that the skull would evolve before the jaw into a form closer to what exists today.

There were skeptics, mostly among French and American paleontologists. But there were also those who saw the discovery as a critically important one, particularly British paleontologists in a pique of nationalist pride. But as time went on and further hominid fossils were discovered, it became increasingly difficult to fit eoanthropus dawsoni into the developing framework of human evolution. Experts perhaps puzzled over it at times, recognizing it as anomalous.

It would be forty years before the discovery of the fossils — made in the quarries of Piltdown — would be discovered to be a fraud. It was an elaborate hoax perpetuated by someone whose identity remains a mystery today, even though there are various theories about who it might have been. Certainly the fossils had been carefully prepared to make them look far older than they actually were, and it was with the use of much more modern dating techniques that the fraud was ultimately exposed. The preparations clearly required knowledge about the techniques that paleontologists would use in analyzing the fossils, making the hoax seem elaborate and almost sinister.

The four-decade episode of Piltdown Man is an instructive one, exposing the limits that may exist with scientific analyses and the ability of some to exploit those limits to mislead.

The more recent episode of “Climategate” is an attempt to suggest that a similar hoax is being perpetuated by some climate scientists today. The episode began in November of last year when a variety of emails and documents were hacked from the University of East Anglia’s Climatic Research Unit computers. Through a rather selective citation of isolated phrases out of context from stolen documents, a scandal was orchestrated, accusing the scientists who were quoted of colluding in a campaign to withhold scientific information, to manipulate data, and to interfere with the peer-review process in order to perpetuate a hoax of increasing global temperatures.

One particular focus of the allegations has been language in an internal email related to the famous “hockey stick” graph that shows sharply increasing global temperatures in modern times. It is a private email written between two scientists and needs to be understood in that context. It undoubtedly refers to a “trick” used to “hide the decline.” But scientists use the word “trick” to describe something clever or insightful to deal with a difficult issue — not as something deceptive. And the decline that is referred to is well-known in dendroclimatology in which the properties of annual growth rings of trees are used to infer temperature changes.

The fact is that since about 1960, tree-ring data has tended to suggest a decline in global temperatures at a time when direct instrumental measurements of temperatures show that it has clearly increased. Before 1960, tree-ring proxy measurements are consistent with other proxies for temperature change at least back to about 1600 AD. Something is obviously amiss with tree-ring proxy measurements, at least after 1960, although the reason for the divergence is not understood. What the Climategate scientists were referring to in their exchange was a known way of dealing with this inconsistency. It is important to recognize that they were not “fooling” anyone — the anomaly with tree-ring data is well-known among climate experts, as is the statistical “trick” used to legitimately reconcile the different types of data.

Yesterday, a British panel exonerated the scientists involved in Climategate, even as it criticized them for some reluctance to release computer files supporting their work. “[W]e find that their rigour and honesty as scientists are not in doubt,” the review states. This is the third review to clear the scientists of allegations of fraud and the vice-chancellor of the university has now expressed his hope that this will “finally lay to rest the conspiracy theories, untruths and misunderstandings that have circulated.”

The history of Piltdown man reminds us that we need to be on guard against scientific misconduct and fraud. But we also need to be on guard against unwarranted allegations of such misconduct when there is no sustainable evidence to show it is there.