Sloppy Science? System Failure? Or...?

This week, something on the order of perhaps 15,000 to 20,000 of the world’s top cancer researchers, from PhD students all the way to Nobel laureates, are gathering in Chicago for the Annual Meeting of the American Association for Cancer Research  (AACR). Although I am not attending this year, if the current program resembles those of years gone by, you can be sure that there will be hundreds, if not thousands, of scientific presentations that are describing potential new targets for cancer therapy. Some of these will be in huge plenary sessions, some will be oral talks in mini-symposia and many will be presented in concurrent poster sessions.

Why? As we learn more and more about what makes cancer cells tick, we are discovering more and more pathways that are implicated in the origin or the continuation of the cancer “state”. Every molecule that gets identified as being part of one of these pathways is, at least potentially, a target for intervention – either to turn it off (if it is implicated in the initiation or progression of cancers), or to turn it on (if it is implicated in some “protective” mechanisms against cancers), and so on.

And every one of these studies could be important in its own right since each one adds to our burgeoning understanding of the molecular basis of cancers.

And some of these might turn out to be even more important if the so called “target” can be validated as really being involved in cancer causation (as opposed to an incidental bystander). 

But the real “jackpot” comes when one of these targets is not only validated as being important and centrally involved in one cancer or another, but is also considered to be a so-called “druggable” target. By that, we mean that we expect to be able to discover or develop a drug, usually a small molecule or an antibody, that can then interfere with, or in some other way modulate, the cancer state and thus be an effective anti-cancer therapeutic.

The success of this model has been evidenced by drugs like Imatinib (Gleevec), a small molecule drug, or Trastuzumab (Herceptin), a monoclonal antibody. The search for the next “druggable targets” and the subsequent discovery of the next Gleevec or the next Herceptin continues to drive preclinical laboratory-based research the world over, since those avenues are where many of the next cancer therapeutics are expected to originate. 

Undoubtedly, only a small number of these putative targets will actually traverse that magic line from being a preclinical “observation” to actually being of demonstrated clinical utility. This is the realm of so-called “translational research” - to translate, or move forward, research from the lab so it can end up in the clinic for the care and treatment of real patients in the real world.

That path is often a long and arduous one, mind you, fraught with frustration, but every long journey starts with a single step, as they say. Still, there will be palpable excitement as more and more of these potential targets are described, understood and tested for clinical utility.

Last week, however, a bit of a bucket of cold water was thrown on a number of highly touted studies that presumably had shown great promise of such translation into the clinic.

In a Commentary entitled “Drug development: Raise standards for preclinical cancer research” published in the respected journal, Nature on March 29, 2012 authors C. Glenn Begley and Lee M. Ellis reported that, sadly, not only have the vast majority of such studies have NOT resulted in translation into the clinic, but worse, they said, reputable scientists working at pharmaceutical or biotech companies have not been able to replicate most of the results that had been lauded at one time as potential “breakthroughs” (italics mine). 

 

In total, they reported that at least 47 out of 53 publications – all from reputable researchers and published in reputable peer-reviewed scientific journals -  had not been able to be replicated during the time the one of the authors (Dr. Begley) had been the head of research at the biotech company Amgen. 

 

This rather shocking finding prompted the authors to make some specific recommendations to try to ensure that this situation did not persist.

 

And it prompted an Editorial in the same issue of Nature, entitled “Must Try Harder” that opined that “too many sloppy mistakes are creeping into scientific papers. Lab heads must look more rigorously at the data — and at themselves”.   

 

The Editorial went on to say:

[This] "Comment article ... exposes one possible impact of such carelessness. Glenn Begley and Lee Ellis analyse the low number of cancer-research studies that have been converted into clinical success, and conclude that a major factor is the overall poor quality of published preclinical data. A warning sign, they say, should be the “shocking” number of research papers in the field for which the main findings could not be reproduced. To be clear, this is not fraud — and there can be legitimate technical reasons why basic research findings do not stand up in clinical work. But the overall impression the article leaves is of insufficient thoroughness in the way that too many researchers present their data."

Please do note that, as the editorial says, no one is suspecting, suggesting nor accusing any fraudulent behaviour. And indeed there are many potential legitimate explanations why not all results can be reproduced. But the publication of this Commentary and the accompanying Editorial have certainly ignited a firestorm of subsequent comments, newspaper articles, blog posts and Twitter activity.

I found one online response to the Nature Editorial to be particularly telling, especially since it came from a friend and colleague whose opinions I respect immensely. Dr. Jim Woodgett, Director of Research at Toronto’s famed Samuel Lunenfeld Research Institute at Mount Sinai Hospital wrote:

"The issue with inaccuracies in scientific publication seems not to be major fraud (which should be correctable) but a level of irresponsibility. When we publish our studies in mouse models, we are encouraged to extrapolate to human relevance. This is almost a requirement of some funding agencies and certainly a pressure from the press in reporting research progress. When will this enter the clinic? The problem is an obvious one. If the scientific (most notably, biomedical community) does not take ownership of the problem, then we will be held to account. If we break the "contract" with the funders (a.k.a. tax payers), we will lose not only credibility but also funding. There is no easy solution. Penalties are difficult to enforce due to the very nature of research uncertainties. But peer pressure is surely a powerful tool. We know other scientists with poor reputations (largely because their mistakes are cumulative) but we don't challenge them. Until we realize that doing nothing makes us complicit in the poor behaviour of others, the situation will only get worse. Moreover, this is also a strong justification for fundamental research since many of the basic principles upon which our assumptions are based are incomplete, erroneous or have missing data. Building only on solid foundations was a principle understood by the ancient Greeks and Egyptians yet we are building castles on the equivalent of swampland. No wonder clinical translation fails so often."

As someone who ran the research operations of two major Canadian national cancer research funding agencies over the past two decades, I wonder if my own organizations have inadvertently been “complicit” in this. We always tried our very best not to “over-hype” any results from investigators we funded, but there is always a need, especially in a national health charity, to “excite” the public and the prospective donor, and to be accountable to previous donors by showcasing for them any success their generosity has won.

Perhaps we all need to take a closer look at the pressures we place on researchers globally to “publish or perish”. Are our incentives and the way we measure “success” all wrong? 

Perhaps, indeed, it is long overdue that we take a very hard look at how we conceive, fund, undertake, promote and analyse cancer research results, and how and what we value in cancer research and in cancer researchers.

"Personal" Genomics...Part 2

Rather than amend or append my post on "personal genomics" from earlier today, I thought I would provide a new one that expands on the "personal" genomics thread.

But this time, it is not about privacy, confidentiality or other kinds of ethical concerns, per se. Instead it goes right to the heart of the matter of "just how useful will this information be to the average person", average meaning a person at 'average' or 'slightly above average' risk for a disease.

Some details of a very interesting study were released today at the Annual Meeting of the American Association for Cancer Research (AACR) in Chicago (meeting website here), and published simultaneously in the journal Science Translational Medicine. Authored by a team at Johns Hopkins in Baltimore and led by world-famous researcher Dr. Bert Vogelstein, the study, entitled The Predictive Capacity of Personal Genome Sequencing suggests that the information that might be provided could actually be of limited usefulness.

Rather than me trying to paraphrase the whole thing, I refer you to an excellent article by Gina Kolata in the New York Times today (Study Says DNA’s Power to Predict Illness Is Limited) that offers an excellent recap.

For a second viewpoint, but same conclusions, you can check out this column from  Robert Bazell entitled "Gene tests: Your DNA blueprint may disappoint, scientists say" online also today at msnbc.com

Bottom line, it seems to me, is that we really have to be more careful than ever about exposing ourselves to privacy, confidentiality, insurabililty and other legal and ethical dilemmas, especially if the risks might outweigh the gains in many, if not most, cases.

Clearly there is no definitive pronouncement to be made one way or the other yet - it is far too early days for that. But it is good to have these debates with our eyes wide open.

Sure beats the alternative...

More About "Personal" Genomics

No, I didn't make a mistake in the title. I know we talk about genomics and personalized medicine a lot, but in this case I really was referring to "personal genomics". We need more conversations, discussions and debates about how we as a society are going to deal with the inundation of personal gene and DNA information that is coming our way in the brave new world of $1000 genomes.

I have already written some on the subject in a previous post about the promise and the concerns.

I have also provided some links last week to some further information on the subject.

This week saw (at least)  two new entries into this important discussion that I thought were worthy of passing on.

The first of these was a wonderful show on the series "Nova" from WGBH Boston called "Cracking Your Genetic Code".  I thought it was a particularly good treatment of the whole science behind genomics and personalized medicine. If you missed it, or can't get it, for now the show is available online at http://www.pbs.org/wgbh/nova/body/cracking-your-genetic-code.html.

I really encourage you to invest the hour watching this show - it is excellent in my view.

The other is a particularly good article written by Christine S. Moyer of the (American Medical Association) Amednews.com staff. In this piece she writes more about the issues of what are we going to do with all of the information that we might be gleaning very soon. Entitled "Genome sequencing to add new twist to doctor-patient talks", it is written from a health care, ethical and societal point of view - exactly the debate that I have been insisting that we have not been having  nearly enough of.

I am very glad to see that these discussions are expanding and that the issues are starting to come out on the table. We owe it to ourselves to become much better informed and to steer this brave new world or else it will swamp us!

Society, Ethics and Cancer Genomics

In one of my earlier posts (you can read it here), I talked about some of the major societal issues that face us in ethics, health services and health policy arenas (to name but a few), now that we are about to turn the corner on the $1000 genome.

I know that I am not the only one (not by a long shot), who is concerned about these important issues) but it was gratifying today to learn that the US Presidential Commission for the Study of Bioethical Issues is taking up the charge. I will be following this to see what happens....

See a report from Bloomberg News.

Welcome to the World of Genomics... [Part Three]

In the last post [World of Genomics Part 2] I tried to give you a flavor of just how far and how fast the science of genomics has progressed in the last decade. We are truly getting to a point where an individual human DNA sequence will be available for $1000 or less.

There is also increasing prospect of what is called “direct to consumer" products where you will be able to order your genome from a company on the Internet without any intervention by your physician. In fact, I suspect this is already happening in limited ways but will pick up steam very soon as the affordability continues to drop.

As much as the technology will afford us an unprecedented technological advance into our understanding of human diseases such as cancer, it is my belief that this era of “cheap” genomes is also ushering in some unprecedented questions of ethics and law that we are not yet facing head on, and need to start debating and discussing as a society ASAP.

Top of the list perhaps are issues of privacy and confidentiality. 

Where is your genome sequence going to be stored? I could imagine an app on your smart phone in the not so distant future! Do we really want to have our genomes floating around in cyberspace? Do we trust some central database (e.g. a government database) to house this information? I will wager that many of you already feel uncomfortable about the fact that the CRA in Canada or the IRS in the United States holds so much informational power over you by having your detailed tax records and related files in their databases. I cannot imagine a piece of information more personal or more confidential than my own detailed DNA sequence; will I really trust that it will be kept secure on the Internet or in someone's file cabinet?

The privacy and confidentiality issues lead us then to questions such as insurability. Suppose an individual is carrying seven particular mutations that might, and I stress might, predispose them to a particular disease. And suppose that information is now made available to an insurance company, and as a result life insurance or mortgage insurance or some other form of insurance is denied because the risk is deemed to be unacceptably high? What happens then?

What, in fact, does “predisposition to disease" really mean anyway? In the vast majority of cases this is not a clarion signal that the disease will develop. It merely says that you MIGHT need to take different levels of precaution than your neighbour in order to prevent or avoid that disease from occurring in the first place.

And if your doctor is able to determine from your mutational status that you have a predisposition to some particular disease, what about your “need to know” vs. your “right to know”? In some jurisdictions, such as France, the obligation of a physician to disclose this information is enshrined in law, as I understand it. In the US and in Canada there are no such regulations yet. Who is going to make the decision about when your health care professional should, or must, advise you of your mutational status, especially if it doesn't actually mean anything finite in the immediate sense of the word? If there is nothing you can do, then how important is it for you to know? Is it your right to know?

And even if it is your right to know, is it possible that we will end up creating so much anxiety and stress in individuals who learn of a particular mutational status that we will in effect “stress” them into the very diseases we are trying to prevent? The notion of creating so many self-fulfilling prophecies is very real in my view .

And then there are issues of economics and policy. The better able we are to define specific sets of mutations and to tailor treatments to those sets of mutations, it could be imagined that we will need more and more targeted drugs. While targeting and specificity are a good thing, most of these drugs are not cheap! One could rightly ask why would we be developing more and more expensive drugs when we can't even afford the ones that we have now.... 

And how will decisions be made about who has access to which drugs? We already see significant differences in Canada from province to province about cancer drugs that are paid for by the public health system in one province but are not available to patients in a neighboring province.

And from a policy maker's point of view, it could be fair to ask "how much is X months of someone's life worth?" If an expensive drug can prolong a cancer patient's life by six months, for example, who makes the decision about "at what cost"? I can easily see that if the patient in question is your mother, or your son, or your sister etc. then you might justifiably argue that ANY cost is worth it - you are prolonging the life of a loved one.

But if you are the Minister of Health and you have to look at this in terms of benefit vs. cost to society at large, you no doubt will need to look at this more objectively and dispassionately.

The answers to these kinds of questions will come from new kinds of cancer research, but it won’t be in the usual laboratory settings. Instead, we need to accelerate our efforts into research in:

  

•  health economics of cancer
• how will money be best spent?
• health services research
• how will services be best organized?)
• health policy research
• how will information be provided to policy makers for best use?)
• ethics research
• how will resources and access to service be maintained in the fairest way for all patients? 
• how will we protect vital personal and confidential information? 
• who owns the data? 
• who defines a patient's “need to know” vs. "right to know”?)  

The point is, that we are at a stage in the development of very powerful technologies that are going to create opportunities but also some very fundamental ethical issues that I do not believe we are ready to deal with at the societal level.

There have been a few other technological “tsunamis” that have broken on society and changed our world irrevocably in the past. One of these was of course the advent of nuclear technology and all of the good and ill that brought with it. 

Another was the development of recombinant DNA technologies that brought with it the modern era of molecular biology, of which these genome science opportunities are the latest wave. I am not old enough to know what sort of public consultations, if any, accompanied our ushering in of the nuclear era, but I do very much remember some of the public debates that happened in the early 70’s around the advent of the new molecular biology.

Without in any way suggesting that the outcomes of those debates and consultations were appropriate or not, at least an attempt was made to engage the public and to inform people of what was coming, and to attempt to assess it from both a benefit and risk perspective. I don't see the same level of engagement happening now with the new genome technologies and I think it is overdue. 

These issues are too important and the ramifications are too far-reaching to not have these debates and discussions right now…