Hope - and hype - in the cancer war

Hope - and hype - in the cancer war
A breakthrough test for ovarian cancer fizzled. But the underlying science still holds promise.

Times Leader
Posted on Sun, Aug. 07, 2005

By Marie McCullough
Inquirer Staff Writer

Over Sunday brunch in 1999, Peter J. Levine threw out an idea that would propel the lawyer-turned-entrepreneur to the frontier of molecular research. Three years later, he was the unlikely coauthor of a paper heralded as a medical breakthrough.

Using blood "from a finger stick," researchers had reliably detected ovarian cancer with "a test that can be completed in 30 minutes," one that was "potentially applicable to any type of disease," said the National Cancer Institute, which paid for the February 2002 study.

In Philadelphia, Fox Chase Cancer Center president Robert C. Young recalls thinking: "This is the Holy Grail. If it works, this is a transforming discovery."

The breakthrough was the use of a protein-measurement tool called mass spectrometry, along with highly sophisticated data analysis, to detect cancer in a new way: by uncovering a pattern of subtle changes in blood proteins caused by the disease.

The novel diagnostic approach seemed poised to subdue ovarian cancer, the deadliest gynecological malignancy, the way the Pap smear had subdued cervical cancer, by making it detectable while it was still easily curable.

Early last year, media outlets ranging from the Today Show to Ob.Gyn. News were reporting that LabCorp and Quest Diagnostics, two national laboratory chains, would soon market Levine's test, OvaCheck, to women at high risk of ovarian cancer.

That news was wishful thinking.

Many scientists now say that the original research underpinning OvaCheck was deeply flawed. That the concept of using protein patterns to detect disease makes sense but that there are big technical obstacles. And that if OvaCheck works, they'd like to see the proof.

Levine insists that it works. OvaCheck could already be saving lives, he says, if its development had not been hampered by ruptured partnerships, congressional inquiries into ethical lapses at the National Institutes of Health, and now, unexpected regulatory hurdles.

But others see the rise and stall of OvaCheck as a classic example of how difficult genuine discovery has become in the era of molecular research.

Captivating new technologies that peer deep within human cells often lead to big claims based on limited, preliminary evidence. The technologies are so arcane and straddle so many scientific disciplines that even world-class scientists have trouble judging the quality of the research, or the conclusions.

A napkin launches a company

Levine, now 57, likes to tell how OvaCheck was born six years ago, over tapas at an Alexandria, Va., restaurant.

He and an acquaintance, Emanuel "Chip" Petricoin III, were with their wives, who were close friends.

Petricoin was 34, an up-and-coming microbiologist at the Food and Drug Administration. He had just cofounded a "clinical proteomics" research program at the National Cancer Institute with Lance Liotta, a distinguished NCI pathologist.

Proteomics - a fledgling field exploding with devices, biotech start-ups, and publications - was the complement to genomics. While genomics focused on how genes directed activities within the cell, proteomics studied how proteins carried out those directions - including bad directions. Inside a malignant cell, for example, damaged genes produced defective signaling proteins that then disrupted cell division.

The goal of the NCI/FDA proteomics program, Petricoin explained, was to discover proteins that surged or ebbed in response to cancer so these could be used as biological warning signs. One well-known "biomarker" was the prostate-specific antigen (PSA), a protein shed by the prostate gland into the blood. A high PSA level was already used as a screening test for prostate cancer - but it wasn't very reliable.

New biomarkers were proving to be elusive. Molecular-research tools produced such an avalanche of data that detecting a subtle but significant protein change was like searching for a needle in the proverbial haystack.

Levine, a self-taught computer whiz, was intrigued. Twenty-five years earlier, in the primitive era of punch cards, the brash civil-rights lawyer had pioneered the use of computer data analysis in litigation to prove that the New York City school system was shortchanging minority students.

Now, scribbling on his napkin, Levine suggested looking not for a single protein, but for changes in the overall pattern of blood proteins. Even if the identities of the proteins were unknown, the pattern itself would be the biomarker: "Rather than looking for the needle in the haystack of data... look at the configuration of the haystack."

After that brunch, Levine enlisted another friend, biochemist and computer expert Ben Hitt, to come up with protein-pattern-recognition software.

Levine, Hitt, Petricoin and Liotta tried the software to see whether it could find clusters of proteins that distinguished blood samples of ovarian-cancer patients from samples of healthy women. To their joy, it worked.

A screening test for early ovarian cancer was "urgently needed," they would later write. The malignancy is notoriously hard to detect in the early stages. Levine's own mother was 67 when she died of the disease, one of its 14,000 annual victims.

In 2000, Levine and Hitt cofounded Correlogic Systems Inc. in Bethesda, Md., and applied for a patent on the pattern-recognition software. The Correlogic partners and their government collaborators, Petricoin and Liotta, negotiated a cooperative research and development agreement - a "CRADA" in government parlance.

Both the National Institutes of Health and the FDA were encouraging such public-private cooperation to turn the basic research into beneficial products more quickly.

"I was thrilled," Levine recalled. He sold his international trade brokerage company to devote himself to Correlogic.

Right answers, but wrong

Science-fiction writer Arthur C. Clarke once said: "Any sufficiently advanced technology is indistinguishable from magic."

And, like magic, it can inspire fascination and exaggeration.

About a decade ago, for example, scientists figured out how to transform genetic instructions into an electronic format. Gene profiling using a "microarray" - a chip of glass arrayed with thousands of gene fragments - was expected to revolutionize medicine by decoding the basis of disease.

"All human illness can be studied by microarray analysis, and the ultimate goal of this work is to develop effective treatments or cures for every human disease by 2050," wrote Mark Schena, an inventor of the technology.

Microarrays remain a powerful tool, but skepticism has set in. In a recent article in the Lancet, researchers reanalyzed the seven largest microarray studies on cancer prognosis.

"In five of the seven... this technology performs no better than flipping a coin. The two other studies barely beat horoscopes," John P. Ioannidis, a clinical epidemiologist with Tufts University School of Medicine, wrote in an accompanying editorial.

To understand why, consider the fable about six blind men and an elephant. Each man feels a different part of the animal. One man argues that the creature is a snake, another a spear, another a wall, and so on. A little girl who can see the elephant says, "Each of you is right, but you are all wrong."

Depending on how researchers "feel" their molecular data - using computer analysis to massage, stroke and ignore certain parts - they may discover right answers that are all wrong.

David Ransohoff, a University of North Carolina epidemiologist, says results cannot be trusted unless they can be produced again and again: "Figuring out whether a result is real and not simply caused by chance is determined in part by validation - by reproducing the result in an independent set of samples."

In other words, go feel another elephant.

But even that is not enough, Ransohoff and other experts say. The ultimate validation requires clinical studies in actual patients. A molecular diagnostic method must be as reliable as traditional tools such as imaging tests and surgical biopsy.

So far, proteomics does not have that kind of proof.

The eureka moment

In the spring of 2001, Petricoin, Liotta, Levine, Hitt, and a team of seven others conducted the experiment that would create a sensation.

The researchers used a mass spectrometer to analyze 100 blood samples - half from ovarian-cancer patients, including early-stage patients, and half from women who were healthy or had benign disorders. The spectrometry data were then analyzed to find five proteins - identities unknown - that had different concentrations in the cancer samples from the non-cancer samples.

The discriminatory pattern was validated by using it on blood samples from 116 more women - 50 with cancer and 66 with ovarian cysts or other disorders - to see whether the test would mistake benign disease for cancer.

The software correctly identified all the cancers, even the early ones, and 63 out of the 66 cases of benign disease.

Although the accuracy wasn't perfect, it was a vast improvement over a well-known ovarian-cancer biomarker, CA125, that was too unreliable to use as a screening test.

The Lancet accepted the study and "fast-tracked" the online publication on Feb. 7, 2002.

In letters to the editor, three experts quickly pointed out a statistical miscalculation, but because it did not undermine the overall strength of the results, it did not temper the outpouring of praise.

USA Today hailed "the first tantalizing evidence that a quick, finger-stick blood test might be useful in detecting ovarian cancer before it becomes deadly."

Health, a consumer magazine, named the discovery one of the year's top-10 medical advances.

Congress took the highly unusual step of passing a resolution urging the government to continue funding the research and urging health plans to cover any test that panned out.

The problem with proteins

Media coverage, as well as releases from the NCI and FDA, made finding a protein pattern sound simple and straightforward.

It was not.

Indeed, the complexity of proteins defied detailed analysis until just 15 years ago.

Although the human genome contains about 20,000 genes, each made of only four molecular building blocks linked in a double helix, the human "proteome" contains a million or more proteins, each an intricate tangle of amino acids. These tangles constantly morph and interact in response to the orders they receive from genes. Only several thousand unique proteins have been isolated and characterized so far.

As ubiquitous as proteins are in the body, a tiny tumor may not send many telltale proteins into the blood, where they can be easily sampled. A milliliter of blood serum from a healthy middle-age man, for example, contains one-billionth of a gram of PSA, the protein shed by the prostate gland - and proteomics seeks to measure far lower concentrations.

Proteomic analysis became feasible in 1990, when two scientists (who subsequently won Nobel prizes) figured out how to separate big proteins and make them fly through a spectrometer - "give wings to molecular elephants," as one inventor put it.

Still, this analysis is tremendously manipulative, indirect, and often ambiguous.

One problem is that trace proteins - the potential biomarkers - may be swamped by other proteins, despite techniques to concentrate the scarcest ones on the special chip that goes into the mass spectrometer.

Another problem is that the spectrometer's measurements - made after vaporizing the proteins and giving them a positive charge - are least reliable in the low range where biomarkers are presumed to exist.

Finally, the spectrometry results can be thrown off by countless variables, including machine miscalibration and handling of blood samples.

All of which makes results difficult to reproduce, even in the same lab using the same blood samples.

Under attack

After the Lancet study, Petricoin and Liotta - as well as many other research teams - worked to repeat and improve the proteomics feat.

By mid-2003, Petricoin's team had conducted four ovarian-cancer experiments, including two in which they claimed perfect accuracy - every sample correctly classified.

Among those who published challenges was Keith Baggerly, a biostatistics and proteomics expert at the M.D. Anderson Cancer Center in Houston.

Not that Baggerly doubted the promise of proteomics; he just didn't think that the studies Petricoin led were well-designed.

Baggerly's lab reanalyzed the data sets from the ovarian-cancer experiments and could not reproduce the patterns. Nor could the diagnostic pattern from one data set be used to distinguish cancers in another data set.

Further, each diagnostic pattern discovered by Petricoin's group included proteins detected below the range that the spectrometer had been calibrated to measure.

And in at least two experiments, Petricoin's team put all the cancer samples through the spectrometer, followed later by all the cancer-free samples. A well-designed study would have processed them randomly. That way, any accidental distortion from the spectrometer would be sprinkled throughout the samples, instead of becoming a false pattern.

"If a test for ovarian cancer ever does come about from these data, I'd need to see a lot more studies before I'd send my mother out to get it," Baggerly declared.

Petricoin and Liotta, in journal rebuttals, denounced their critics as "misguided," "unfortunate," "judgmentally biased and scientifically unfounded," and revealing a "possible lack of understanding" of mass spectrometry.

As the debate raged, the scientific community, initially enthralled by diagnostic proteomics, was coming to appreciate how daunting the technology really was.

Conflicts and controversy

While Petricoin and Liotta were defending their work, a rift was growing between them and their Correlogic collaborators - and Levine couldn't understand why.

For a small biotech start-up, Correlogic had come very far, very fast. Levine had recruited top-notch scientists in key disciplines, wooed investment capital, and licensed Quest and LabCorp to market OvaCheck.

Yet the partnership with the government that was supposed to be helping Correlogic was doing just the opposite.

NCI and FDA officials, led by Liotta and Petricoin, kept revising the contract, minimizing Correlogic's role. Correlogic went from being an integral part of plans for human clinical studies to being on the margins, then to being excluded entirely.

Eventually, the government's Web site said that OvaCheck was "being independently developed by Correlogic," and was "unrelated to previous published work with the NCI and FDA," and that the NCI/FDA team was developing its own ovarian cancer proteomics test.

"There was always something else," Levine later recalled. "The goalposts kept shifting, but I couldn't find out what was really going on."

In July 2003, Levine received a series of e-mail tips that pointed to a possible answer.

Petricoin and Liotta, it turned out, had gotten permission from their agencies to work as paid consultants to an even newer firm, Biospect, that was also developing diagnostic protein patterns. The company was cofounded by Richard Klausner, the previous NCI director - and Liotta's former boss.

Last year, the consulting deal came to the attention of a congressional subcommittee investigating ethics policies at the National Institutes of Health, which oversees the NCI. The case was highlighted during the May and June hearings, which ultimately prompted the NIH to ban all biotech and drug-industry consulting by NIH scientists.

Petricoin and Liotta steadfastly denied any conflict of interest, though they quit consulting for Biospect shortly before the hearings.

Lawmakers seemed unconvinced.

U.S. Rep. Joe Barton (R., Texas) declared, "As a result of these secret deals, progress may have been slowed on the public-private partnership that could have led to prompt commercialization of a lifesaving ovarian-cancer diagnostic test."

The battling and hand-wringing over a test that didn't yet exist gave the impression that it was on the horizon - even as many proteomics experts were adopting a slow-down, be-more-careful attitude toward the technology.

And there was a new wrinkle for Correlogic: The FDA sent a letter to Levine saying that OvaCheck was a new medical device - not a lab-based test, as Levine contended. By law, that meant OvaCheck could not be marketed without successful clinical testing in patients - the ultimate proof of safety and effectiveness.

"If OvaCheck works, it could save many women's lives," the journal Nature said in an editorial in June 2004. "But the risks of an imperfect diagnostic test are not slight... . Before OvaCheck hits the market, Correlogic and its licensees need to publish evidence from a large clinical trial."

Promise remains

For all the controversy over ovarian-cancer screening and proteomics, the promise of the field still tantalizes.

Several hundred diagnostic proteomics studies have been published since the seminal 2002 Lancet paper. Many companies - among them Biospect, now called Predicant Biosciences - are racing to develop cancer-detection tests. And new proteomics programs are proliferating across the country; Petricoin and Liotta left the government this spring to set one up at George Mason University in Manassas, Va.

Levine, too, remains optimistic.

"I'm very confident we'll get one or more of these tests to market," Levine said, sitting in his small, neat Correlogic office in Maryland. "I'm in this for the long haul, win, lose or draw. I've always been an idealist. Through a strange coincidence, I found myself in a position to create a company like this, and surround myself with brilliant people, and develop technology that could save lives."

But there is also much talk about learning lessons from the controversy. One clear lesson, according to experts such as Ransohoff, the North Carolina epidemiologist, is that proteomics research must learn to walk before it runs. The field must develop standards, show reproducibility, follow rules to avoid error, and report enough detail for reviewers to evaluate quality.

"Proteomics is hugely exciting and has lots of merit," said John Semmes, a molecular biologist at Eastern Virginia Medical School who is working with others to develop standards for proteomics research. "The problem is that the hype around that particular [Lancet] study and the subsequent failure to produce that test has cast a bad shadow over the rest of the field.

"You'd think you could turn to the NCI, the NIH, and their announcements would be solid-gold, and they'd be conservative. But that's not true."

Conservatism seems to be the new watchword.

The NCI/FDA proteomics program - which now says on its Web site that it "is not currently, and has not previously," worked to develop a commercial test for ovarian cancer - is retrenching. The program is currently "working to refine, improve, and optimize its techniques."

Those same goals are part of a new $104 million NCI "clinical proteomics technologies initiative" approved in June by the NCI's Board of Scientific Advisors, led by Fox Chase Cancer Center's Robert Young. The board initially rejected the plan because it glossed over the technical obstacles to proteomics.

Young does not fault NCI officials, or anyone, for past statements that made the technology sound like magic.

"In retrospect, should we have been more conservative? Yes." he said. "But most of us were captivated by this."

Source[/color]