Data and Safety Monitoring Board Recommends Discontinuation of Canvaxin Phase 3 Clinical Trial for Patients with Stage III Melanoma
What can investors and others learn from these repeated failings of Phase 3 clinical trials of cancer vaccines?
If there is only one lesson to take away, it is this. Phase 2 clinical trial designs and endpoints with cancer vaccines have thus far not been adequate predictors of vaccine efficacy. So let’s have a look at the CancerVax Phase 2 study design and endpoints to see why. I’ll tell you up front that I have little prior knowledge of this company’s Phase 2 program. I’m going to read about it and comment on it in “real-time” as I write this.
First, I’ll look for the data. CancerVax’s website has had a link to a summary of the program. It says said that some of the data have been published. Usually, however, salient details have been released to the press. I check my computer for this using my desktop search (having tried several, I’ve settled on X1 for now). As I’ve been keeping detailed press-release archives since the beginning of 2004, I’m not finding anything on the Phase 2 program there. I’ll check Medline for the publication(s). Note that if I weren’t aware of the publications, I probably would first do a search on ASCO’s meeting abstract database. I find three publications using PubMed that appear promising. All three were published in the second half of 2002, two in JCO and 1 in Ann Surg. Let’s have a look at the paper by Hsueh et al in the JCO.
The study design is found under “Patients and Methods”. Always read this section first when interpreting a study finding. The study population was culled from a database of over over 10,000 patients with AJCC Stage IV melanoma who were clinically disease-free after surgical resection. Now, I don’t know exactly what Stage IV melanoma is, but I can surmise that it is either the most advanced stage or close to it. Typically Stage IV solid cancers have spread to distant sites from the original tumor location. “One hundred fifty patients identified during the database search were enrolled onto phase II protocols for postoperative adjuvant therapy with Canvaxin therapeutic cancer vaccine.” Hmmm…note that, for the first time, we learn that this “study” is actually an agglomeration of Phase 2 studies and maybe something else too. “The patients had no clinical or radiographic evidence of disease before vaccine initiation.” So, the studies were designed to prevent a disease recurrence after a clinical remission period that varied from patient to patient (the amount of variance in this factor is likely important, but we’re not told yet what it is. We’re only told that it must have been at least 30 days since the last prior treatment). “One hundred thirteen patients identified during the database search did not receive Canvaxin vaccine after complete resection of distant melanoma metastases or at any point during their treatment at our cancer institute.” So, in this “study” 150 patients retrospectively identified as having been treated with Canvaxin were compared with 113 patients retrospectively identified who were not.
This is your first RED FLAG. Stop here and think about this a bit. Retrospective Phase 2 studies were conducted, but they were small, too small probably to convince anyone that Canvaxin would be effective in Phase 3. The authors of this paper surmised that if they could combine data from subjects treated with Canvaxin, they might be better able to provide convincing evidence of efficacy. One has to wonder whether they would have published this study had they not. But let’s move on. So the study is essentially a retrospective, uncontrolled treatment comparison. Remmebr, the untreated population has been taken from a cancer treatment database that has nothing to do with Canvaxin Phase 2 studies. The fact that the treated population was “nested” in a Phase 2 program is a red herring. It might as well not have been. This form of evidence is superior only to a case report and a case series in its accuracy for predicting treatment efficacy in a larger population. There are simply too many biases that can influence the findings. We might surmise, for example, that patients who volunteer for a secondary prevention trial with a vaccine might be more diligent about their overall health care and lifestyle, factors that could influence their overall immunity and health. We could stop here, concluding that this study is not worth our effort, but let’s take a look a bit further.
“Both groups had a preponderance of male patients and patients with only one tumor-involved anatomic site. Thirty-one nonvaccine patients (27%) did not receive any adjuvant treatment. Fifty-four patients (48%) received bacille Calmette-Guerin (BCG) by means of various administering modalities, 33 (29%) received chemotherapy, and 16 (14%) received radiation therapy postoperatively. There was a higher proportion of vaccine than nonvaccine patients with more than one tumor-involved organ site (29% v 12%).” Because this is an uncontrolled treatment comparison, we should look carefully at the measurable characteristics of subjects. There was a preponderance of male subjects and one involved site, so this alerts us to the possibility that any findings might not be transferable to a population with more women or in those with multiple anatomic sites of tumor. Furthermore, more of the vaccine-treated subjects had more than one tumor-involved site. This could suggest that those who received the vaccine were more ill and thus more likely to die during the study period, but perhaps not. For instance, suppose that having disease at more than one anatomic site actually improves the odds that a vaccine will be effective. We just don’t know.
“Each patient in the vaccine group was enrolled onto one of five prospective phase II vaccine protocols that differed only in the type of immunoadjuvant (cyclophosphamide, indomethacin, sargramostim [Leukine; Immunex, Seattle, WA], cimetidine, or ranitidine).” So, now we know that the 150 Canvaxin-treated subjects were enrolled in one of five studies, an average of only 30 subjects per study. Would you bank on a single 30-subject Phase 2 program before beginning a lengthy and expensive Phase 3 program? Guess what, if you invested in CancerVax during Phase 3 that is exactly what you did. Combining 5 or 100 studies, each with inadequate power to determine the true efficacy of a drug on a predictive endpoint (that is, an accepted surrogate of efficacy) is no more useful that conducting a single under-powered study and forging ahead. It’s just smoke and mirrors.
“Assessment of Immunologic Response
Immediately before initiation of vaccine therapy and at each scheduled therapeutic dose of vaccine, the DTH response to the vaccine was determined by injecting 2.4 x 106 vaccine cells (one tenth of the therapeutic dose) at a separate intradermal site. DTH response to vaccine was determined 48 hours later. Control DTH response to nonmelanoma antigens was monitored by administering a PPD skin test to PPD-negative patients at monthly intervals until the patient tested positive.” Immunologic response should, in theory, correlate with survivall. However, this has yet to have been shown in a definitive efficacy study of a cancer vaccine in the treatment setting. Therefore, it is an exploratory biomarker. One would hope that if the vaccine were effective, it’s efficacy could be predicted by an early immunological response to better guide treatment.
“Overall survival (OS) for vaccine patients was the interval between initiation of vaccine therapy and death; OS for nonvaccine patients was the interval between surgery and death. Multivariate analysis was performed by Cox proportional hazards regression.” This was the only way to compare the OS using the lifetable analysis (Kaplan Meier). It doesn’t make it a valid comparison. The authors are using different definitions of the time until death (the survival time). The ad hoc adjustment they have chosen makes logical sense given the nature of the study, but it again highlights the difficulty on relying on this study design for meaningful results.
“To control for inadvertent selection bias, a computer program was written for matched-pair analysis on the basis of sex, site of initial distant metastases (soft tissue/distant lymph nodes, liver/brain/bone, or lung/gastrointestinal/other), and number of tumor-involved anatomic sites (one or two). In addition, each pair was matched so that the nonvaccine patient’s OS exceeded the interval between complete resection and initiation of vaccine therapy in the matched vaccine patient.” This is our second RED FLAG. Why? The study is really an uncontrolled retrospective analysis. However, the authors have attempted to turn shit into shinola by creating an untreated “control” for each treated “case,” lending the appearance of a genuine nested case-control study. It’s not. I’ve highlighted the most worrisome part of the method used to generate matched pairs of patients. Logically, it makes sense to select untreated “controls” on this basis. But matching on the primary endpoint, for whatever the reason, creates an insurmountable bias. It may be argued that this bias should favor the untreated group. However, that is not necessarily the case. Again, there is no way of knowing for sure.
“Vaccine and nonvaccine patients in each pair were compared with respect to OS. If both patients died, a survival benefit was attributed to the treatment received by the patient with the longer survival. If one patient died, a survival benefit was attributed to the treatment received by the surviving patient only if that patient survived longer than the patient who died; otherwise the survival benefit of treatment was considered indeterminate. If both patients were alive at the end of the study, the survival benefit of treatment also was considered indeterminate. The log-rank score test was used to estimate the treatment effect on the basis of the number of pairs favoring vaccine versus nonvaccine therapy. A value of P a priori -evidence of efficacy based on Phase 2 data for a therapy in or about to enter Phase 3. This doesn’t mean that we can predict the outcome of Phase 3, but we can know whether Phase 3 is more or less likely to fail an efficacy hurdle compared with an average cohort of studies. We know that for Phase 3 studies in solid cancers, for example, that the average industry success rate for drugs entering into Phase 3 and later being filed for regulatory approval is around 50%, perhaps slightly higher. Given our low confidence in the Phase 2 results presented here, and the failure of other cancer vaccines to prove their worth in Phase 3, we should assume that the “prior probability-adjusted” Phase 3 success outome is substantially less than 50%.
