Non-inferiority clinical trials: What they are and when they should be done

Null hypothesis in a superiority RCT

In the analysis of a clinical trial, rather than prove the hypothesis being tested, the statistician sets out to disprove the ‘null’ hypothesis. If such a null hypothesis can be statistically rejected, the alternative hypothesis stands. The null hypothesis in a superiority trial would be that the new treatment is ‘not superior’ to the control treatment. If this is disproved, the alternate hypothesis, that the new treatment is ‘superior’ to control, stands. To disprove the null hypothesis in a superiority trial where the outcome of interest is death, occurrence of disease, or a serious condition, the RR should be less than 1, and the upper limit of the 95% CI should also be within 1, where 1 represents a neutral effect. Further, the p value would be expected to be <0.05, indicating a less than 5% chance that the result is due to chance.

In the example from the SPAF trial quoted above, all these criteria are met, and the null hypothesis could be rejected and the alternate hypothesis of superiority established. Thus, in SPAF, warfarin was demonstrated to be superior to placebo in preventing stroke in NVAF. The results of the SPAF trial led to warfarin (or other vitamin K antagonists) becoming the standard of care in stroke prevention in AF. The clopidogrel and aspirin versus aspirin alone for the prevention of atherothrombotic events (CHARISMA) trial⁴ offers a contrast. CHARISMA was also a superiority trial testing if a combination of aspirin and clopidogrel is superior to aspirin alone in stable cardiovascular disease (CVD). In CHARISMA, the RR was 0.93 (95% CI 0.83–1.05, p=0.22). In this case, while the point estimate (RR) was less than 1, the upper limit for the 95% CI 1.05 was greater than 1, and the p value did not achieve a level of significance. Thus, in CHARISMA, the null hypothesis could not be rejected. Clopidogrel plus aspirin was not superior to aspirin alone in this population.

Where would one consider using an NI design in an RCT?

Consider a situation where a proven and effective medication exists for a potentially severe disease, and is the standard of care. A new molecule becomes available, which potentially has advantages over the existing medication. It may be safer, more convenient to administer, have better pharmacodynamics, and/or cost less. Because of these advantages, the new molecule may be useful even if it is as effective or almost as effective as the standard medication.

How would one design an RCT for the new molecule? Since an effective drug is available, testing against a placebo would be unethical. The new molecule would need to be compared to the existing standard therapy. With a superiority design, if the null hypothesis (Box 1) could not be rejected, the design would not permit statements of it being ‘as good’ or ‘almost as good’. Regulatory authorities would be unlikely to license the new molecule, and the potential benefits would be lost. This is a situation for the use of an NI design.

A concrete example illustrates this well. Warfarin and vitamin K antagonists (VKAs) are effective in preventing stroke and SE in AF and have been the standard of care until recently. However, their unstable pharmacokinetics/pharmacodynamcs (PK/PD) result in significant inter and intra-individual dose variability, necessitating frequent INR moni-toring, dietary restrictions, and carry a risk of intracerebral haemorrhage (ICH). NOACS offered some advantages over warfarin—more predictable absorption and PK/PD and blood levels making INR monitoring unnecessary, use of a fixed dose regimen, and possibly less ICH. It seemed that with these potential advantages, even if their efficacy was not superior to warfarin but was as good or even almost as good in preventing stroke/SE in AF, they would be useful. Hence, NI designs were used in an RCT comparing NOACS to warfarin.

Since these molecules were initially focused on more developed countries, testing was mainly done in NVAF. This is the case in both the RELY trial (dabigatran versus warfarin in patients with AF)⁵ and ARISTOTLE trial (apixaban versus warfarin in patients with AF).⁶ In contrast, the AVERROES (apixaban in patients with AF) trial was a superiority trial.⁷ This was done because in AVERROES, apixaban was compared to a placebo in patients with NVAF considered unsuitable for VKA.

How would NI be defined?

When designing an NI trial, an NI margin must be defined in advance. An NI margin is a predetermined margin of difference between the new and standard treatments, which represents how much loss of efficacy one is prepared to accept in the new treatment compared to the control treatment, keeping in mind the potential advantages of the new treatment. To arrive at such a margin, one should first consider the benefit of the existing active control treatment against placebo or other control medication when it was originally tested. The NI margin should preserve a major part of this benefit. Thus, one would have to review the data from RCTs that established the efficacy of the control medicine over placebo, either from a large definitive trial or a meta-analysis that led to the regulatory approval of the control drug. The upper margin of the 95% CI of the control drug against placebo is identified and is termed M1. For setting the NI margin, M2, we should preserve much of this benefit. Thus, for designing an NI trial of a new anticoagulant against warfarin in AF, M1 would be derived from the results of the SPAF trial referred to above in which warfarin was tested against placebo. The results of the SPAF trials showed that warfarin would result in a 33% point-estimate reduction of events compared to placebo, with a minimum of 15% reduction (RR 0.67; 95% CI 0.27–0.85, p=0.01). The NI margin should be set to preserve this 15% benefit. For NOAC trials, generally, the NI margin is set as less than 1.46. That is, any result in which the upper confidence limit is within 1.46 will be accepted as non-inferior. A point midway between the 95% CI of the RR reduction was taken as explained in Box 2.

Consider the Rivaroxaban Once Daily Oral Direct Factor Xa Inhibition Compared with Vitamin K Antagonism for Prevention of Stroke and Embolism Trial in Atrial Fibrillation (ROCKET AF) trial.⁸ It’s primary objective was to compare the effect of rivaroxaban 20 mg once daily with dose-adjusted warfarin in the prevention of stroke and SE in patients with NVAF. An NI design was used. As described above, the NI margin (M2) was set at 1.46. Thus, to reject the null hypothesis that rivaroxaban was inferior to warfarin, it was required that the upper limit of the 95% CI should be less than 1.46. A sequential analysis for superiority was planned if NI was achieved. For the latter to be successful, the upper limit of the 95% CI would have to be less than 1. In ROCKET AF, there were 269 primary events in the 7081 patients randomised to rivaroxaban (2.1 events per 100 patient years) and 306 such events in 7090 patients in the warfarin arm (2.4 events per 100 patient years; HR 0.88; 95% CI 0.75–1.03). Since the upper margin of the CI (1.03) is less than 1.46, the null hypothesis of inferiority was rejected, and it was concluded that rivaroxaban was non-inferior to warfarin in this population (p for NI <0.001). However, since this margin exceeded 1.0, sequential analysis could not reject the null hypothesis that rivaroxaban was not superior to warfarin (p for superiority 0.12). The trial concluded that ‘In patients with AF, rivaroxaban was noninferior to warfarin for the prevention of stroke or SE’. Since the trial also showed reduced intracranial and fatal bleeding in the rivaroxaban arm, despite failure to prove superiority, rivaroxaban was approved for the prevention of stroke and SE in NVAF. It may be noted that such trials will have 2 p values—1 for NI and 1 for superiority, which, as in this case, may go in different directions.

The RELY trial⁵ illustrates the issue of NI and sequential superiority particularly well (Fig. 1). The trial tested the efficacy of dabigatran versus dose-adjusted warfarin in a population with NVAF. The study design and endpoints were similar to ROCKET AF, with one difference: Two doses of dabigatran were tested. The trial had three arms—one in which patients were treated with dose-adjusted warfarin (control arm) and two intervention arms—one in which patients received dabigatran in a dose of 110 mg twice daily and a second in which a dose of 150 mg twice daily was given. The results of RELY in summary were (i) dabigatran 110 mg arm versus warfarin (HR 0.91; 95% CI 0.74–1.11) and (ii) dabigatran 150 mg arm versus warfarin (HR 0.66; 95% CI 0.54–0.82). As in ROCKET AF, the NI margin was set at 1.46. The results show that dabigatran 110 mg twice daily (upper 95% CI limit 1.11) was non-inferior but not superior to warfarin—p value for NI <0.001 ; p value for superiority 0.34. On the other hand, the 150 mg dose (upper 95% CI limit 0.82) was both non-inferior and superior to warfarin, p value for NI as well as for superiority <0.001.

Close

FIG 1.

Diagram to illustrate the difference between superiority, non-inferior (NI), and equivalence designs. In superiority trials (A), the point estimate (black square) as well as the 95% CI (lines on either side of the point estimate) should be less than an HR of 1.00, showing clear statistical superiority over control treatment. In a NI trial (B), the point estimate may be on either side of an HR 1.00, but the upper 95% CI should be within the preset NI margin M2 (see text). In an equivalence trial (C), both the point estimate and the 95% CI should mirror the control

Export to PPT

Fig. 2 illustrates the results of the RELY trial. In Fig. 2, the upper CI of the 110 mg dose (1.11) lies within the NI margin (1.46) but not within the superiority margin (1.0), indicating NI but not superiority. In the case of the 150 mg dose, the upper margin of the 95% CI (0.82) lies within both the NI and superiority margin, indicating both NI and superiority. However, while the 110 mg dose was non-inferior but not superior to warfarin it resulted in lower major bleeding (2.71% per year in the dabigatran arm and 3.36% per year in warfarin arm, p=0.003) as well as lower haemorrhagic stroke (0.12% per year in dabigatran arm, 0.38% per year in warfarin arm, p<0.001). For this reason, it is perceived to have an advantage over warfarin, especially in those at increased risk of haemorrhage and in older adults. The 150 mg twice daily dabigatran arm, besides being superior to warfarin in preventing stroke and SE, produced less haemorrhagic stroke (0.10% per year, p<0.001) and a similar risk of major bleeding (0.311% per year, p=0.31) and is considered the standard dose in NVAF.⁹ Although not of direct relevance, it is interesting to note that in comparing the 110 mg and 150 mg doses of dabigatran, the higher dose significantly reduced the rate of stroke/SE (p=0.005) but with a strong trend to an increased risk of major bleeding (p=0.052). The rate of death and haemorrhagic stroke (as against all major bleeds) was similar for the two doses.

Close

FIG 2.

Results of the RELY trial illustrating the concept of non-inferiority (NI) and superiority (sup) in the 2 doses of dabigatran used in the trial versus warfarin HR hazard ratio 95% CI 95% confidence interval

Export to PPT