Real-world evidence comparing oral anticoagulants in non-valvular atrial fibrillation: a systematic review and network meta-analysis

Background

Atrial fibrillation, the most common sustained cardiac arrhythmia, is associated with a fivefold increased risk of ischemic stroke and other systemic embolic events [1]. Atrial fibrillation that is not related to moderate-to-severe mitral valve stenosis or an artificial (mechanical) is recommended by the 2019 American Heart Association/American College of Cardiology/Heart Rhythm Society guideline as the definition of non-valvular atrial fibrillation (NVAF) [2]. In addition, the prescribing information of non-vitamin K antagonist oral anticoagulants (NOACs) excludes use in the presence of all prosthetic valves, including bioprosthetic valves. Warfarin has been the gold standard for stroke prevention in patients with atrial fibrillation for the past few decades, but its use is complicated by the need for regular coagulation monitoring (with dosage adjustments, as necessary) as well as the potential for numerous food and drug interactions [3]. Recently, NOACs have emerged as approved/recommended treatments for stroke prevention in patients with NVAF [4]. Both vitamin K antagonists (VKAs) and NOACs have had a significant impact on stroke prevention for patients with NVAF in clinical trials – RE-LY [5], ROCKET-AF [6], ARISTOTLE [7] and ENGAGE AF-TIMI 48 [8] – that have demonstrated how NOACs are at least as efficacious and safe as warfarin. Recent guidelines recommend NOACs over warfarin in eligible patients with NVAF [4,9], and there is increasing real-world evidence regarding the impact of oral anticoagulants on outcomes of interest for patients with NVAF. Previously published systematic literature reviews and network meta-analyses (NMAs) have compared NOACs to warfarin/VKAs and other NOACs; however, there is substantial variation in the design and methodology of published real-world evidence studies, and quantitative analyses of real-world evidence studies poses many challenges [10–23]. The aim of this project was to conduct a systematic literature review and NMA of real-world evidence studies comparing the effectiveness and safety of NOACs and VKAs in patients with NVAF, seeking to address these two main concerns:

• What is the comparative clinical effectiveness and safety of NOACs compared with VKAs in NVAF patients in terms of composite all-cause stroke or systemic embolism, ischemic stroke, major bleeding, gastrointestinal bleeding, intracranial hemorrhage and all-cause mortality?

• What is the impact of NOAC doses used and industry sponsorship on these relationships?

Methods

A systematic literature review was conducted in accordance with the standards set forth in established guidelines (i.e., Preferred Reporting Items for Systematic Reviews and Meta-Analyses [PRISMA] guidelines) [24,25] following a protocol that was developed a priori based on guidance from the Cochrane Handbook for Systematic Reviews of Interventions [26]. MEDLINE, MEDLINE In-Process and Embase (via Ovid) were searched on 30 January 2020 to capture peer-reviewed journal articles published from January 2013 to January 2020 (search strategy in Supplementary Table 1). The searches were conducted by one investigator; a second investigator validated the searches and search yield.

Predefined criteria for selecting real-world evidence studies (i.e., population, interventions and comparisons, outcomes, study design, and time period) are outlined in Supplementary Table 2. All studies eligible in the systematic literature review were eligible for NMA, including studies with outlier or comorbid populations. Eligible real-world evidence studies reported ≥1 outcome(s) of interest and compared ≥2 of the following: apixaban, dabigatran, edoxaban, rivaroxaban and VKAs (including warfarin). Observational comparative effectiveness studies were eligible for inclusion (e.g., prospective and retrospective cohort studies, cross-sectional studies, case–control studies and pragmatic trials). Only studies applying methods for reducing bias or confounding were included. Primary outcomes of interest were all-cause (ischemic or hemorrhagic) stroke or systemic embolism and major bleeding, and secondary outcomes included ischemic stroke, gastrointestinal bleeding, intracranial hemorrhage and all-cause mortality. Outcome definitions were grouped to combine similarities. The systematic literature review required articles to be written in English and available in full text. Real-world studies from any geographic location were considered for inclusion. All abstracts and full-text articles were screened by two researchers; discrepancies were resolved via discussion, with any remaining disagreements resolved by a third researcher.

Data extraction was conducted by a single researcher and cross-checked by a second, using a predefined spreadsheet form that captured study characteristics and methods, patient and treatment characteristics, outcomes assessed and timepoints of assessments (including follow-up duration), results, and conclusions. A third investigator was consulted to resolve any disagreements.

The Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) tool was used for evaluating the risk of bias in the results for each primary and secondary outcome of interest derived from each of the individual real-world studies informing the NMA [27]. Possible confounding bias was assessed for each study included in the base case. This tool assesses the risk of bias with respect to selection, measurement interventions, deviations from intended interventions, missing data, outcome assessment and selection of reported results.

Multiple articles reported analyses of the same database, with similar inclusion/exclusion criteria, study periods, interventions, comparators and dosing (i.e., related or ‘kin’ studies); there was also substantial population overlap across articles. When there is overlapping data (i.e., two or more estimates for the same treatment comparison in the same or overlapping population), it is necessary to select a single estimate to include in the NMA to avoid double-counting patients and overweighting study results [28]. A “parent” study was selected for inclusion in the analyses from each set of related publications, guided by a set of principles; the process is shown in Supplementary Table 3. One set of data, per database and per network, was selected for analysis. Because different networks require distinct sets of data (based on outcomes, doses or subgroups), various kin publications from the same database may have been selected for different analyses. Related/kin studies were retained in the base case analysis if they contributed unique treatment comparisons or did not overlap time periods with the ‘parent’ study. Similarly, for studies that presented data on both NOACs versus VKAs and NOACs versus NOACs, only NOAC versus VKA comparisons were included to prevent double counting.

Random-effects Bayesian NMAs [29] were performed to estimate hazard ratios comparing NOACs to VKAs and to each other for the outcomes of interest. These methods allowed for simultaneous consideration of all direct and indirect evidence, as well as proper modeling of the relationships in studies; a random effects model was chosen, as it was expected that the clinical and methodological heterogeneity across trials would lead to statistical heterogeneity in effects.

The base case analysis included studies on mixed, standard or not reported dosages. Dosages of NOACs were categorized as mixed or not reported, standard only (i.e., apixaban: 5 mg; dabigatran: 150 mg; edoxaban: 60 mg; and rivaroxaban: 20 mg) or reduced only (i.e., apixaban: 2.5 mg; dabigatran: 75 mg; edoxaban: 30 mg; and rivaroxaban: 15 mg). Subgroup analyses included dose categories (mixed or not reported only, standard only, reduced only) and study sponsorship (industry vs non-industry sponsored). Most studies reported hazard ratios, but a few studies reported relative risks, incidence rate ratios or odds ratios; these were considered to be similar enough to hazard ratios for inclusion. Adjusted (or matched) arm-level rates were used to calculate relative risks in a small number of papers.

A uniform prior of U [0,1] was used for the square root of between-study variation of effect, tau [τ] [30]. Other values were investigated; however, as expected given the number of large studies aiding in estimating a random effects variance component, there was no sensitivity of results to prior chosen. Noninformative priors (e.g., normal [0,10,000]) were used for other parameters (e.g., the treatment effect log hazards). Statistical heterogeneity was assessed by examining the size of τ estimated in the random effects models. Heterogeneity for each comparison was also assessed by conducting classical pairwise meta-analysis and calculating I², an estimate of the percentage of variability due to heterogeneity [31]. An I² value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity. Typically, I² values of 25%, 50% and 75% are considered low, moderate and high heterogeneity, respectively [31].

Analyses involved a 100,000 run-in iteration phase and a 100,000-iteration phase for parameter estimation. The median of the posterior distribution is reported as the point estimate (hazard ratio), and the corresponding 95% credible intervals were obtained with the 2.5th and 97.5th percentiles of the posterior distribution. Pairwise probabilities of each treatment being better than its comparator were calculated using the Bayesian posteriors of model estimates for each treatment comparison. All calculations were performed using OpenBUGS 3.2.3 [32].

Results

Systematic literature review

Systematic searches of electronic databases identified a total of 10,471 records; 7661 following duplicate removal. Of these, 6782 were excluded during title/abstract screening and 879 full-text publications were further evaluated for inclusion. After full-text screening, 143 publications were included (Figure 1). Across all studies eligible for base case analysis, 38 of these publications reported data on stroke or systemic embolism, 115 on major bleeding, 88 on ischemic stroke, 84 on intracranial hemorrhage, 89 on gastrointestinal bleeding and 56 on all-cause mortality. Following the application of the kinning hierarchy to select individual publications from among related publications analyzing each unique database, 55 publications were eligible for the base case analysis and 76 across subgroup analyses. Key details for studies included in the base case are described in the following, and additional data for both base case and subgroup analyses can be found in Supplementary Tables 4 & 5.

Most data included in the base case analyses were sourced from pharmacy/claims databases (n = 30), followed by registries (n = 17) and hospital databases (n = 8). Limited evidence was identified for edoxaban (n = 3), and all studies included an Asian population. The majority of VKA evidence was specific to warfarin; however, there were a few publications on non-warfarin VKAs (phenprocoumon, acenocoumarol) or nonspecific VKAs. Of the included publications, 13 were from North America (1 from Canada and the rest from the United States), 22 from Europe and 20 from the rest of the world (mostly Asian countries). Industry sponsorship supported 25 publications; 30 were not industry sponsored.

Some variation was observed with respect to baseline characteristics of the study participants. Mean age ranged from 55.8 to 83.9 years among treatment arms, with most patients aged ≥65 years. The proportion of males ranged from 36.3% to 87.1%. The mean congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, stroke or transient ischemic attack, vascular disease, age 65 to 74 years, sex category score varied between 1.1 and 5.27, with most patients scoring between 3 and 4. The mean HAS-BLED (Hypertension, Abnormal renal and liver function, Stroke, Bleeding, Labile international normalized ratio, Elderly, Drugs or alcohol) score varied between 1.4 and 3.7, with most patients scoring between 2 and 3. Most studies were in general NVAF patients, although some studies included NVAF patients with (an increased risk of) comorbidities. However, among all included publications, comorbidities were common, with diabetes, hypertension, heart failure and renal disease being the most commonly reported. There was some variation in the outcome definition of major bleeding but little variation for any of the other outcomes. The majority of studies used propensity score matching to adjust for potential confounding, with others using some form of regression modeling.

Observational studies have a higher risk of bias relative to randomized controlled trials. We found a moderate to serious overall risk of bias with the ROBINS-I tool (Supplementary Figures 1 & 2) for studies included in the base case. Most cases of serious bias were due to selection of participants into the study, confounding or deviations from intended interventions. It was expected that the propensity score or regression adjustment approaches would adjust for patient channeling/selection bias to a large extent. Bleeding outcomes were also limited by the risk of bias due to increased monitoring of patients receiving VKAs.

Network meta-analysis

Figure 2 presents the global evidence network of included studies across all outcomes (base case). NMA results for the primary outcomes of interest are presented in Figures 3 & 4. Base case results for secondary outcomes are shown in Figure 5, with all NMA results presented in Supplementary Figures 3–6. In what follows, and to aid interpretation, we use the common (but frequentist) term statistically significant when referring to 95% credible intervals that do not include 1.0 for a treatment comparison.

In the base case analysis, treatment with apixaban, dabigatran and rivaroxaban were each significantly associated with a reduced risk of all-cause stroke or systemic embolism compared with VKAs (Figure 3). Findings from the base case all-cause stroke or systemic embolism analysis were consistent with the mixed and standard dose subgroups, but not with the reduced dose subgroup, where findings were not significant. In the industry-sponsored subgroup, apixaban and dabigatran were significantly associated with a reduced risk of stroke or systemic embolism compared with VKAs. Contrary to the industry subgroup results, none of the findings were significant in the non-industry subgroup, although point estimates were similar and probabilities that treatments were better than VKAs were all above 90%.

With respect to major bleeding, all treatments except for rivaroxaban were significantly associated with a reduced risk versus VKAs in the base case; rivaroxaban had a >90% probability of being better than VKAs (Figure 4). All dose subgroup findings, as well as the subgroups by industry sponsorship, for major bleeding were consistent with the base case analysis. Across all analyses, wide credible intervals were observed for the edoxaban versus VKAs comparison.

In the base case analysis, treatment with all NOACs was significantly associated with a reduced risk of ischemic stroke compared with VKAs (Supplementary Figure 3). These findings were mostly consistent with the dose subgroups, although some findings were not significant, but all NOACs had a >90% probability of being better than VKAs. In the industry subgroup analyses, only apixaban was associated with a significant difference compared with VKAs, although the other NOACs all had a 90% probability of being better than VKAs. In the non-industry subgroup, edoxaban and rivaroxaban were also associated with a reduced risk of ischemic stroke.

Treatment with all NOACs was significantly associated with a reduced risk of intracranial hemorrhage compared with VKAs across the base case and subgroup analyses by dose and industry/non-industry sponsorship (Supplementary Figure 4). For the edoxaban versus VKAs comparison, a rather wide credible interval was observed.

In the base case analysis, treatment with apixaban and edoxaban was significantly associated with a reduced risk of gastrointestinal bleeding compared with VKAs, although for the edoxaban comparison, a wide credible interval was observed (Supplementary Figure 5). Dabigatran and rivaroxaban showed an increased risk of gastrointestinal bleeding relative to VKAs, although these findings were only significant for the rivaroxaban comparison. Findings were mostly consistent with all subgroup analyses, except for edoxaban versus VKAs where findings were not significant.

With the exception of edoxaban, NOACs were significantly associated with a reduced risk of all-cause mortality relative to VKAs (Supplementary Figure 6). Findings were consistent with the mixed and standard dose subgroups, but not the reduced dose subgroup, where no significant differences were observed between edoxaban or rivaroxaban and VKAs. Results from the industry and non-industry subgroups were mostly consistent with the base case, with the exception of the edoxaban comparison.

Statistical heterogeneity, defined as the variance of observed treatment effects more than would be expected due to sampling error alone, was calculated by examining I² values from direct, frequentist meta-analyses. Investigation of inconsistency was difficult; inconsistency occurs when there is conflict between independent sources of direct and indirect evidence, but the majority of trials had three or more arms, and a full third of trials had four or more arms, meaning that consistency was to some degree guaranteed. For instance, in the apixaban/VKA/rivaroxaban loop, for each outcome, at least half of the studies contributing to connecting those treatments had all three arms. Thus, we focused on examination of heterogeneity.

In the base case, there was little evidence of detectable heterogeneity in intracranial hemorrhage (all comparisons had I² <40%), whereas there was some evidence of heterogeneity in all-cause stroke or systemic embolism (particularly apixaban vs VKAs: I² = 85.70%, although this direct meta-analyses for this outcome only had five studies, whereas the other direct meta-analyses in other outcomes with high I² values had >10 studies), all-cause mortality (particularly rivaroxaban vs VKAs: I² = 96.40%, apixaban vs VKAs: I² = 96.10%, and dabigatran vs VKAs: I² = 87.10%), gastrointestinal bleeding (particularly rivaroxaban vs VKAs: I² = 81.10%), ischemic stroke (particularly apixaban vs VKAs: I² = 80.80%) and major bleeding (particularly rivaroxaban vs VKAs: I² = 87.50%). The pattern of heterogeneity in the subgroup analyses was similar to the base case except in the industry-sponsored subgroup, with less heterogeneity observed in the subgroup analyses for intracranial hemorrhage and more heterogeneity in the subgroup analyses for all-cause stroke or systemic embolism, ischemic stroke, major bleeding, gastrointestinal bleeding and all-cause mortality.

Discussion

Summary of main findings

The results provided by this NMA provide a more comprehensive understanding of the effectiveness and safety of NOACs compared with VKAs in the real world. The current NMA assessed the risk of all-cause stroke or systemic embolism, major bleeding, ischemic stroke, intracranial hemorrhage, gastrointestinal bleeding and all-cause mortality among all NOACs compared with VKAs based on real-world evidence. In the base case analyses, treatment with apixaban, dabigatran and rivaroxaban was significantly associated with a reduced risk of stroke or systemic embolism. With respect to major bleeding, all NOACs except for rivaroxaban were associated with a reduced risk of major bleeding.

Treatment with apixaban, dabigatran and rivaroxaban was further significantly associated with a reduced risk of ischemic stroke, intracranial hemorrhage and all-cause mortality when compared against VKAs. Edoxaban showed a reduced risk of ischemic stroke, intracranial hemorrhage, gastrointestinal bleeding and all-cause mortality when compared with VKAs, but wide credible intervals were observed for this comparison across the analyses. Apixaban and edoxaban were associated with a significantly reduced risk of gastrointestinal bleeding versus VKAs, whereas dabigatran and rivaroxaban showed no significant differences; however, wide credible intervals were observed for the edoxaban comparison.

Given the concern that exclusion of dependent data within the database might result in missing important findings, a sensitivity analysis (an ‘all comers analysis,’ across dependent data) in which the primary unit is the publication, not the database, was also conducted (results are not shown). In this analysis, patient overlap was ignored as a concern when selecting data within databases. When comparing the relative treatment effects for NOACs versus VKAs across all outcomes, only minor differences were observed from the base case.

This NMA offers a comprehensive assessment of the impact of NOAC treatment in the real-world practice. By combining findings from several studies, this review makes the available evidence more accessible to healthcare professionals, and it offers a more reliable and precise estimate of NOAC effects than one study alone. As such, the evidence contained in this review may be considered as part of the body of evidence informing future clinical guidelines for NOACs, and it can guide routine evidence-based clinical decision making.

Relationship with existing work (in terms of results & methods)

The findings of our NMA for the primary outcomes of interest are broadly consistent with previously published indirect treatment comparisons/NMAs of real-world evidence [10,14,16,18,33] and randomized controlled trials [34–37]. Similar to the findings of our present study, NMAs of clinical trial data have showed that apixaban is consistently associated with a reduced risk of major bleeding relative to warfarin. Deitelzweig et al. previously published an NMA in real-world evidence and found that NOACs were associated with a similar of lower risk of major bleeding when compared against VKAs [16]. Furthermore, the authors reported that no significant differences were observed between rivaroxaban and warfarin, whereas apixaban and dabigatran were associated with a lower risk of major bleeding versus warfarin. These findings are similar to findings from our NMA.

Another previously conducted NMA by Escobar et al. in real-world evidence also investigated some of the secondary outcomes that were of interest in our NMA (ischemic stroke and intracranial hemorrhage) [18]. With respect to ischemic stroke, apixaban and dabigatran were not associated with a reduced risk of ischemic stroke versus warfarin, whereas rivaroxaban significantly reduced the risk of ischemic stroke versus warfarin. Contrary to the findings from that NMA, in our NMA all NOACs showed a reduced risk of ischemic stroke compared with VKAs. With respect to intracranial hemorrhage, all NOACs significantly reduced the risk of intracranial hemorrhage compared with warfarin in the other NMA, which was also observed in our NMA.

Strengths

A comprehensive search strategy was used, capturing a great number of studies with large sample sizes. Meta-analyses require that patients included in multiple studies not be double counted, as doing so risks spurious or overprecise results. To avoid double counting of patients, we applied a rigorous kinning process to select studies for the NMA. Another strength of the study is that we only included studies that controlled for confounding factors, using propensity score matching or other forms of multivariate adjustments, which minimizes the risk of selection or confounding bias inherent to real-world evidence studies. Additionally, specific analyses according to NOAC dosages and study sponsorship were performed, increasing the validity and generalizability of the results. We further conducted a detailed assessment of the methodological and clinical heterogeneity among studies. The real-world evidence studies included in this NMA evaluated a much broader NVAF population than included in randomized controlled trials. Despite this, findings are generally consistent with trials.

Limitations

Although one of the primary benefits of conducting a systematic review is to capture all available evidence, this study is limited by its search strategies and is only relevant up until the more recent search date (January 2020). Despite this limitation, the abundance of literature identified in this field, as well as the quality of the studies, provides a sufficient evidence base for the analyses, and additional availability of more recent literature may have little impact on the quality and direction of the NMA results.

Evidence on edoxaban was limited, and wide credible intervals were observed for the edoxaban comparison across the analyses. In addition, when comparing subgroup analyses against the base case, more variation was observed for the edoxaban comparisons. As such, these data should be interpreted with caution.

Despite attempts to limit the presence of methodological and clinical heterogeneity between studies (e.g., systematic literature review selection criteria, removing outliers during kinning and variations in outcome definitions), any remaining heterogeneity may compromise the findings of this NMA. Outlier or nongeneralizable populations were not excluded from the analysis, to maintain a comprehensive approach to study inclusion. Although after application of our kinning rules most outlier populations fell out of the analyses (e.g., patients with NVAF + a specific comorbidity or characteristic), some remained in the analysis because they were the only available evidence. We do not believe that this has impacted the results, as NVAF patients generally have several comorbid conditions.

In most of the identified real-world evidence studies involving VKAs, warfarin was the VKA studied. Therefore, the few studies of phenprocoumon, acenocoumarol and general VKAs were combined into a single VKA node. Given that there were only a limited number of studies that included phenprocoumon or acenocoumarol and estimates did not substantially differ from warfarin, we did not conduct a sensitivity analysis excluding non-warfarin VKAs.

Some differences were observed across the outcomes with respect to risk of bias. These differences are largely due to variability in how clearly the studies reported the measurement for the outcomes, and for outcomes where more monitoring could lead to the outcome being diagnosed more frequently (i.e., bleeding outcomes in patients receiving VKAs), this was a potential source of bias. For example, all-cause mortality would not be subject to such bias. Differential monitoring and outcome reporting between different anticoagulant treatments may bias findings; however, this is more likely to occur when NOACs are compared with VKAs, which need routine laboratory monitoring. Differential monitoring among anticoagulants from the same class (i.e., NOACs vs NOACs) was not considered a limitation, as guidelines for monitoring patients (i.e., for renal function) do not differ by the type of NOAC [2].

Future research

Future research should focus on head-to-head comparisons of NOACs in randomized controlled trials and pragmatic controlled trials, as these data are currently lacking in the literature. Furthermore, an NMA combining randomized controlled trials and real-world evidence would be beneficial. Studies were typically not able to adequately determine treatment adherence, and thus there was little information available on the risk of bias from that perspective; future studies evaluating adherence to the treatments could help confirm the findings of these database studies. Studies should also be comprehensive in reporting/measuring potential confounders, and they should not exclude treatment switchers from analysis.

Conclusion

In the absence of a large, rigorous randomized controlled trial pooling all marketed NOACs and VKAs, NMAs represent the best method for assessing comparative effectiveness and safety. Further evaluation of real-world evidence has the potential of showing the effectiveness and safety of these treatments in the clinical setting. Studies of real-world data serve as an important supplement to randomized controlled trials when evaluating the comparative effectiveness and safety profiles of the various NOACs across classes of, and in comparison to, VKAs.

The present NMA based on real-world evidence found that treatment with NOACs was associated with a similar or reduced risk of all-cause stroke or systemic embolism, major bleeding, ischemic stroke, intracranial hemorrhage and all-cause mortality versus VKAs. The findings from this NMA are consistent with evidence from clinical trials.