Abstract: PD58-11
Sources of Funding: None

Introduction

Randomized controlled trials (RCTs) have the potential of providing high quality evidence to inform clinical practice. This quality relies not only on safeguards against bias, but also on statistical power. In this study, we determined the Fragility Index of urological RCTs as a novel metric (Walsh M et al, JCE 2014) to assess the robustness of statistically significant results.

Methods

Statistical significance implies that an observed event is unlikely to occur by chance alone. The fragility index is defined as the minimum number of patients in an arm of a trial whose status would have to change from "non-event" to "event" in order to turn a statistically significant result into a non-significant one. All RCTs published in the 4 major urology journals between 2011-2015 were identified. We excluded studies not reporting dichotomous outcomes, as well as those with non-significant results and non-parallel designs. We applied the Fisher exact test to determine fragility index values.

Results

332 RCTs were identified, and 42 studies met inclusion criteria. Median sample size (IQR) was 99 (65, 179), while median event rate per study outcome was 38 (24, 65). The median fragility index was 3 (1, 4.5), indicating that an addition of only three alternate events to an arm of the average trial would have eliminated its statistical significance. There was statistically significant correlation between the fragility index and events per study (ρ=0.552, p=0.01) as well as sample size (ρ=0.493, p=0.01).

Conclusions

Statistically significant results in urology RCTs are often fragile, with significance hinging on few events. This is of particular concern in studies that may have large loss to follow-up numbers. Urologists should therefore interpret RCTs cautiously. There may be a role for reporting fragility index values routinely alongside the p-value to provide additional guidance as to the statistical robustness of findings.

Funding

None