Deriving Real-World Insights From Real-World Data: Biostatistics to the Rescue

In reviewing promising approaches to derive real world insights from real-world data, Pencina et al. (1) discuss comparative research methods that mimic randomized trials. One such comparative research method, which was not mentioned in the article but should be in the “tool kit” of clinical researchers, is the paired availability design (2-5), a design and analysis for multiple before-and-after studies over a short time.
The design aspect involves two steps. First, investigators collect outcome data at medical centers in the “before” period, when there is limited or no availability of a new treatment. Second, investigators increase the availability of the new treatment in these medical centers in the “after” period. In each period some patients receive the old treatment, and some receive the new treatment. To make the two periods like randomization groups, investigators should select medical centers that are geographically isolated or army medical centers, so there is little in- or out- migration that could bias results. Investigators should also select centers where aspects of patient management, excluding the treatment of interest, remain the same in the two periods.
The analysis aspect also involves two steps. First, investigators estimate the causal effect of the new treatment in each medical center. Let D denote the difference in fractions with a positive outcome between the “before” and “after” periods. The drawback with using D is that it is diluted by the availability of the new treatment. To circumvent this problem, the paired availability design uses D/∆, where ∆ denote the difference in the fraction who receive the new treatment in the two periods. Under reasonable assumptions, D/∆ is the estimated causal effect of receiving new treatment. Second, investigators average of D/∆ over multiple studies to obtain an overall estimate that reduces chance variation. To investigate generalizability, investigators should plot D/∆ against ∆.
The quintessential application of the paired availability design was estimating the effect of labor epidural analgesia (LEA) on the probability of Cesarean section (C/S) (4). Randomized trials, which showed little effect of LEA on C/S, are subject to bias from crossovers occurring after randomization. Multivariate adjustments in observational studies, which showed a large effect of LEA on C/S, are likely biased by the omission of a key confounder, namely pain during labor. The paired availability design, which showed no effect of LEA on C/S, avoided the biases of the other methods.

References
1. Pencina MJ, Rockhold FW, D'Agostino RB. Deriving Real-World Insights From Real-World Data: Biostatistics to the Rescue. Ann Intern Med. [Epub ahead of print 24 July 2018] doi: 10.7326/M18-1093
2. Baker SG. and Lindeman KS. The paired availability design, a proposal for evaluating epidural analgesia during labor. Stat Med, 1994; 13, 2269-2278.
3. Baker SG, Lindeman KL, and Kramer, BS The paired availability design for historical controls. BMC Med Res Method 2001; 1:9
4. Baker SG and Lindeman KL Revisiting a discrepant result: a propensity score analysis, the paired availability design for historical controls, and a meta-analysis of randomized trials, J Causal Inference 2013; 1:51–82.
5. Baker, S. G., Kramer, B. S. and Lindeman, K. L. 6), Latent class instrumental variables. A clinical and biostatistical perspective. Stat Med, 2016; 35:147-160.