Target Trial Emulation for Evaluating Health Policy
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Target trial emulation is an approach to designing rigorous nonexperimental studies by “emulating” key features of a clinical trial. Most commonly used outside of policy contexts, this approach is also valuable for policy evaluation as policies typically are not randomly assigned. In this article, we discuss the application of the target trial emulation framework in a policy evaluation context. The policy trial emulation framework includes 7 components: the units and eligibility criteria, definitions of the exposure and comparison conditions, assignment mechanism, baseline (“time zero”) and follow-up, outcomes, causal estimand, and statistical analysis and assumptions. Policy evaluations that emulate a randomized trial across these dimensions can yield estimates of the causal effects of the policy on outcomes. Using the policy trial emulation framework to conduct and report on research design and methods supports transparent assessment of threats to causal inference in nonexperimental studies intended to assess the effect of a health policy on clinical or population health outcomes.
Get full access to this article
View all available purchase options and get full access to this article.
Supplemental Material
Supplementary Material
- Download
- 378.11 KB
References
1.
Kingdon JW. Agendas, Alternatives, and Public Policies. Longman Classics in Political Science. 2nd ed. Longman; 2003:253.
2.
Rawat P, Morris JC. Kingdon’s “Streams” model at thirty: still relevant in the 21st century? Polit Policy. 2016;44:608-638. [10.1111/POLP.12168] doi: 10.1111/POLP.12168
3.
Dahabreh IJ, Matthews A, Steingrimsson JA, et al. Using trial and observational data to assess effectiveness: trial emulation, transportability, benchmarking, and joint analysis. Epidemiol Rev. 8 February 2023. [Epub ahead of print]. [PMID: 36752592] doi: 10.1093/epirev/mxac011
4.
Hansford HJ, Cashin AG, Jones MD, et al. Reporting of observational studies explicitly aiming to emulate randomized trials: a systematic review. JAMA Netw Open. 2023;6:e2336023. [PMID: 37755828] doi: 10.1001/jamanetworkopen.2023.36023
5.
Wan EYF, Yan VKC, Mok AHY, et al. Effectiveness of molnupiravir and nirmatrelvir-ritonavir in hospitalized patients with COVID-19: a target trial emulation study. Ann Intern Med. 2023;176:505-514. [PMID: 36913693] doi: 10.7326/M22-3057
6.
Hulme WJ, Williamson E, Horne EMF, et al. Challenges in estimating the effectiveness of COVID-19 vaccination using observational data. Ann Intern Med. 2023;176:685-693. [PMID: 37126810] doi: 10.7326/M21-4269
7.
Lazzati A, Epaud S, Ortala M, et al. Effect of bariatric surgery on cancer risk: results from an emulated target trial using population-based data. Br J Surg. 2022;109:433-438. [PMID: 35136932] doi: 10.1093/bjs/znac003
8.
Fu EL. Target trial emulation to improve causal inference from observational data: what, why, and how? J Am Soc Nephrol. 2023;34:1305-1314. [PMID: 37131279] doi: 10.1681/ASN.0000000000000152
9.
Hernán MA, Wang W, Leaf DE. Target trial emulation: a framework for causal inference from observational data. JAMA. 2022;328:2446-2447. [PMID: 36508210] doi: 10.1001/jama.2022.21383
10.
Matthews AA, Danaei G, Islam N, et al. Target trial emulation: applying principles of randomised trials to observational studies. BMJ. 2022;378:e071108. [PMID: 36041749] doi: 10.1136/bmj-2022-071108
11.
Ben-Michael E, Feller A, Stuart EA. A trial emulation approach for policy evaluations with group-level longitudinal data. Epidemiology. 2021;32:533-540. [PMID: 34001754] doi: 10.1097/EDE.0000000000001369
12.
McGinty EE, Tormohlen KN, Seewald NJ, et al. Effects of U.S. state medical cannabis laws on treatment of chronic noncancer pain. Ann Intern Med. 2023;176:904-912. [PMID: 37399549] doi: 10.7326/M23-0053
13.
Hansford HJ, Cashin AG, Jones MD, et al. Development of the TrAnsparent ReportinG of observational studies Emulating a Target trial (TARGET) guideline. BMJ Open. 2023;13:e074626. [PMID: 37699620] doi: 10.1136/bmjopen-2023-074626
14.
Hemming K, Taljaard M. Reflection on modern methods: when is a stepped-wedge cluster randomized trial a good study design choice? Int J Epidemiol. 2020;49:1043-1052. [PMID: 32386407] doi: 10.1093/ije/dyaa077
15.
Stuart EA, Huskamp HA, Duckworth K, et al. Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Serv Outcomes Res Methodol. 2014;14:166-182. [PMID: 25530705] doi: 10.1007/s10742-014-0123-z
16.
Feller A, Connors MC, Weiland C, et al. Addressing missing data due to COVID-19: two early childhood case studies. J Res Educ Eff. 15 August 2024. [Epub ahead of print]. doi: 10.1080/19345747.2024.2321438.
17.
Miller D, Spybrook J, Caverly S. Missing Data in Group Design Studies: Revisions in WWC Standards Version 4.0. Institute of Education Sciences; 2019.
18.
Incze MA, Kelley AT, Singer PM. Heterogeneous state cannabis policies: potential implications for patients and health care professionals. JAMA. 2021;326:2363-2364. [PMID: 34797377] doi: 10.1001/jama.2021.21182
19.
Ramanathan T, Hulkower R, Holbrook J, et al. Legal epidemiology: the science of law. J Law Med Ethics. 2017;45:69-72. [PMID: 28661299] doi: 10.1177/1073110517703329
20.
Griffin BA, Schuler MS, Pane J, et al. Methodological considerations for estimating policy effects in the context of co-occurring policies. Health Serv Outcomes Res Methodol. 2023;23:149-165. [PMID: 37207017] doi: 10.1007/s10742-022-00284-w
21.
Matthay EC, Hagan E, Joshi S, et al. The revolution will be hard to evaluate: how co-occurring policy changes affect research on the health effects of social policies. Epidemiol Rev. 2021;43:19-32. [PMID: 34622277] doi: 10.1093/epirev/mxab009
22.
Hayes RJ, Moulton LH. Cluster Randomised Trials. CRC Biostatistics Series. 2nd ed. CRC Pr; 2017.
23.
Verbitsky-Savitz N, Raudenbush SW. Causal inference under interference in spatial settings: a case study evaluating community policing program in Chicago. Epidemiol Methods. 2012;1:107-130.
24.
Baker AC, Larcker DF, Wang CCY. How much should we trust staggered difference-in-differences estimates? J Financ Econ. 2022;144:370-395.
25.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183:758-764. [PMID: 26994063] doi: 10.1093/aje/kwv254
26.
Cengiz D, Dube A, Lindner A, et al. The effect of minimum wages on low-wage jobs. Q J Econ. 2019;134:1405-1454.
27.
Deshpande M, Li Y. Who is screened out? Application costs and the targeting of disability programs. Am Econ J Econ Policy. 2019;11:213-248.
28.
Wing C, Yozwiak M, Hollingsworth A, et al. Designing difference-in-difference studies with staggered treatment adoption: key concepts and practical guidelines. Annu Rev Public Health. 2024;45:485-505. [PMID: 38277791] doi: 10.1146/annurev-publhealth-061022-050825
29.
Hernán MA, Robins JM, García Rodríguez LA. Discussion on “Statistical Issues Arising in the Women’s Health Initiative”. Biometrics. 2005;61:922-930.
30.
Schaubel DE, Wolfe RA, Port FK. A sequential stratification method for estimating the effect of a time-dependent experimental treatment in observational studies. Biometrics. 2006;62:910-917. [PMID: 16984335] doi: 10.1111/j.1541-0420.2006.00527.x
31.
Rothbard S, Etheridge JC, Murray EJ. A tutorial on applying the difference-in-differences method to health data. Curr Epidemiol Rep. 2024;11:85-95.
32.
Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;66:688-701.
33.
Musci RJ, Stuart E. Ensuring causal, not casual, inference. Prev Sci. 2019;20:452-456. [PMID: 30613853] doi: 10.1007/s11121-018-0971-9
34.
Hernán MA. The C-word: scientific euphemisms do not improve causal inference from observational data. Am J Public Health. 2018;108:616-619. [PMID: 29565659] doi: 10.2105/AJPH.2018.304337
35.
Hernán MA, Sauer BC, Hernández-Díaz S, et al. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. 2016;79:70-75. [PMID: 27237061] doi: 10.1016/j.jclinepi.2016.04.014
36.
Rassen JA, Brookhart MA, Glynn RJ, et al. Instrumental variables I: instrumental variables exploit natural variation in nonexperimental data to estimate causal relationships. J Clin Epidemiol. 2009;62:1226-1232. [PMID: 19356901] doi: 10.1016/j.jclinepi.2008.12.005
37.
Campbell MJ, Walters SJ. How to Design, Analyse and Report Cluster Randomised Trials in Medicine and Health Related Research. Statistics in Practice. J Wiley; 2014.
38.
Roth J, Sant’Anna PHC, Bilinski A, et al. What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. J Econom. 2023;235:2218-2244.
39.
Zeldow B, Hatfield LA. Confounding and regression adjustment in difference-in-differences studies. Health Serv Res. 2021;56:932-941. [PMID: 33978956] doi: 10.1111/1475-6773.13666
40.
Roth J, Sant’Anna PHC. When is parallel trends sensitive to functional form? Econometrica. 2023;91:737-747.
41.
Daw JR, Hatfield LA. Matching and regression to the mean in difference-in-differences analysis. Health Serv Res. 2018;53:4138-4156. [PMID: 29957834] doi: 10.1111/1475-6773.12993
42.
Daw JR, Hatfield LA. Matching in difference-in-differences: between a rock and a hard place. Health Serv Res. 2018;53:4111-4117. [PMID: 30047131] doi: 10.1111/1475-6773.13017
43.
Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s Tobacco Control Program. J Am Stat Assoc. 2010;105:493-505.
44.
Callaway B, Sant’Anna PHC. Difference-in-differences with multiple time periods. J Econom. 2021;225:200-230.
45.
Ben-Michael E, Feller A, Rothstein J. The augmented synthetic control method. J Am Stat Assoc. 2021;116:1789-1803.
46.
Bandara SN, Kennedy-Hendricks A, Stuart EA, et al. The effects of the Maryland Medicaid Health Home Waiver on Emergency Department and inpatient utilization among individuals with serious mental illness. Gen Hosp Psychiatry. 2020;64:99-104. [PMID: 31948691] doi: 10.1016/j.genhosppsych.2019.12.004
47.
Griffin BA, Schuler MS, Stuart EA, et al. Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies. BMC Med Res Methodol. 2021;21:279. [PMID: 34895172] doi: 10.1186/s12874-021-01471-y
48.
Dahabreh IJ, Bibbins-Domingo K. Causal inference about the effects of interventions from observational studies in medical journals. JAMA. 2024;331:1845-1853. [PMID: 38722735] doi: 10.1001/jama.2024.7741
49.
Guallar E, Goodman SN, Localio AR, et al. Seeing the positive in negative studies. Ann Intern Med. 2023;176:561-562. [PMID: 36940441] doi: 10.7326/M23-0576
Information & Authors
Information
Published In
Annals of Internal Medicine
Volume 177 • Number 11 • November 2024
Pages: 1530 - 1538
History
Published online: 8 October 2024
Published in issue: November 2024
Keywords
Copyright
© 2024 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Target Trial Emulation for Evaluating Health Policy. Ann Intern Med.2024;177:1530-1538. [Epub 8 October 2024]. doi:10.7326/M23-2440
Target Trial Emulation for Evaluating Health Policy. Ann Intern Med.2024;177:1530-1538. [Epub 8 October 2024]. doi:10.7326/M23-2440
View More
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Comments
0 Comments