Free access
Research and Reporting Methods
13 April 2021

A Deferred-Vaccination Design to Assess Durability of COVID-19 Vaccine Effect After the Placebo Group Is VaccinatedFREE

Publication: Annals of Internal Medicine
Volume 174, Number 8


Multiple candidate vaccines to prevent COVID-19 have entered large-scale phase 3 placebo-controlled randomized clinical trials, and several have demonstrated substantial short-term efficacy. At some point after demonstration of substantial efficacy, placebo recipients should be offered the efficacious vaccine from their trial, which will occur before longer-term efficacy and safety are known. The absence of a placebo group could compromise assessment of longer-term vaccine effects. However, by continuing follow-up after vaccination of the placebo group, this study shows that placebo-controlled vaccine efficacy can be mathematically derived by assuming that the benefit of vaccination over time has the same profile for the original vaccine recipients and the original placebo recipients after their vaccination. Although this derivation provides less precise estimates than would be obtained by a standard trial where the placebo group remains unvaccinated, this proposed approach allows estimation of longer-term effect, including durability of vaccine efficacy and whether the vaccine eventually becomes harmful for some. Deferred vaccination, if done open-label, may lead to riskier behavior in the unblinded original vaccine group, confounding estimates of long-term vaccine efficacy. Hence, deferred vaccination via blinded crossover, where the vaccine group receives placebo and vice versa, would be the preferred way to assess vaccine durability and potential delayed harm. Deferred vaccination allows placebo recipients timely access to the vaccine when it would no longer be proper to maintain them on placebo, yet still allows important insights about immunologic and clinical effectiveness over time.
Safe and durably effective SARS-CoV-2 vaccines hold the potential to dramatically alter the COVID-19 pandemic (1, 2). Several vaccine candidates are currently in phase 3 clinical testing, with key criteria for success including vaccine efficacy and safety. Early results from multiple trials (3–8) suggest high efficacy that far exceeds the U.S. Food and Drug Administration guidance threshold of 50% for symptomatic disease and severe disease. Yet critical questions remain, including the effects in such subgroups as elderly persons and ethnic minority populations and assessment of longer-term efficacy and safety, given theoretical concerns for harm, including vaccine-associated enhanced disease (a potential concern in such subgroups as elderly persons [9–11]), and waning protection (9, 12, 13). The latter concern must be considered in light of studies of seasonal coronaviruses (14–16) and natural infection by SARS-CoV-2 (17–20), which suggest that immunity may wane within 6 months to 2 years in some people.
Understanding whether durability of vaccine efficacy might be improved by revaccination is another critical question (13, 15, 16). Although long-term safety and durability of efficacy are best evaluated by continued blinded follow-up of the original study groups (21), at some point there is an ethical imperative for placebo recipients to be offered the vaccine. The timing of this offer is complex, with individual risk being weighed against the societal value of the continued placebo-controlled follow-up, society's perception of fairness, and the availability of the vaccine outside the trial (22).
We demonstrate the remarkable fact that vaccine efficacy versus placebo can still be estimated even after the placebo group has been completely immunized, provided that a trial continues with high levels of follow-up and compliance. We assert that the ideal way to maintain rigor is by implementing a blinded crossover design in which the original study groups receive the opposite product during the deferred immunization period (creating early versus deferred vaccination groups). Besides assessment of durability, continued follow-up allows for collecting measurements of postvaccination immune responses in the newly vaccinated recipients. These additional measurements can substantially increase the statistical power in immune correlate analyses that assess whether and how these immune responses correlate with risk for disease or with vaccine efficacy (23), 2 important goals in vaccine trials. In addition, continued follow-up allows a quick pivot to a randomized trial of a booster dose of the vaccine to assess whether waning efficacy can be reversed.

Recovery of Vaccine Efficacy Without a Placebo Group

Figure 1 shows a schematic for deferred immunization of the placebo group via blinded crossover. After randomization to the vaccine or the placebo group, participants are followed for COVID-19 case accrual and early efficacy is established. After some time (for example, once regulatory approval has been granted), the placebo group is offered vaccination so that all willing volunteers receive the efficacious vaccine. To keep the blind, the original vaccine group receives placebo and vice versa. Thus, the trial has changed into a blinded randomized crossover trial of immediate (original vaccine) versus deferred (original placebo) vaccination, so that 2 distinct interventions remain that can be contrasted. Intuitively, if the case accrual rate for the original vaccine group is higher than that for the recently vaccinated group, efficacy must be waning. Critically, post-crossover vaccine efficacy can be estimated by assuming the newly vaccinated persons in period 1 or period 2 receive the same benefit.
Figure 1. Schematic of a standard trial of vaccine versus placebo that pivots to a trial of immediate versus deferred vaccination using blinded crossover.
At some point after a positive primary efficacy signal, placebo group participants receive the vaccine and the vaccine group participants receive placebo. A balanced case split between study groups in period 2 supports maintenance of the period 1 vaccine efficacy. A key assumption is that vaccine efficacy for the newly vaccinated is the same whether at the start of period 1 or at the start of period 2. The tapering and fading blue wedge after vaccination indicates a potential waning of efficacy.
Figure 2 shows this concept for a vaccine with 80% efficacy in period 1. We deduce the case count for an inferred period 2 placebo group by requiring that the inferred count aligns with the vaccine efficacy observed in period 1 for the original vaccine recipients. We then use this inferred case count to deduce the period 2 vaccine efficacy. The approach generalizes beyond 2 equal periods and equal randomization, but the key aspects are easiest to develop in this simpler setting.
Figure 2. Schematic of how deferred placebo vaccination allows imputation of the case counts for an inferred placebo group.
The vaccine efficacy in period 2 for the newly vaccinated (deferred vaccine group) is assumed to be the same 80% that was observed in the newly vaccinated (immediate vaccine group) in period 1. This logic implies that a counterfactual placebo group of 35 persons would have about 10 cases, because this satisfies 100%[1 – (2/35)/(10/35)] = 80%. Thus the vaccine efficacy for the original vaccine group in period 2 has waned 55%, calculated as 100% [1 – (5/39)/(10/35)]. The lighter blue shade of the immediate vaccine group participants in period 2 indicates waning immunity.
Table 1 provides more detail. During period 1, vaccine efficacy is estimated as VE1 = 100%(1 − 25/125) = 80%, the reduction in observed cases as a result of the vaccine. After vaccination of the placebo group, we evaluate 2 scenarios. In scenario 1, the number of cases is similar in period 2 with a relative risk (original vaccine/original placebo) in period 2 of RR2 = 41/39, which suggests that little vaccine efficacy has been lost. Using the period 1 vaccine efficacy, we infer the number of placebo cases that could have been seen had there been a standard trial, as 39 × (125/25) = 195. We then calculate vaccine efficacy in period 2 as:
We obtain the same estimate of 79% even if the period 2 case counts are halved or tripled, and thus this procedure does not require a constant placebo case accrual rate. For scenario 2, there is clear evidence of loss of durability, as RR2 = 53/9 = 5.89. Indeed, the vaccine efficacy becomes negative, suggesting potential harm in period 2. Such a scenario is extreme but illustrates that one can assess whether a vaccine eventually turns harmful.
Table 1. Calculation of Placebo-Controlled Vaccine Efficacy by Inferring a Placebo Group After Placebo Group Vaccination


To obtain unbiased assessments about durability of efficacy, a standard trial requires 1) adequate and nondifferential retention after the original randomization such that the 2 randomized cohorts remain balanced at the start of period 2; 2) that there are no unmeasured host or viral modifiers of vaccine efficacy that have a different distribution across the follow-up periods; and 3) consistency in case assessment across periods. The only additional assumption required of the deferred vaccination design is that the newly vaccinated obtain the same benefit of vaccination in period 1 as in period 2. Of note, the deferred vaccination design is valid without making assumptions about whether and how background incidence changes from period 1 to period 2.
Because participants are removed from analysis for dropout or after acquiring disease, the two groups might not be similar at the start of period 2. One example would be differential censoring due to death or dropout. With an effective vaccine, the more vulnerable placebo group participants (for example, those with riskier behavior or frailer health) will acquire COVID-19 during period 1 at a greater rate than the matching vulnerable population of vaccinated participants, who were protected by vaccination (24–27). So period 2 can start with fewer vulnerable people in the original placebo group compared with the original vaccine group. Thus, the vaccine efficacy may appear to wane, but only because of a biased comparison. For large COVID-19 vaccine trials in which perhaps 1% to 5% of the participants develop disease, the proportion of participants excluded from period 2 and the resulting bias are unlikely to be a substantial concern (28). Nonetheless, any potential bias can be ameliorated by statistically adjusting for differential risk using measured characteristics that predict risk for disease. In addition, true waning vaccine efficacy is supported if the kinetics of antibody waning in individuals track with individual risk over time.
A developing issue that could lead to differences between the 2 periods is the emergence of variants of concern (VOCs) of SARS-CoV-2 (29–35) against which established vaccines may have reduced efficacy (36). Underlying this concern is the observation that differential vaccine efficacy by pathogen feature (such as genotype or amino acid sequence feature) has been seen, at least in some populations, for vaccines against malaria (37), HIV (38), and dengue (39). Provided that the VOCs are present in period 1, the methods in this article can be applied separately to different strains including original and VOCs where, for example, disease is redefined as disease caused by a specific VOC. More sophisticated sieve analysis methods that were developed for strain-specific efficacy of HIV can also be applied (40). If a completely new VOC emerges in period 2, one could check for waning vaccine efficacy for the new VOC by comparing case counts of the new VOC in the immediate versus deferred vaccination groups. If these were similar, it would suggest the vaccine efficacy for the VOC had not changed. However, the actual efficacy for the new VOC would be unknown and could be large or close to zero.

Estimation and Comparative Performance

A modified Poisson method to estimate overall vaccine efficacy can be generalized to provide period-specific estimates of relative risk and vaccine efficacy (41, 42). This person-year approach allows for differential follow-up within each period due to rolling enrollment or dropout. More than 2 periods with different durations can be assessed, and with sophisticated survival analysis methods, a period-free curve of placebo-controlled vaccine efficacy throughout follow-up can be derived (Supplement Figure 1) (28, 43, 44).
Table 2 evaluates the performance of a standard trial compared with a deferred vaccination trial in terms of power or the probability of detecting waning efficacy and the power of detecting harm in period 2, using the R package plaXdesign (45) (Supplement). We also report sample size ratios; for example, a sample size ratio of 2 means a deferred vaccination trial would require twice as many participants to achieve the power of a given standard trial. The first 4 rows of Table 2 correspond to a scenario with high initial efficacy that accrues about 200 COVID-19 cases among placebo recipients during period 1 and accrues 200 or 100 COVID-19 cases among placebo recipients during period 2 in a standard trial. Under deferred vaccination these period 2 placebo cases are inferred. We have good power to detect waning efficacy when vaccine efficacy changes from 90% to 75% with 200 cases in period 2 under either the deferred vaccination (0.96 power) or standard design (0.92 power). Power is lessened with 100 cases in period 2, with deferred vaccination having 0.72 power and standard design having 0.81 power.
Table 2. Statistical Performance of the Deferred Vaccination Design Compared With the Standard Design
The bottom 4 rows of Table 2 focus on a scenario where a subgroup may experience late harm from the vaccine. The subgroup accrues about 25 placebo group cases in period 1—thus, about one eighth of the total of 200 placebo cases. As before, we anticipate around 25 or 12 placebo group cases in period 2 under the standard trial. We consider subgroup vaccine efficacies in period 2 of −100% and −300% or a doubling and quadrupling of the case rate on vaccine compared with a placebo group, respectively. These estimates are meant to roughly parallel the vaccine-enhanced risk for hospitalized dengue disease and for severe dengue disease seen with the CYD-TDV dengue vaccine in baseline-seronegative participants (46). Power to detect waning efficacy is greater than 0.80 for all scenarios. The power to detect harm is substantially lower under deferred vaccination compared with the standard design, with poor power to detect harm with a period 2 vaccine efficacy of −100%, but powers of 0.71 and 0.84 to detect harm with a VE of −300% with 12 and 25 expected cases in period 2, respectively.
The approach can be generalized beyond 2 periods. For example, the vaccine efficacy estimate for 3 periods depends on all previous periods via the compounding formula 100% (1 − RR1 × RR2 × RR3), where RR k is the relative risk for the original study groups in period k. The compounding suggests a chain that is as strong as its weakest link. That is, if any period has an unreliable estimate of the relative risk, so will later periods. Conversely, if all periods have reliable estimates, later vaccine efficacy estimates will be reliable. This underscores the statistical benefit of maintaining the original placebo-controlled design for as long as possible to maximize the reliability of the first and thus subsequent estimates of vaccine efficacy. This behavior is illustrated in Supplement Figure 1.

Collateral Benefits of Placebo Crossover

Correlates of risk and correlates of protection analyses involve describing associations between the measured immune response shortly after the last dose of vaccine with subsequent risk for COVID-19 or with vaccine efficacy against COVID-19 (47–49). For example, neutralizing or binding antibodies have been identified as correlates of protection for many licensed vaccines (50, 51). If higher vaccine-induced levels of binding antibody to the spike protein or of SARS-CoV-2 neutralizing antibody responses are found to track with lower risk for disease (as might reasonably be hypothesized from, for example, retrospective analyses [52] and observational cohort studies [53] of natural infection), future vaccines that achieve high binding antibody or neutralizing antibody levels might be licensed on the basis of small-scale immunogenicity studies alone, thus precluding the need for large-scale efficacy trials. In fact, the U.S. Food and Drug Administration recently issued new guidance (54) that such an approach may be taken to support licensure of modified COVID-19 vaccines that have been tailored for enhanced efficacy against newly emerging VOCs. A proven or “reasonably likely” immune correlate of protection can also help inform assessments of durability and allow for immunobridging to populations not included in the trials either through traditional or provisional approval, such as through the U.S. Food and Drug Administration's accelerated approval mechanism (55, 56). After a strong positive vaccine efficacy result (for example, more than 80%), an immune correlates analysis (23) may be underpowered owing to relatively few breakthrough cases among the vaccine recipients. However, vaccination of the placebo group could effectively double the sample size for assessing immune correlates of risk and protection, a notable benefit of deferred vaccination.
If vaccine efficacy wanes substantially over the course of follow-up, a natural question will be whether revaccination can reverse the loss. Continued follow-up of the study participants provides a ready-to-go vehicle to experimentally evaluate revaccination by randomly assigning those who received vaccine to receive another course of vaccine or placebo. Although a boost trial can be quickly conducted in a standard trial that maintains follow-up, deferred vaccination allows for a doubling of those who are available for the boost trial relative to a trial that maintains a placebo control. A boost trial after deferred vaccination could proceed in 2 stages: The original vaccine group would be randomized first and the original placebo group randomized later, if needed (Supplement Figure 2). If the vaccine efficacy wanes from 80% to 40% and it is hoped that boosting will recover the vaccine efficacy to 80%, about 42 cases are required to achieve 90% power, substantially fewer than the roughly 150 cases required for a typical COVID-19 vaccine trial (3). The above discussion also applies to a heterologous boost or vaccination by a vaccine from the same platform but with an immunogen based on a newly emerged VOC. Another possibility is for the original vaccine group to be randomly assigned to receive a boost with the original vaccine versus a heterologous boost. The relative efficacy of the different boosting strategies could be directly compared. Furthermore, an inferred placebo-controlled vaccine efficacy for each boost could be derived by using the approach that we describe in this article, provided that the VOC was present in both periods.


Once the vaccine is available, principles of ethics (including an evaluation of the societal benefit and individual risk profile of the trial), transparency, and fairness should govern the timing and implementation of the deferred vaccination (22). Ideally, the deferred vaccination would be part of the original design and part of the initial informed consent. Otherwise, a protocol amendment could be introduced and reconsent obtained, though this could be challenging to design and execute amidst the considerable activity after a positive efficacy signal.
Deferred vaccination should be initiated at a second immunization visit where blinded participants arrive and are immunized. The immunizations could be open-label (where participants are unblinded and only the placebo group is immunized) or blinded (where the participants who received vaccine now receive placebo and vice versa). This blinded crossover has several scientific advantages. Just as the pre-crossover period was blinded to address initial vaccine efficacy, so should the post-crossover period be blinded to address durability. Both questions are critical and deserve the same rigor. With open-label immunization, there is concern that the newly unblinded original vaccine recipients will forgo masks and physical distancing while the newly unblinded placebo recipients who are immunized will not. Another concern is that mild subjective symptoms might be differentially dismissed or elevated in the unblinded study groups. This is particularly important because efficacy is entirely predicated on volitional presentation with signs or symptoms of COVID-19. Such differential behavior could confound the assessment of waning vaccine efficacy.
With minimal assessment of safety and reactogenicity in newly vaccinated persons, operationally maintaining the blind could essentially only require the addition of dummy shots in the original vaccine recipients with some additional blood draws. The ongoing Novavax phase 3 SARS-CoV-2 vaccine trial, which started enrollment in late December 2020, has adopted a blinded crossover design (57).
Nonetheless, open label follow-up is logistically easier, which is especially important if deferred vaccination was not part of the original design. In addition, differential reactogenicity might allow study participants to guess the vaccine group to which they were assigned, thus undermining the benefits of maintaining the blind. Open-label follow-up provides important information, though at risk for some bias. To best analyze such trials, baseline variables predictive of risk and simple questions about actual risk behavior can be used to statistically ameliorate bias (28, 43). The Pfizer and Moderna phase 3 trials, for which deferred vaccination was considered around the time an efficacy signal was reached, are open-label.
An important goal for COVID-19 vaccine trials is to assess the effect of vaccine on asymptomatic infection, which can be assessed by periodic serologies and nasal or nasopharyngeal sampling. To maintain this goal, serologic and upper respiratory samples should be collected in all study participants while still blinded at the second immunization visit, with both study groups having the same schedules post crossover to ensure even-handed evaluation of this end point. Although more complex, our methods can be generalized to study vaccine efficacy against infrequently assessed end points, such as seroconversion.
Different approaches to the second immunization visit could be implemented, with a key requirement that both study groups are treated the same. If the vaccine is initially recommended for a subgroup, such as higher-risk individuals, their second immunization visit could be scheduled, with the blinded placebo-controlled trial continuing for the lower-risk subgroup. Second immunization visits could later occur for the lower-risk subgroup. Operationally, crossover could follow the order of the original enrollment but occur at an accelerated pace compared with enrollment, so that different cohorts would cross over on different days relative to the initial vaccination. Another possibility is to randomize the time of crossover. To reduce trial burden, late follow-up might be lessened provided that critical end points are still assessed as in the pre-crossover period.
In conclusion, the high efficacies reported for multiple vaccine candidates (3–8), although universally welcomed, add complexity and uncertainty to the environment surrounding access to the vaccine for trial participants who were assigned to receive placebo. Continued blinded follow-up in the original study groups is optimal to assess vaccine efficacy over time and is endorsed by the U.S. Food and Drug Administration in their guidance pertaining to COVID-19 vaccine development. Deferred vaccination allows placebo recipients timely access to the vaccine when it would no longer be proper to maintain participants on placebo. We demonstrate that critical information regarding durability of the vaccine effect can be obtained even after the placebo group participants receive the vaccine and argue that studies should maintain rigorous blinded follow-up after deferred vaccination via a blinded crossover design. Additional benefits of continued follow-up include a doubling of the number of participants who can contribute to an immune correlate analysis and the possibility of a quick pivot to a trial of boosting should the vaccine demonstrate waning efficacy.

Supplemental Material

Supplement. Supplementary Material


Cohen MSCorey L. Combination prevention for COVID-19 [Editorial]. Science. 2020;368:551. [ 32381692]  doi: 10.1126/science.abc5798
Corey LMascola JRFauci ASet al. A strategic approach to COVID-19 vaccine R&D. Science. 2020;368:948-950. [ 32393526]  doi: 10.1126/science.abc5312
Baden LREl Sahly HMEssink Bet alCOVE Study Group. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403-416. [ 33378609]  doi: 10.1056/NEJMoa2035389
Polack FPThomas SJKitchin Net alC4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. N Engl J Med. 2020;383:2603-2615. [ 33301246]  doi: 10.1056/NEJMoa2034577
Voysey MClemens SACMadhi SAet alOxford COVID Vaccine Trial Group. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet. 2021;397:99-111. [ 33306989]  doi: 10.1016/S0140-6736(20)32661-1
Novavax. Novavax COVID-19 vaccine demonstrates 89.3% efficacy in UK phase 3 trial. 28 January 2021. Accessed at on 8 March 2021.
Janssen. Johnson & Johnson announces single-shot Janssen COVID-19 vaccine candidate met primary end points in interim analysis of its phase 3 ENSEMBLE trial. 29 January 2021. Accessed at on 5 April 2021.
McGill COVID19 Vaccine Tracker Team. COVID-19 Vaccine Tracker. 5 March 2021. Accessed at on 8 March 2021.
U.S. Food and Drug Administration. Development and Licensure of Vaccines to Prevent COVID-19: Guidance for Industry. June 2020. Accessed at on 17 November 2020.
Lee WSWheatley AKKent SJet al. Antibody-dependent enhancement and SARS-CoV-2 vaccines and therapies. Nat Microbiol. 2020;5:1185-1191. [ 32908214]  doi: 10.1038/s41564-020-00789-5
Haynes BFCorey LFernandes Pet al. Prospects for a safe COVID-19 vaccine. Sci Transl Med. 2020;12. [ 33077678]  doi: 10.1126/scitranslmed.abe0948
U.S. Food and Drug Administration. Emergency Use Authorization for Vaccines to Prevent COVID-19: Guidance for Industry. 2020. Accessed at on 17 November 2020.
Mehrotra DVJanes HEFleming TRet al. Clinical endpoints for evaluating efficacy in COVID-19 vaccine trials. Ann Intern Med. 2021;174:221-228. [ 33090877].  doi: 10.7326/M20-6169
Callow KAParry HFSergeant Met al. The time course of the immune response to experimental coronavirus infection of man. Epidemiol Infect. 1990;105:435-46. [ 2170159]
Huang ATGarcia-Carreras BHitchings MDTet al. A systematic review of antibody mediated immunity to coronaviruses: kinetics, correlates of protection, and association with severity. Nat Commun. 2020;11:4704. [ 32943637]  doi: 10.1038/s41467-020-18450-4
Poland GAOvsyannikova IGKennedy RB. SARS-CoV-2 immunity: review and applications to phase 3 vaccine candidates. Lancet. 2020;396:1595-1606. [ 33065034]  doi: 10.1016/S0140-6736(20)32137-1
Choe PGKim KHKang CKet al. Antibody responses 8 months after asymptomatic or mild SARS-CoV-2 infection. Emerg Infect Dis. 2021;27:928-931. [ 33350923]  doi: 10.3201/eid2703.204543
Ibarrondo FJFulcher JAGoodman-Meza Det al. Rapid decay of anti-SARS-CoV-2 antibodies in persons with mild covid-19 [Letter]. N Engl J Med. 2020;383:1085-1087. [ 32706954]  doi: 10.1056/NEJMc2025179
Long QXTang XJShi QLet al. Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections. Nat Med. 2020;26:1200-1204. [ 32555424]  doi: 10.1038/s41591-020-0965-6
Wang XGuo XXin Qet al. Neutralizing antibody responses to severe acute respiratory syndrome coronavirus 2 in coronavirus disease 2019 inpatients and convalescent patients. Clin Infect Dis. 2020;71:2688-2694. [ 32497196]  doi: 10.1093/cid/ciaa721
Krause PRFleming TRLongini IMet alWHO Ad Hoc Expert Group on the Next Steps for Covid-19 Vaccine Evaluation. Placebo-controlled trials of covid-19 vaccines — why we still need them. N Engl J Med. 2021;384:e2. [ 33264543]  doi: 10.1056/NEJMp2033538
Wendler DOchoa JMillum Jet al. COVID-19 vaccine trial ethics once we have efficacious vaccines. Science. 2020;370:1277-1279. [ 33273060]  doi: 10.1126/science.abf5084
USG COVID-19 Response Team/Coronavirus Prevention Network (CoVPN) Biostatistics Team. USG COVID-19 Response Team/CoVPN Vaccine Efficacy Trial Immune Correlates Statistical Analysis Plan. 15 March 2021. Accessed at on 15 March 2021.
Lipsitch M. Challenges of vaccine effectiveness and waning studies [Editorial]. Clin Infect Dis. 2019;68:1631-1633. [ 30204853]  doi: 10.1093/cid/ciy773
Durham LKLongini IM JrHalloran MEet al. Estimation of vaccine efficacy in the presence of waning: application to cholera vaccines. Am J Epidemiol. 1998;147:948-59. [ 9596473]
Kanaan MN, Farrington CP. Estimation of waning vaccine efficacy. J Am Stat Assoc. 2002;97:389-97.
Nason MFollmann D. Design and analysis of crossover trials for absorbing binary endpoints. Biometrics. 2010;66:958-65. [ 19930189]  doi: 10.1111/j.1541-0420.2009.01358.x
Fintzi J, Follmann D. Assessing vaccine durability in randomized trials following placebo crossover. arXiv. Preprint posted online 4 January 2021. arXiv:2101.01295v2
Tegally H, Wilkinson E, Giovanetti M, et al. Emergence and rapid spread of a new severe acute respiratory syndrome-related coronavirus 2 (SARS-CoV-2) lineage with multiple spike mutations in South Africa. Preprint posted online 22 December 2020. doi: 10.1101/2020.12.21.20248640
Voloch CMda Silva Francisco R Jrde Almeida LGet alCovid19-UFRJ Workgroup, LNCC Workgroup, Adriana Cony Cavalcanti. Genomic characterization of a novel SARS-CoV-2 lineage from Rio de Janeiro, Brazil [Letter]. J Virol. 2021. [ 33649194]  doi: 10.1128/JVI.00119-21
Wang P, Nair MS, Liu L, et al. Antibody resistance of SARS-CoV-2 variants B.1.351 and B.1.1.7. bioRxiv. Preprint posted online 12 February 2021. doi: 10.1101/2021.01.25.428137
Collier DA, De Marco A, Ferreira IA, et al. SARS-CoV-2 B.1.1.7 escape from mRNA vaccine-elicited neutralizing antibodies. medRxiv. Preprint posted online 15 February 2021. doi: 10.1101/2021.01.19.21249840
Choi BChoudhary MCRegan Jet al. Persistence and evolution of SARS-CoV-2 in an immunocompromised host [Letter]. N Engl J Med. 2020;383:2291-2293. [ 33176080]  doi: 10.1056/NEJMc2031364
Avanzato VAMatson MJSeifert SNet al. Case study: prolonged infectious SARS-CoV-2 shedding from an asymptomatic immunocompromised individual with cancer. Cell. 2020;183:1901-1912.e9. [ 33248470]  doi: 10.1016/j.cell.2020.10.049
Volz E, Mishra S, Chand M, et al. Transmission of SARS-CoV-2 lineage B.1.1.7 in England: insights from linking epidemiologic and genetic data. medRxiv. Preprint posted online 4 January 2021. doi: 10.1101/2020.12.30.20249034
Kennedy DARead AF. Monitor for COVID-19 vaccine resistance evolution during clinical trials. PLoS Biol. 2020;18:e3001000. [ 33166303]  doi: 10.1371/journal.pbio.3001000
Neafsey DEJuraska MBedford Tet al. Genetic diversity and protective efficacy of the RTS,S/AS01 malaria vaccine. N Engl J Med. 2015;373:2025-2037. [ 26488565]  doi: 10.1056/NEJMoa1505819
Rolland MEdlefsen PTLarsen BBet al. Increased HIV-1 vaccine efficacy against viruses with genetic signatures in Env V2. Nature. 2012;490:417-20. [ 22960785]  doi: 10.1038/nature11519
Juraska MMagaret CAShao Jet al. Viral genetic diversity and protective efficacy of a tetravalent dengue vaccine in two phase 3 trials. Proc Natl Acad Sci U S A. 2018;115:E8378-E8387. [ 30127007]  doi: 10.1073/pnas.1714250115
Rolland MGilbert PB. Sieve analysis to understand how SARS-CoV-2 diversity can impact vaccine protection. PLoS Pathog. 2021;17:e1009406. [ 33765086]  doi: 10.1371/journal.ppat.1009406
Zeger SLLiang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42:121-30. [ 3719049]
Zou G. A modified Poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004;159:702-6. [ 15033648]
Tsiatis AA, Davidian M. Estimating vaccine efficacy over time after a randomized study is unblinded. arXiv. Preprint posted online 24 February 2021. arXiv:2102.13103
Lin DYZeng DGilbert PB. Evaluating the long-term efficacy of COVID-19 vaccines. Clin Infect Dis. 2021. [ 33693529]  doi: 10.1093/cid/ciab226
Juraska M, Fay MP, Follmann D. plaXdesign: assessing vaccine durability and harm following placebo crossover in two-group vaccine efficacy trials. R Package. Accessed at on 9 December 2020.
Sridhar SLuedtke ALangevin Eet al. Effect of dengue serostatus on dengue vaccine safety and efficacy. N Engl J Med. 2018;379:327-340. [ 29897841]  doi: 10.1056/NEJMoa1800820
Qin LGilbert PBCorey Let al. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. 2007;196:1304-12. [ 17922394]
Plotkin SAGilbert PB. Nomenclature for immune correlates of protection after vaccination. Clin Infect Dis. 2012;54:1615-7. [ 22437237]  doi: 10.1093/cid/cis238
Follmann D. Augmented designs to assess immune response in vaccine trials. Biometrics. 2006;62:1161-9. [ 17156291]
Plotkin SA. Correlates of protection induced by vaccination. Clin Vaccine Immunol. 2010;17:1055-65. [ 20463105]  doi: 10.1128/CVI.00131-10
Plotkin S, Gilbert PB. Correlates of protection. In: Plotkin S, Orenstein W, Offit P, et al, eds. Plotkin's Vaccines. 7th ed. Elsevier; 2018:35-40.
Addetia ACrawford KHDDingens Aet al. Neutralizing antibodies correlate with protection from SARS-CoV-2 in humans during a fishery vessel outbreak with a high attack rate. J Clin Microbiol. 2020;58. [ 32826322]  doi: 10.1128/JCM.02107-20
Harvey RARassen JAKabelac CAet al. Real-world data suggest antibody positivity to SARS-CoV-2 is associated with a decreased risk of future infection. med. Rxiv. 2020. [ 33354682]  doi: 10.1101/2020.12.18.20248336
U.S. Department of Health and Human Services; U.S. Food and Drug Administration; Center for Biologics Evaluation and Research. Emergency use authorization for vaccines to prevent COVID-19: guidance for industry. Appendix 2: pp 17-21. 22 February 2021. Accessed at on 26 February 2021.
Fleming TRPowers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med. 2012;31:2973-84. [ 22711298]  doi: 10.1002/sim.5403
U.S. Food and Drug Administration. US Code of Federal Regulations FDA Subpart H. Accelerated approval of new drugs for serious or life-threatening illnesses. 21 CFR, Secs. 314.500–314.560. Accessed at on 4 December 2020.
Novavax. Current protocol for phase 3 clinical trial of NVX-CoV2373 in the U.S. and Mexico. 16 February 2021. Accessed at on 7 March 2021.


Sign In to Submit A Comment

Information & Authors


Published In

cover image Annals of Internal Medicine
Annals of Internal Medicine
Volume 174Number 8August 2021
Pages: 1118 - 1125


Published online: 13 April 2021
Published in issue: August 2021




    National Institute of Allergy and Infectious Diseases, Bethesda, Maryland (D.F., J.F., M.P.F., M.N.)
    National Institute of Allergy and Infectious Diseases, Bethesda, Maryland (D.F., J.F., M.P.F., M.N.)
    National Institute of Allergy and Infectious Diseases, Bethesda, Maryland (D.F., J.F., M.P.F., M.N.)
    Holly E. Janes, PhD
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Lindsey R. Baden, MD
    Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts (L.R.B.)
    Baylor College of Medicine, Houston, Texas (H.M.E.)
    Thomas R. Fleming, PhD
    University of Washington, Seattle, Washington (T.R.F., A.L., M.C.)
    Merck & Co., Inc., North Wales, Pennsylvania (D.V.M.)
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Rollins School of Public Health, Emory University, Atlanta, Georgia (D.B.)
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Youyi Fong, PhD
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Shu Han, PhD
    Moderna, Inc., Cambridge, Massachusetts (S.H., H.Z.)
    AstraZeneca, Cambridge, United Kingdom (I.H.)
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Ollivier Hyrien, PhD
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    University of Washington, Seattle, Washington (T.R.F., A.L., M.C.)
    Marco Carone, PhD
    University of Washington, Seattle, Washington (T.R.F., A.L., M.C.)
    National Institute of Allergy and Infectious Diseases, Bethesda, Maryland (D.F., J.F., M.P.F., M.N.)
    An Vandebosch, PhD
    Janssen Pharmaceuticals NV, Beerse, Belgium (A.V.)
    Honghong Zhou, PhD
    Moderna, Inc., Cambridge, Massachusetts (S.H., H.Z.)
    Iksung Cho, MS
    Novavax, Inc., Gaithersburg, Maryland (I.C.)
    Erin Gabriel, PhD
    Karolinska Institutet, Solna, Sweden (E.G.)
    James G. Kublin, MD
    Fred Hutchinson Cancer Research Center, Seattle, Washington (H.E.J., L.N.C., M.J., D.D., Y.F., Y.H., Y.H., O.H., J.G.K.)
    Myron S. Cohen, MD
    Institute for Global Health and Infectious Diseases, University of North Carolina, Chapel Hill, North Carolina (M.S.C.)
    Fred Hutchinson Cancer Research Center and University of Washington, Seattle, Washington (L.C., P.B.G.)
    Fred Hutchinson Cancer Research Center and University of Washington, Seattle, Washington (L.C., P.B.G.)
    Kathleen M. Neuzil, MD
    University of Maryland School of Medicine, Baltimore, Maryland (K.M.N.).
    Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
    Grant Support: By the National Institute of Allergy and Infectious Diseases, National Institutes of Health (UM1AI068635, R37AI054165, UM1AI068617, UM1Al148684, and R37-AI29168).
    Corresponding Author: Dean Follmann, PhD, Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases, National Institutes of Health, 5601 Fishers Lane, MSC 9820, Rockville, MD 20892-9820; e-mail, [email protected].
    Previous Posting: This manuscript was posted as a preprint on medRxiv on 14 December 2020. doi:10.1101/2020.12.14.20248137
    Current Author Addresses: Drs. Follmann, Fintzi, Fay, and Nason: Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases, 5601 Fishers Lane, MSC 9820, Bethesda, MD 20892.
    Drs. Janes, Carpp, Juraska, Donnell, Fong, Ying Huang, Yunda Huang, Hyrien, Carone, Kublin, Corey, and Gilbert: Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, Seattle, WA 98109.
    Dr. Baden: Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115.
    Dr. El Sahly: Baylor College of Medicine, 2015 Thomas Street, Houston, TX 77009.
    Dr. Fleming: Department of Biostatistics, University of Washington, 1705 NE Pacific Street, Seattle, WA 98195.
    Dr. Mehrotra: Biostatistics and Research Decision Sciences, Merck & Co., Inc., 351 North Sumneytown Pike, North Wales, PA 19454.
    Dr. Benkeser: Rollins School of Public Health, Emory University, 1518 Clifton Road, Atlanta, GA 30322.
    Drs. Han and Zhou: Moderna, Inc., 200 Technology Square, Cambridge, MA 02139.
    Dr. Hirsch: Biometrics, Late-stage Development, BioPharmaceuticals R&D, AstraZeneca, 1 Francis Crick Avenue, Cambridge CB2 0AA, United Kingdom.
    Dr. Luedtke: Department of Statistics, University of Washington, B313 Padelford Hall, Northeast Stevens Way, Seattle, WA 98195.
    Dr. Vandebosch: Janssen R&D, Janssen Pharmaceuticals NV, Turnhoutseweg 30, B-2340, Beerse, Belgium.
    Mr. Cho: Biostatistics, Novavax, Inc., 21 Firstfield Road, Gaithersburg, MD 20878.
    Dr. Gabriel: Medicinsk epidemiologi och biostatistik, Karolinska Institutet, Nobels väg 12A, Solna, 171 65, Sweden.
    Dr. Cohen: Institute for Global Health and Infectious Diseases, University of North Carolina, 130 Mason Farm Road, Second Floor, Chapel Hill, NC 27599.
    Dr. Neuzil: University of Maryland School of Medicine, 655 West Baltimore Street, Baltimore, MD 21201.
    Author Contributions: Conception and design: D. Follmann, L.R. Baden, T.R. Fleming, D. Benkeser, D. Donnell, Y. Fong, S. Han, H. Zhou, M.S. Cohen, L. Corey, P.B. Gilbert, K.M. Neuzil.
    Analysis and interpretation of the data: D. Follmann, H. Janes, L.R. Baden, M. Juraska, D. Donnell, O. Hyrien, J. Kublin, M.S. Cohen, L. Corey.
    Drafting of the article: D. Follmann, J. Fintzi, H. Janes, L.R. Baden, T.R. Fleming, L.N. Carpp, I. Hirsch, O. Hyrien, I. Cho, J. Kublin, K.M. Neuzil.
    Critical revision for important intellectual content: J. Fintzi, M.P. Fay, L.R. Baden, H. El Sahly, T.R. Fleming, D.V. Mehrotra, M. Juraska, D. Donnell, Y. Fong, Y. Huang, O. Hyrien, A. Luedtke, M. Carone, M.S. Cohen, L. Corey, P.B. Gilbert.
    Final approval of the article: D. Follmann, J. Fintzi, M.P. Fay, H. Janes, L.R. Baden, H. El Sahly, T.R. Fleming, D.V. Mehrotra, L.N. Carpp, M. Juraska, D. Benkeser, D. Donnell, Y. Fong, S. Han, I. Hirsch, Y. Huang, Y. Huang, O. Hyrien, M. Carone, A. Luedtke, M.C. Nason, A. Vandebosch, H. Zhou, I. Cho, E.E. Gabriel, J. Kublin, M.S. Cohen, L. Corey, P.B. Gilbert, K.M. Neuzil.
    Statistical expertise: D. Follmann, M.P. Fay, J. Fintzi, H. Janes, T.R. Fleming, D.V. Mehrotra, M. Juraska, Y. Fong, I. Hirsch, Y. Huang, Y. Huang, O. Hyrien, A. Luedtke, M. Carone, M.C. Nason, A. Vandebosch, I. Cho, E.E. Gabriel, P.B. Gilbert.
    Obtaining of funding: L. Corey, P.B. Gilbert.
    Administrative, technical, or logistic support: L.N. Carpp, L. Corey.
    Collection and assembly of data: D. Follmann.
    This article was published at on 13 April 2021.

    Metrics & Citations



    If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

    For more information or tips please see 'Downloading to a citation manager' in the Help menu.


    Download article citation data for:
    Dean Follmann, Jonathan Fintzi, Michael P. Fay, et al. A Deferred-Vaccination Design to Assess Durability of COVID-19 Vaccine Effect After the Placebo Group Is Vaccinated. Ann Intern Med.2021;174:1118-1125. [Epub 13 April 2021]. doi:10.7326/M20-8149

    View More

    Get Access

    Login Options:

    You will be redirected to to sign-in to Annals to complete your purchase.

    Create your Free Account

    You will be redirected to to create an account that will provide access to Annals.

    View options


    View PDF/ePub







    Copy the content Link

    Share on social media