Original Research9 January 2024

Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents

FREE
    Author, Article, and Disclosure Information

    Visual Abstract. Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents

    Using electronic health record data from a national collaboration of pediatric health systems, this study examined the effectiveness of BNT162b2 vaccines for preventing COVID-19 infections and severe disease among previously uninfected children and adolescents. It found moderate effectiveness of the vaccine against the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant.

    Abstract

    Background:

    The efficacy of the BNT162b2 vaccine in pediatrics was assessed by randomized trials before the Omicron variant’s emergence. The long-term durability of vaccine protection in this population during the Omicron period remains limited.

    Objective:

    To assess the effectiveness of BNT162b2 in preventing infection and severe diseases with various strains of the SARS-CoV-2 virus in previously uninfected children and adolescents.

    Design:

    Comparative effectiveness research accounting for underreported vaccination in 3 study cohorts: adolescents (12 to 20 years) during the Delta phase and children (5 to 11 years) and adolescents (12 to 20 years) during the Omicron phase.

    Setting:

    A national collaboration of pediatric health systems (PEDSnet).

    Participants:

    77 392 adolescents (45 007 vaccinated) during the Delta phase and 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) during the Omicron phase.

    Intervention:

    First dose of the BNT162b2 vaccine versus no receipt of COVID-19 vaccine.

    Measurements:

    Outcomes of interest include documented infection, COVID-19 illness severity, admission to an intensive care unit (ICU), and cardiac complications. The effectiveness was reported as (1-relative risk)*100, with confounders balanced via propensity score stratification.

    Results:

    During the Delta period, the estimated effectiveness of the BNT162b2 vaccine was 98.4% (95% CI, 98.1% to 98.7%) against documented infection among adolescents, with no statistically significant waning after receipt of the first dose. An analysis of cardiac complications did not suggest a statistically significant difference between vaccinated and unvaccinated groups. During the Omicron period, the effectiveness against documented infection among children was estimated to be 74.3% (CI, 72.2% to 76.2%). Higher levels of effectiveness were seen against moderate or severe COVID-19 (75.5% [CI, 69.0% to 81.0%]) and ICU admission with COVID-19 (84.9% [CI, 64.8% to 93.5%]). Among adolescents, the effectiveness against documented Omicron infection was 85.5% (CI, 83.8% to 87.1%), with 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The effectiveness of the BNT162b2 vaccine against the Omicron variant declined 4 months after the first dose and then stabilized. The analysis showed a lower risk for cardiac complications in the vaccinated group during the Omicron variant period.

    Limitation:

    Observational study design and potentially undocumented infection.

    Conclusion:

    This study suggests that BNT162b2 was effective for various COVID-19–related outcomes in children and adolescents during the Delta and Omicron periods, and there is some evidence of waning effectiveness over time.

    Primary Funding Source:

    National Institutes of Health.

    The U.S. Food and Drug Administration expanded the emergency use authorization of the BNT162b2 messenger RNA COVID-19 vaccine (Pfizer–BioNTech) to 12 to 15 year olds on 10 May 2021 and to 5 to 11 year olds on 29 October 2021. As of 5 April 2023, the Centers for Disease Control and Prevention reports indicate that 40% of U.S. children aged 5 to 11 years and 72% of adolescents aged 12 to 18 years had received at least 1 dose of the vaccine. The emergence of the Omicron variant (B.1.1.529) and its subvariants in early 2022 led to a new surge in COVID-19 cases worldwide (1). The randomized trials of the BNT162b2 vaccine, which demonstrated high efficacy of 2 doses against COVID-19 (100% and 91% among those aged 12 to 15 years and 5 to 11 years, respectively), were done before the emergence of the Omicron variant (2, 3).

    Several observational studies have been done to investigate the effectiveness of vaccination in real-world settings (4–8). However, prior studies have had limited follow-up periods, covering the Delta variant or earlier subvariants of Omicron periods only. Studies evaluating the Omicron variant have only assessed the short-term effects of the vaccine, with only 1 study involving children evaluating the effect beyond 3 months (9). There is limited information on the long-term durability of vaccine protection during the Omicron period. Few existing studies on U.S. pediatric populations have covered both hospitalized patients and those with mild or asymptomatic conditions. Furthermore, although studies have acknowledged limitations due to misclassification in vaccination status in real-world effectiveness studies, none have rigorously evaluated the effects of such misclassification nor accounted for the potential bias it may introduce.

    To address these gaps in our knowledge of the pediatric effectiveness of SARS-CoV-2 vaccination, we designed this study to assess the real-world effectiveness of BNT162b2 among children and adolescents during the periods when the Delta and Omicron variants were predominant using electronic health record (EHR) data from a national network of U.S. pediatric medical centers. Our study used a novel comparative effectiveness research (CER) method and adjusted for underreporting issues in vaccination status and has several attractive features that strengthen the generalizability and reliability of our inference. First, it is the largest study to date in the United States estimating vaccine effectiveness in children and adolescents, covering a broad spectrum of the U.S. pediatric population. Second, the study examined the effectiveness against infection over a longer follow-up than any previous study, enabling evaluation of the durability of vaccine protection. Third, the study included a diverse representation of U.S. pediatric populations from primary care, specialty care, emergency department (ED), testing centers, and inpatient settings. Fourth, the study was the first to account for the incomplete capture of vaccination status by health systems in the United States. Finally, in addition to infection and severe diseases end points, we also studied the effect of vaccination on the incidence of myocarditis, pericarditis, or multisystem inflammatory syndrome (MIS) to assess the effect of vaccination relative to the potential risk for cardiac complications.

    Methods

    Data Sources

    This study used EHR data from PEDSnet (10), which is a national collaboration of pediatric health systems that share EHR data, conduct research, and improve outcomes together. Participating institutions in this study included the Children’s Hospital of Philadelphia, Cincinnati Children’s Hospital Medical Center, Children’s Hospital Colorado, Ann & Robert H. Lurie Children’s Hospital of Chicago, Nationwide Children’s Hospital, Nemours Children’s Health System (inclusive of the Delaware and Florida health system), Seattle Children’s Hospital, and Stanford Children’s Health. Data were extracted from the PEDSnet COVID-19 Database Version Week 141 (11). A detailed description of EHR data is available in section 1 of the Supplement.

    Specification of Hypothetical Trials and CER Studies

    Hypothetical randomized controlled trials were specified to guide the design of observational studies to assess the real-world effectiveness of treatments (12). We designed and conducted CER studies to investigate the effectiveness of the BNT162b2 vaccine in preventing infection with various strains of the SARS-CoV-2 virus in children and adolescents in the United States. The 3 study cohorts focused on documented SARS-CoV-2 infection and outcomes: study cohort 1 (Delta study in adolescents): adolescents aged 12 to 20 years during the period when the Delta variant was prevalent from 1 July to 30 November 2021; study cohort 2 (Omicron study in children): children aged 5 to 11 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022; and study cohort 3 (Omicron study in adolescents): adolescents aged 12 to 20 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022.

    The design of hypothetical trials and implementation procedures in real-world data are summarized in Supplement Table 1.

    Eligibility criteria included age of 5 to 11 years for children or 12 to 20 years for adolescents at the start of the study period and no previous COVID-19 vaccination or documented SARS-CoV-2 infection. In addition, participants were required to have a prior encounter (including telephone or telehealth encounters) within 18 months of cohort entrance to ensure that they had an ongoing interaction with the health system.

    The intervention of interest was vaccination, in comparison with no receipt of any type of COVID-19 vaccine. Given that the BNT162b2 vaccine covered more than 85% of documented vaccinations among children and adolescents in the PEDSnet database, in this study we focused primarily on studying the effectiveness of this vaccine, although the Supplement reports a sensitivity analysis investigating all types of reported COVID-19 vaccines assessed in the United States, with 85.7% BNT162b2, 1.9% mRNA-1273 (Moderna), and 12.3% unspecified COVID-19 vaccine.

    In this CER study using real-world data, the cohort entrance date for the intervention group was defined as the date of the first dose of the BNT162b2 vaccine, whereas for the comparator group, it was a randomly selected date from visits, chosen to ensure that the distribution of index dates for the control group matched the distribution of index dates for the vaccination group to control for time effects. The risk period for the study began 28 days after the index date, such that infections within 28 days were excluded.

    Randomized trials achieve balance across potential confounders by randomly allocating the treatment to intervention and comparator groups. In our study, we attempted to balance the intervention and comparator groups by adjusting for a large number of measured confounders collected before cohort entry using propensity score stratification (13). We built the propensity score model on the basis of demographic factors, including age, sex, race, and ethnicity; clinical factors, including obesity status, a baseline chronic condition indicator as defined by the Pediatric Medical Complexity Algorithm (14), and a list of preexisting chronic conditions; and health care utilization factors collected before the cohort entry, including the number of inpatients, outpatients, ED visits, unique mediations, and negative COVID-19 tests. We stratified the patients into propensity score quintiles on the basis of these factors. See Supplement Table 3 for detailed definitions of study variables.

    The 4 COVID-19 outcomes of interest were documented SARS-CoV-2 infection, mild COVID-19, moderate or severe COVID-19, and ICU admission with COVID-19. We did not evaluate death from COVID-19 because it was too rare among children and adolescents to study quantitatively. In our study, SARS-CoV-2 infections were defined by positive polymerase chain reaction, serology, or antigen tests or diagnoses of COVID-19 or by occurrence of postacute sequelae of SARS-CoV-2 or MIS in children regardless of the presence of symptoms. Classification of mild, moderate, or severe COVID-19 illness was defined on the basis of the symptoms and health conditions diagnosed from 7 days before to 13 days after the date of a documented COVID-19 infection, as in Forrest and colleagues study (15). Admission to an ICU with COVID-19 was defined by any ICU visit 7 days before and 13 days after documented SARS-CoV-2 infection. In addition, we considered the clinical outcomes of cardiac complications identified as incidence of myocarditis, pericarditis, or MIS to allow for a comprehensive capture of potential cardiac complications after infection and evaluate the effect of the BNT162b2 vaccine in terms of cardiac risks.

    Statistical Analysis

    We evaluated covariate balancing after propensity score stratification by plotting the standardized mean differences (SMDs) between variable values for the vaccinated and unvaccinated groups, with a difference of 0.1 or less indicating an acceptable balance. We used Poisson regression to estimate the relative risk between 2 treatment groups for the risk for each outcome while adjusting for different follow-up lengths among participants. Because immunization records are often captured and stored across multiple disconnected sources, resulting in underreported vaccination records in patients’ EHRs, we mitigated the potential bias arising from this underreporting issue by incorporating an integration likelihood (16, 17) of the Poisson regression with a prespecified range of misclassification rates. The vaccine effectiveness was defined as (1-relative risk)*100. The details of statistical methods are described in section 2 of the Supplement.

    We conducted secondary analyses stratified by 2-month intervals since receipt of vaccination to investigate the durability of vaccine protection. Subgroup analyses were also done to investigate differences in vaccine protection according to age groups (5 to 8, 9 to 11, 12 to 15, and 16 to 20 years).

    Sensitivity Analyses

    Extensive sensitivity analyses were done to evaluate the robustness of the research findings; see sections 6 to 15 and section 19 of the Supplement for the effects of the cohort design. In scenarios in which any categorical covariates were unbalanced (with an SMD >0.1), we included a sensitivity analysis excluding participants in that category. A sensitivity analysis defining the risk period with no waiting period after the index date was done. To assess whether population heterogeneity, established by applying eligibility criteria that required a prior encounter within 18 months of cohort entry, could influence our vaccine effectiveness estimates, we conducted sensitivity analyses within a more restricted time window. We conducted a dose-response study of BNT162b2 by defining a single-dose vaccination as 1 dose 28 days before infection and a 2-dose vaccination as the receipt of a second dose 7 days before infection. A sensitivity analysis was done to compare the proposed method to a sequential target trial emulation pipeline not accounting for underreporting issues to assess the robustness of findings (18). Because the proportion of patients entered with ED visits is relatively low in the vaccinated cohort compared with the unvaccinated, we conducted a sensitivity analysis excluding all participants who entered the cohort during an ED visit. Residual study bias from unmeasured and systematic sources can still exist in observational studies after controlling for measured confounders; thus, we conducted negative control outcome experiments (13, 19, 20) in which the null hypothesis of no effect was believed to be true using 40 negative control outcomes prespecified by pediatric physicians. The empirical null distribution and calibrated effectiveness were reported as sensitivity analyses. The relative risks for cardiac complications defined by myocarditis or pericarditis (excluding MIS) were estimated. We also reported the estimated vaccine effectiveness from all brands of COVID-19 vaccines. Given the prolonged presence of the Omicron variant and the emergence of several subvariants, we conducted a secondary analysis to assess the vaccine effectiveness related to subvariants of Omicron.

    Missingness in Vaccine Records

    Vaccine status may be missing for persons whose vaccine doses were administered by a site outside of the PEDSnet network care delivery sites. It is likely that patients recorded as vaccinated in the EHR are true positives, so specificity could be very high, but sensitivity would be reduced by undocumented vaccinations (false negatives). To account for potential bias from the underreporting issue in vaccination status, a range of possible sensitivities based on our prior study was prespecified for each study. The sensitivity range was considered to be 0.8 to 1 for the study involving children and 0.7 to 0.9 for the studies involving adolescents. By accounting for the underreporting and specifying a range of sensitivity, the study aimed to minimize the effect of bias caused by the underreporting in the estimation of the effectiveness of the COVID-19 vaccine among children and adolescents. The details of statistical methods are described in section 2 of the Supplement.

    To further evaluate the robustness of statistical methods we used to account for underreporting, we varied our CER method by considering alternative methods for bias correction, including the naive method (without adjusting for underreporting), using different ranges of misclassification rates, and using a fully Bayesian method (21). To evaluate the effect of differential misclassification on effectiveness estimates, we conducted sensitivity analyses simulating vaccination status according to various differential misclassification scenarios. Results from these sensitivity analyses are summarized in sections 16 to 18 of the Supplement.

    Role of the Funding Source

    The funders had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; approval of the manuscript; or the decision to submit the manuscript for publication.

    Results

    Study Population

    Figure 1 shows the process of selection of 3 study cohorts. A total of 77 392 adolescents (45 007 vaccinated) within the PEDSnet network were identified to study the effectiveness of vaccination against infection with the Delta variant and severe outcomes (see Table 1 for baseline characteristics). A total of 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) were included in the cohort to study the effectiveness of vaccination against infection with the Omicron variant (see Table 2 for baseline characteristics). The vaccinated and unvaccinated groups had a slightly unbalanced distribution of testing rates before cohort entry across all 3 cohorts. After propensity score stratification, all covariates were well balanced between vaccinated and unvaccinated groups with an SMD smaller than 0.1 in the Omicron study involving children (Supplement Figure 12) and involving adolescents (Supplement Figure 13). In the study evaluating vaccine effectiveness for adolescents during the Delta period, 1 site remained unbalanced after propensity score stratification with an SMD larger than 0.1, and thus a sensitivity analysis was done by excluding participants from this site, which gave consistent results with the primary analysis (see Supplement Figure 11 and section 6 in the Supplement). Additional characteristics of the study cohorts, including additional medical conditions, deaths, and follow-up durations, are summarized in section 5 of the Supplement.

    Figure 1. Selection of participants for the 3 study cohorts evaluating the effectiveness of the BNT162b2 vaccine in preventing infection and severe diseases with SARS-CoV-2 in adolescents aged 12 to 20 years during the period when the Delta variant was prevalent (study cohort 1) and children aged 5 to 11 years and adolescents aged 12 to 20 years during the period when the Omicron variant was prevalent (study cohorts 2 and 3, respectively).

    Table 1. Baseline Characteristics of Adolescents Aged 12 to 20 Years in the Study of the Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 During the Period When the Delta Variant Was Prevalent

    CharacteristicVaccinated (n = 45 007)Unvaccinated (n = 32 385)Overall (n = 77 392)
    Median age (Q1–Q3), y14 (13–16)15 (13–17)15 (13–16)
    Distribution, n (%)
     12 y8922 (19.8)4926 (15.2)13 848 (17.9)
     13 y8099 (18.0)4864 (15.0)12 963 (16.7)
     14 y8037 (17.9)4923 (15.2)12 960 (16.7)
     15 y7311 (16.2)4749 (14.7)12 060 (15.6)
     16 y5450 (12.1)4546 (14.0)9996 (12.9)
     17 y4076 (9.1)3991 (12.3)8067 (10.4)
     18 y1674 (3.7)2075 (6.4)3749 (4.8)
     19 y942 (2.1)1461 (4.5)2403 (3.1)
     20 y496 (1.1)850 (2.6)1346 (1.7)
    Sex, n (%)
     Female23 589 (52.4)16 500 (50.9)40 089 (51.8)
     Male21 416 (47.6)15 880 (49.0)37 296 (48.2)
    Ethnicity, n (%)
     White16 446 (50.8)17 964 (39.9)34 410 (44.5)
     Black/African American6019 (18.6)12 012 (26.7)18 031 (23.3)
     Hispanic4925 (15.2)9629 (21.4)14 554 (18.8)
     Other/unknown4995 (15.4)5402 (12.0)10 397 (13.4)
    Hospital, n (%)
     A5424 (12.1)7385 (22.8)12 809 (16.6)
     B12 884 (28.6)5216 (16.1)18 100 (23.4)
     C6333 (14.1)3457 (10.7)9790 (12.6)
     D1723 (3.8)914 (2.8)2637 (3.4)
     E4369 (9.7)6063 (18.7)10 432 (13.5)
     F12 831 (28.5)3409 (10.5)16 240 (21.0)
     G1430 (3.2)1457 (4.5)2887 (3.7)
     H13 (0.0)4484 (13.8)4497 (5.8)
    Entry time, n (%)
     July–September 202138 335 (85.2)24 509 (75.7)62 844 (81.2)
     October–November 20216672 (14.8)7876 (24.3)14 548 (18.8)
    Obesity, n (%)
     028 029 (62.3)22 479 (69.4)50 508 (65.3)
     116 978 (37.7)9906 (30.6)26 884 (34.7)
    Pediatric Medical Complexity Algorithm, n (%)
     025 634 (57.0)19 916 (61.5)45 550 (58.9)
     110 915 (24.3)6417 (19.8)17 332 (22.4)
     28458 (18.8)6052 (18.7)14 510 (18.7)
    Negative tests before entry, n (%)
     01330 (3.0)2888 (8.9)4218 (5.5)
     134 272 (76.1)16 299 (50.3)50 571 (65.3)
     27388 (16.4)9739 (30.1)17 127 (22.1)
     ≥32017 (4.5)3459 (10.7)5476 (7.1)

    Q = quartile.

    Table 2. Baseline Characteristics of Children Aged 5 to 11 Years and Adolescents Aged 12 to 20 Years in the Study of the Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 During Periods When the Omicron Variant Was Prevalent

    CharacteristicOmicron Study in Children
    Omicron Study in Adolescents
    Vaccinated (n = 50 398)Unvaccinated (n = 61 141)Overall (n = 111 539)Vaccinated (n = 21 180)Unvaccinated (n = 34 900)Overall (n = 56 080)
    Median age (Q1–Q3), y8 (6–10)7 (6–9)8 (6–10)14 (13–16)15 (13–17)15 (13–17)
    Distribution, n (%)
     5 y8165 (16.2)13 321 (21.8)21 486 (19.3)
     6 y7447 (14.8)11 314 (18.5)18 761 (16.8)
     7 y7090 (14.1)9151 (15.0)16 241 (14.6)
     8 y7028 (13.9)7922 (13.0)14 950 (13.4)
     9 y6773 (13.4)7085 (11.6)13 858 (12.4)
     10 y7011 (13.9)6434 (10.5)13 445 (12.1)
     11 y6884 (13.7)5914 (9.7)12 798 (11.5)
     12 y4754 (22.4)5760 (16.5)10 514 (18.7)
     13 y3421 (16.2)5520 (15.8)8941 (15.9)
     14 y3338 (15.8)5299 (15.2)8637 (15.4)
     15 y3123 (14.7)5315 (15.2)8438 (15.0)
     16 y2634 (12.4)4944 (14.2)7578 (13.5)
     17 y2201 (10.4)3836 (11.0)6037 (10.8)
     18 y986 (4.7)2114 (6.1)3100 (5.5)
     19 y475 (2.2)1400 (4.0)1875 (3.3)
     20 y248 (1.2)712 (2.0)960 (1.7)
    Sex, n (%)
     Female23 962 (47.5)28 669 (46.9)52 631 (47.2)11 402 (53.8)17 954 (51.4)29 356 (52.3)
     Male26 436 (52.5)32 468 (53.1)58 904 (52.8)9775 (46.2)16 939 (48.5)26 714 (47.6)
    Ethnicity, n (%)
     White14 399 (28.6)24 644 (40.3)39 043 (35.0)16 240 (46.5)6836 (32.3)23 076 (41.1)
     Black/African American13 711 (27.2)13 733 (22.5)27 444 (24.6)6154 (17.6)6157 (29.1)12 311 (22.0)
     Hispanic12 119 (24.0)12 781 (20.9)24 900 (22.3)6287 (18.0)3784 (17.9)10 071 (18.0)
     Other/unknown10 169 (20.2)9983 (16.3)20 152 (18.1)6219 (17.8)4403 (20.8)10 622 (18.9)
    Hospital, n (%)
     A5019 (10.0)9266 (15.2)14 285 (12.8)2131 (10.1)5183 (14.9)7314 (13.0)
     B15 229 (30.2)13 168 (21.5)28 397 (25.5)6397 (30.2)6556 (18.8)12 953 (23.1)
     C5482 (10.9)7409 (12.1)12 891 (11.6)1719 (8.1)4075 (11.7)5794 (10.3)
     D4766 (9.5)2878 (4.7)7644 (6.9)678 (3.2)1337 (3.8)2015 (3.6)
     E5843 (11.6)10 551 (17.3)16 394 (14.7)2047 (9.7)4263 (12.2)6310 (11.3)
     F9786 (19.4)11 348 (18.6)21 134 (18.9)3563 (16.8)5186 (14.9)8749 (15.6)
     G1250 (2.5)3239 (5.3)4489 (4.0)622 (2.9)2353 (6.7)2975 (5.3)
     H3023 (6.0)3282 (5.4)6305 (5.7)4023 (19.0)5947 (17.0)9970 (17.8)
    Entry time, n (%)
     January–March 202237 970 (75.3)32 523 (53.2)70 493 (63.2)14 684 (69.3)19 032 (54.5)33 716 (60.1)
     April–June 20225882 (11.7)11 919 (19.5)17 801 (16.0)3344 (15.8)7087 (20.3)10 431 (18.6)
     July–September 20224994 (9.9)10 329 (16.9)15 323 (13.7)2206 (10.4)5479 (15.7)7685 (13.7)
     October–November 20221552 (3.1)6370 (10.4)7922 (7.1)946 (4.5)3302 (9.5)4248 (7.6)
    Obesity, n (%)
     033 381 (66.2)42 165 (69.0)75 546 (67.7)13 832 (65.3)23 895 (68.5)37 727 (67.3)
     117 017 (33.8)18 976 (31.0)35 993 (32.3)7348 (34.7)11 005 (31.5)18 353 (32.7)
    Pediatric Medical Complexity Algorithm, n (%)
     033 870 (67.2)40 976 (67.0)74 846 (67.1)13 482 (63.7%)21 079 (60.4)34 561 (61.6)
     110 000 (19.8)11 189 (18.3)21 189 (19.0)4382 (20.7%)6764 (19.4)11 146 (19.9)
     26528 (13.0)8976 (14.7)15 504 (13.9)3316 (15.7%)7057 (20.2)10 373 (18.5)
    Negative tests before entry, n (%)
     02337 (4.6)5640 (9.2)7977 (7.2)768 (3.6)2966 (8.5)3734 (6.7)
     134 077 (67.6)28 417 (46.5)62 494 (56.0)15 707 (74.2)17 303 (49.6)33 010 (58.9)
     210 514 (20.9)19 816 (32.4)30 330 (27.2)3654 (17.3)11 012 (31.6)14 666 (26.2)
     ≥33470 (6.9)7268 (11.9)10 738 (9.6)1051 (5.0)3619 (10.4)4670 (8.3)

    Q = quartile.

    Vaccine Effectiveness

    Table 3 summarizes the estimated vaccine effectiveness in 3 study cohorts, and Figure 2 shows the durability of protection. The vaccine effectiveness was estimated to be 98.4% (95% CI, 98.1% to 98.7%) among adolescents in the Delta period, 74.3% (CI, 72.2% to 76.2%) against documented infection among children in the Omicron period, and 85.5% (CI, 83.8% to 87.1%) among adolescents in the Omicron period. During the Delta period, the vaccine effectiveness against documented infection remained stable throughout the follow-up period of the study. Four months after the first dose, vaccine effectiveness against documented infection with Omicron declined from 82.3% (CI, 77.9% to 85.8%) to 70.6% (CI, 65.9% to 74.6%) among children, and from 91.3% (CI, 87.6% to 94.0%) to 82.9% (CI, 79.0% to 86.1%) among adolescents. Although vaccine effectiveness against documented infection stabilized after this initial decline, the corresponding CIs were much wider, indicating higher levels of uncertainty.

    Figure 2. Stratified effectiveness of the BNT162b2 vaccine in preventing infection with SARS-CoV-2 in children and adolescents by 2-month intervals since receipt of the first dose.

    Table 3. Estimated Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 in Children and Adolescents

    Epidemiologic MeasureVaccinatedUnvaccinatedOverallVaccine Effectiveness (95% CI), %
    Delta study in adolescents
     Follow-up
      Total follow-up, n of person-weeks644 162398 9061 043 068
      Median (Q1–Q3)16 (12–18)13 (9–17)15 (10–18)
     Absolute risk, %
      Documented infection0.355.262.4198.4 (98.1–98.7)
      Mild COVID-190.061.430.6399.0 (98.5–99.3)
      Moderate or severe COVID-190.030.490.2298.7 (97.4–99.3)
      ICU admission with COVID-19≤0.010.05≤0.0399.0 (92.5–99.9)
      Cardiac complication0.020.030.021.22 (0.34–4.35)*
     Age 12–15 y
      Total follow-up, n of person-weeks458 981229 083688 064
      Documented infection, %0.345.522.2899.0 (98.6–99.3)
     Age 16–21 y
      Total follow-up, n of person-weeks185 181169 823355 003
      Documented infection, %0.364.912.6497.0 (95.9–97.8)
    Omicron study in children
     Follow-up
      Total follow-up, n of person-weeks1 925 6861 911 5993 837 285
      Median (Q1–Q3)44 (35–46)36 (19–44)40 (25–45)
     Absolute risk, %
      Documented infection1.895.463.8574.3 (72.2–76.2)
      Mild COVID-190.541.551.0973.5 (69.2–77.1)
      Moderate or severe COVID-190.190.670.4575.5 (69.0–81.0)
      ICU admission with COVID-190.020.080.0584.9 (64.8–93.5)
      Cardiac complication≤0.010.03≤0.020.28 (0.08–0.95)*
     Age 5–8 y
      Total follow-up, n of person-weeks1 101 4181 254 8192 356 236
      Documented infection, %1.965.103.7871.3 (68.2–74.1)
     Age 9–11 y
      Total follow-up, n of person-weeks824 268656 7801 481 049
      Documented infection, %1.806.113.9577.9 (75.1–80.4)
    Omicron study in adolescents
     Follow-up
      Total follow-up, n of person-weeks772 1761 113 5611 885 736
      Median (Q1–Q3)42 (30–45)37 (22–44)39 (25–45)
     Absolute risk, %
      Documented infection1.828.175.7785.5 (83.8–87.1)
      Mild COVID-190.432.071.4587.0 (83.5–89.8)
      Moderate or severe COVID-190.150.880.6084.8 (77.3–89.9)
      ICU admission with COVID-19≤0.020.14≤0.1091.5 (69.5–97.6)
      Cardiac complication≤0.020.05≤0.040.10 (0.02–0.57)*
     Age 12–15 y
      Total follow-up, n of person-weeks524 053654 3151 178 368
      Documented infection, %1.938.245.6785.8 (83.6–87.7)
     Age 16–21 y
      Total follow-up, n of person-weeks248 123459 246707 368
      Documented infection, %1.608.075.9385.9 (82.7–88.5)

    ICU = intensive care unit; Q = quartile.

    * Relative risk in the vaccinated groups compared with the unvaccinated.

    Vaccine Effectiveness: Severe Illness and Complications

    During the Delta period, the vaccine was found to have high effectiveness against severe infections. The estimated relative risk of the vaccine on cardiac complications was 1.22 (CI, 0.34 to 4.35). The estimated vaccine effectiveness against the Omicron variant in children was 73.5% (CI, 69.2% to 77.1%) against mild COVID-19, 75.5% (CI, 69.0% to 81.0%) against moderate or severe COVID-19, and 84.9% (CI, 64.8% to 93.5%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.28 (CI, 0.08 to 0.95). In the Omicron study in adolescents, the vaccine effectiveness was estimated to be 87.0% (CI, 83.5% to 89.8%) against mild COVID-19, 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.10 (CI, 0.02 to 0.57) in this cohort.

    Sensitivity Analyses and Addressing Misclassification Bias

    Section 9 of the Supplement presents the vaccine effectiveness against various Omicron subvariants. The effectiveness against subvariants BA.1, BA.2, BA.4, and BA.5 aligns with our primary findings, whereas the vaccine effectiveness seems to be lower against the BQ.1, XBB, and subsequent subvariants. Section 11 of the Supplement presents the results for the dose-response analysis. Section 14 of the Supplement presents negative control experiments of 3 study cohorts using 40 negative control outcomes. After accounting for systematic error through calibration using negative control outcomes, our findings indicate a slight shift in point estimates accompanied by wider CIs. This suggests the presence of a minor degree of systematic error, as well as additional uncertainty characterized by the estimated distribution derived from the negative control outcomes. Section 19 of the Supplement summarizes the comparative results to a sequential target trial emulation not accounting for underreporting issues of vaccination, which indicated reasonably consistent findings.

    Section 16 of the Supplement shows effectiveness estimated from the naive method and proposed CER method with different ranges of sensitivity of vaccination status captured by EHR. The comparison results indicated that the vaccine effectiveness was reasonably consistent across different sensitivity ranges, suggesting that our primary analysis was robust to changes in the range of sensitivity considered. Section 17 of the Supplement shows the comparative results to a fully Bayesian method indicating nearly identical results. Section 18 of the Supplement shows sensitivity analyses on differential misclassifications, which shows that the novel CER method corrects the bias even when the nondifferential misclassification assumption does not hold.

    Discussion

    We estimated the effectiveness of BNT162b2 vaccines for the prevention of documented SARS-CoV-2 infections and severe diseases in a national network of pediatric health systems in the United States for 3 study cohorts. During the period when the Delta variant was dominant, the BNT162b2 vaccine was associated with strong protection in adolescents, with effectiveness higher than 95%, and with little evidence of waning during the follow-up period. Our findings against the Delta infection among adolescents are consistent with vaccine efficacy seen in the BNT162b2 clinical trial involving adolescents aged between 12 and 15 years, which demonstrated vaccine efficacy of 100% (CI, 75.3% to 100%) against documented SARS-CoV-2 infection (2). Our estimates of vaccine effectiveness against severe diseases are consistent with a case–control study based on test-negative design, which found an effectiveness of 94% (CI, 90% to 96%) against hospitalization and 98% (CI, 93% to 99%) against ICU admission (7).

    In our study, during the period when the Omicron variant was dominant, the estimated vaccine effectiveness was approximately 70% for children and 85% for adolescents. The estimated protection decreased by roughly 10% around 4 months from the first dose and slightly waned over time. Previous studies have shown vaccine effectiveness against Omicron infection, ranging from 20% to 80% among children and adolescents. An analysis using a test-negative design in Scotland during the Omicron period found a vaccine effectiveness of 81.2% (CI, 77.7% to 84.2%) for adolescents aged 12 to 15 years and 65.5% (CI, 56.0% to 73.0%) for those aged 16 to 17 years 2 to 5 weeks after full vaccination. This effectiveness decreased to 43.3% (CI, 30.0% to 54.2%) and 8.9% (CI, −19.1% to 30.3%) after 10 to 13 weeks, respectively (22). With a test-negative design, data from U.S. pharmacy-based, drive-through SARS-CoV-2 testing sites indicated that the estimated vaccine effectiveness for children aged 5 to 11 years was 60.1% within 2 to 4 weeks after the second dose and declined to 28.9% during the second month after vaccination. For adolescents aged 12 to 15 years, the effectiveness was 59.5% within 2 to 4 weeks after the second dose and decreased to 16.6% during the subsequent month (23). Another test-negative design analysis in the United States indicated that in children aged 5 to 11 years, the effectiveness was 59.9% (CI, 58.5% to 61.2%) at 1 month, 33.7% (CI, 32.6% to 34.8%) at 4 months, and 14.9% (CI, 12.3% to 17.5%) at 10 months after the first dose (24). A retrospective study among children aged 5 to 11 years in Italy found vaccine effectiveness to be 29.4% (CI, 28.5% to 30.2%) against documented infection and 41.1% (CI, 22.2% to 55.4%) against severe COVID-19 (8). A Singapore study found that in fully vaccinated children aged 5 to 11 years, vaccine effectiveness was 65.3% (CI, 62.0% to 68.3%) against polymerase chain reaction–confirmed SARS-CoV-2 infection and 82.7% (CI, 74.8% to 88.2%) against COVID-19–related hospitalization (23). In addition, we found that vaccine effectiveness is substantially lower against the later subvariants of Omicron, including BQ.1 and XBB. However, it remains unclear whether this is a true subvariant effect or evidence of further waning over time. Continued research is desirable to understand vaccine effectiveness on future subvariants and its potential waning effects.

    This study did not identify a statistically significant elevated risk for cardiac complications among vaccinated adolescents during the Delta variant period, and it even demonstrated a lower risk in the vaccinated group during the Omicron variant period. This finding may seem unexpected given that cases of myocarditis and pericarditis after messenger RNA COVID-19 vaccines received substantial attention (26–28). However, it is essential to note that previous studies indicated a much higher risk for myocarditis or pericarditis after a documented SARS-CoV-2 infection in the pediatric population (29), with an article finding 36.8 times higher risk (CI, 25.0 to 48.6) in persons younger than 16 years after COVID-19 (30), and that myocarditis is a common symptom for patients diagnosed with MIS (31, 32). It is possible that cases of myocarditis, pericarditis, or MIS occurred after undocumented SARS-CoV-2 infections. For example, 31.5% of occurrences of MIS did not have SARS-CoV-2 infections documented in the PEDSnet EHR database. Also note that during this period, especially the Omicron period, positive tests may have been from at-home tests or otherwise outside the system. Further assessment of cardiac complications after vaccination and COVID-19 is warranted to help provide a more complete picture of risk or benefit during a changing pandemic.

    Our study has several strengths. First, we used a national network of academic medical centers that covered a diverse cohort being more representative of the general pediatric population, provided a robust sample size, and allowed for multiple subgroup analyses and detection of rare outcomes. Second, the richness of these EHR data allowed us to investigate the effectiveness against infection of different levels of severity as well as adjust for a broad set of confounders. Third, we conducted the negative control outcome experiments to assess the potential residual bias due to unmeasured confounders and other potential sources of systematic bias in the data. These experiments revealed a small amount of systematic error but with excessive uncertainty across different negative control outcomes, leading to wider CIs of our estimated effectiveness that honestly reflect the effects of unmeasured confounding and other potential sources of residual biases (33). Finally, to the best of our knowledge, this is the first real-world effectiveness study evaluating COVID-19 vaccines against infection and severe outcomes that explicitly handle the underreporting in vaccination.

    Our study also has several potential limitations. First, effectiveness was investigated against documented infection in a cohort without previous infection, whereas the potential inclusion of patients with undocumented infections exists. Nevertheless, if this potential lack of data is evenly distributed across treatment groups, it could potentially attenuate the true vaccine effectiveness, thereby making our analysis more conservative in its estimations. Our inclusion of previous negative COVID-19 tests as a confounder aims to balance the probability of testing between treatment groups, which could partially adjust for this factor. Moreover, the increasing availability of at-home rapid antigen testing kits over time could have further reduced the testing frequency captured by the EHR. However, persons with severe illness who test positive through home kits typically seek medical care and report their results to hospitals. This would lead to a better capture of severe infection in our database and more reliable vaccine effectiveness estimates. Baseline confounders were balanced between vaccinated and unvaccinated groups, which should adjust for between-cohort differences in exposure risks and risks for severe infections. Second, as in any observational study, assignment to the vaccine group was nonrandom and the validity of the results could be affected by unmeasured confounders. To evaluate the effect of unmeasured confounders and residual bias, we conducted negative control experiments that quantified the robustness of our results.

    Third, in this study, patients who had received vaccinations before the start of the study period were excluded. Due to missing vaccine records, some patients who had previously been vaccinated may have still entered the cohort, particularly in the unvaccinated group. However, the CER method used in this study adjusted for potential bias resulting from unrecorded vaccinated patients, which could also reduce the bias resulting from this issue. Finally, in the Omicron study involving adolescents, the cohort included adolescents who had their first vaccine after 1 January 2022. Because the use of BNT162b2 vaccines was authorized in adolescents aged 12 to 15 years on 10 May 2021, this cohort may represent a population with late vaccines, which reduces the generalizability of the findings.

    Although this study provides evidence of a slight waning of vaccine effectiveness 4 months after the first dose against Omicron infection and the effectiveness is stabilized after 4 months, waning can be affected by vaccines during the follow-up period and other factors. Patients who received boosters during the follow-up period were not excluded from the study. A sensitivity analysis evaluating the durability of 2-dose vaccine effectiveness considering the third dose as censoring did not suggest a statistically significant different conclusion. A future study is warranted to investigate the effect of booster vaccination among children and adolescents. Furthermore, despite the recognized risk for myocarditis associated with COVID-19 vaccines in young men and teen boys, the study shows a lower relative risk for myocarditis, pericarditis, or MIS in vaccinated groups, which may be partially explained by reduced likelihood of infection during the study period (34).

    In conclusion, this study involving national pediatric cohorts in the United States estimates moderate effectiveness of the BNT162b2 vaccine for preventing infection and severe diseases of the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant. The study shows a lower risk for cardiac complications among children and adolescents who were vaccinated during the Omicron period and no significant difference between vaccinated and unvaccinated groups during the Delta period. Our assessment of vaccine effectiveness across diverse outcomes underscores the vaccine's pivotal role in reducing SARS-CoV-2 transmission, minimizing COVID-19–related sick leaves, and alleviating economic burdens during the pandemic. This study substantially contributes to our knowledge of the BNT162b2 vaccine in the U.S. pediatric population using a rigorously designed CER method accounting for the incomplete capture of vaccination status in EHR data in the United States.

    References

    Comments

    0 Comments
    Sign In to Submit A Comment
    Peter Yim10 January 2024
    Unjustified assumptions about vaccination status

    In this study, positive COVID vaccination status was determined from the EHR and is expected to be nearly 100% accurate. However, determination of negative COVID vaccination status was defined to be the absence of an EHR record of COVID vaccination. As such, the authors correctly observe that negative COVID vaccination status may be less than 100% accurate. False negative vaccination status occurs, for example, when a study subject receives COVID vaccination in a pharmacy not associated with one of the study institutions. The authors provide evidence that the effect of uncertainty of vaccination status on estimates of vaccine efficacy are limited provided that the "sensitivity" of the EHR to positive vaccination is 0.8 or greater for children and 0.7 - 0.9 for adolescents. However, the authors do not justify those ranges, only stating: "a range of possible sensitivities based on our prior study was prespecified for each study." No reference to that study is provided. These unjustified assumptions raise doubts about the study methodology and should limit confidence in the study conclusions.

    Qiong Wu (1), Jiayi Tong, Jeffrey S. Morris (1), Christopher B. Forrest(2), and Yong Chen(1, 3) on the authors’ behalf30 January 2024
    Author Response to Yim

    Note that some of the EHR data partners have augmented their data quality by including the vaccine records from a Health Information Exchange (HIE) program specifically focused on vaccine registries. The range of possible sensitivities was prespecified based on a pilot study, which compared the vaccination rates estimated from vaccine data obtained from health systems against the Centers for Disease Control and Prevention (CDC) reports. Specifically, a reference vaccination rate was calculated by weighting the CDC's county-level vaccination statistics by the patients’ residential addresses. This comparison between the health system-derived vaccination rates and the CDC's reference rates allowed us to predefine a range of possible values of sensitivity. The research paper detailing this pilot study is being finalized and will be submitted soon. For the pediatric population who receive their care during the pandemic from the participating institutions (all of which provided vaccination programs to the communities as well as the primary care physician units). A sensitivity of lower than 70% would be found uncommon.

    In addition, to evaluate the robustness of findings from the statistical methods we used to account for the underreporting, we have conducted sensitivity analyses using alternative methods, including the naive method (without adjusting for underreporting) and using different ranges of possible sensitivity values. Please refer to Section 16 of the Supplement for details.

    Disclosures:

    Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-1754

    Peter Yim31 January 2024
    Response to authors

    One question that may arise is the extent to which the EHR data partners with the augmented data sets overlapped with the study of the accuracy of COVID vaccination status classification.