Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement

The CONSORT (Consolidated Standards of Reporting Trials) statement, a checklist (Table 1) flow diagram first published in 1996 and revised 5 years later (2, 3), is an effort to standardize, and thereby improve, published reports of randomized, controlled trials (RCTs). One of the additions to the 2001 revision was an item about reporting adverse events. This single item did not do full justice to the importance of harms-related issues. The CONSORT Group met in September 2001 to discuss how to correct this deficiency. We aimed to provide evidence-based guidance on the reporting of harms in RCTs. First, we searched MEDLINE, EMBASE, Web of Science, and the Cochrane Library using a wide array of terms related to harms and identified pertinent evidence. We also communicated with experts and reviewed bibliographies of identified articles to find additional studies. At a meeting in Montebello, Quebec, Canada, in May 2003, CONSORT Group members, including several journal editors and additional experts in related fields, held a structured discussion of recommendations about reporting of harms-related issues in RCTs. The discussions led to a written document that we circulated among the team members for comment. The present manuscript describes our recommendations on the appropriate reporting of harms in RCTs.

Table 1. Original CONSORT Checklist

The terminology of harms-related issues in RCTs is confusing and often misleading or misused (see Glossary) (4, 5). “Safety” is a reassuring term that may obscure the real and potentially major “harms” that drugs and other interventions may cause. We encourage authors to use the term “harms” instead of “safety.” In addition to misused terminology, reporting of harms in RCTs has received less attention than reporting of efficacy and effectiveness and is often inadequate (6-14). In short, both scientific evidence and ethical necessity call for action to improve the quality of reporting of harms in RCTs (15, 16). Here, we present a set of recommendations and accompanying explanations for the proper reporting of harms in RCTs. These recommendations should complement the existing CONSORT statement (Table 2). Examples are presented on the Annals and CONSORT (www.consort-statement.org) Web sites.

Table 2. Checklist of Items To Include When Reporting Harms in Randomized, Controlled Trials

Recommendations

Title and Abstract

Recommendation 1. If the study collected data on harms and benefits, the title or abstract should so state.

The title should mention harms if the study of harms was a key trial objective. Many phase I and phase II trials, some phase II/III trials, and most phase IV trials (17, 18) target harms as primary outcomes. Yet, the title and abstract seldom contain the word “harm.” Among 375 143 entries in the Cochrane Central Register of Controlled Trials (Cochrane Library, issue 3, 2003), searching titles with the search terms harm or harms yielded 337 references (compared with 55 374 for efficacy and 23 415 for safety). Of the 337, excluding several irrelevant articles on self-harm or harm reduction, only 3 trial reports and 2 abstracts contained the word “harm” in their titles.

Authors should present information on harms in the abstract. If no important harms occurred, authors should so state. Explicit reference to the reporting of adverse events in the title or abstract is also important for appropriate database indexing and information retrieval (19).

Introduction

Background

Recommendation 2. If the trial addresses both harms and benefits, the introduction should so state.

The introduction states the scientific background and rationale of an RCT. This requires a balanced presentation whereby the possible benefits of the intervention under investigation are outlined along with the possible harms associated with the treatment. Randomized, controlled trials that focus primarily on harms should clearly state this interest when describing the study objectives in the Introduction and in defining these objectives in the Methods.

Methods

Outcomes

Recommendation 3. List addressed adverse events with definitions for each (with attention, when relevant, to grading, expected vs. unexpected events, reference to standardized and validated definitions, and description of new definitions).

The Methods section should succinctly define the recorded adverse events (clinical and laboratory). Authors should clarify whether the reported adverse events encompass all the recorded adverse events or a selected sample. They should explain how, why, and who selected adverse events for reporting. In trials that do not mention harms-related data, the Methods section should briefly explain the reason for the omission (for example, “the design did not include the collection of any information on harms”).

Authors should also be explicit about separately reporting anticipated and unexpected adverse events. Expectation may influence the incidence of reported or ascertained adverse events. Making participants aware in the consent form of the possibility of a specific adverse event (“priming”) may increase the reporting rate of the event (20). Another example of priming is the finding that the rates of withdrawals due to adverse events and the rates of specific adverse events were significantly higher in trials of aspirin, diclofenac, or indomethacin with comparator drugs compared with placebo-controlled trials (21). Presumably, participants were more eager to come forth and report an adverse event or to withdraw from treatment when they knew they could not be receiving inactive placebo.

Authors should report whether they used standardized and validated measurement instruments for adverse events. Several medical fields have developed standardized scales (22-32). Use of nonvalidated scales is common. The source document for well-established definitions and scales should be referenced. New definitions for adverse events should be explicit and clear. Authors should describe how they developed and validated new scales.

For interventions that target healthy individuals (for example, many preventive interventions), any harm, however minor, may be important to capture and report because the balance of harms and benefits may easily lean toward harms in a low-risk population. For other populations and for interventions that improve major outcomes (for example, survival), severe and life-threatening adverse events may be the only ones that are important in the balance of benefits and harms.

Recommendation 4. Clarify how harms-related information was collected (mode of data collection, timing, attribution methods, intensity of ascertainment, and harms-related monitoring and stopping rules, if pertinent).

It is important to describe the questionnaires, interviews, and tests used to collect information on harms, as well as their timing during follow-up. Passive surveillance of harms leads to fewer recorded adverse events than active surveillance (4). Open-ended questions may yield different information, both quantitatively and qualitatively, than structured questionnaires (33). Studies of nonsteroidal, anti-inflammatory drugs (NSAIDs) exemplify how data collection methods can affect the detection and reporting of harms. When selective NSAIDs with fewer gastrointestinal adverse events became available, trials reported more than 10 times as many ulcers when comparing these drugs with older NSAIDs as when older NSAIDs were compared with placebo. In the newer trials, more ulcers were detected because participants had regular endoscopy, and the case definition of ulcers was more sensitive (34).

Authors should specify the time frame of surveillance for adverse events. Some investigators stop recording adverse events at the end of the intervention period or a certain number of days afterward (for example, 30 days after discontinuation of drug therapy) and miss events with long latency (35). Surgical trials often capture only the adverse events that occur intraoperatively. Several important surgical complications are likely to occur later. Finally, in crossover trials, delayed events might occur while the patient is taking a subsequent assigned treatment.

Attribution is the process of deciding whether an adverse event is related to the intervention. Whenever authors filter events through an attribution process, they should state who makes the attribution (investigators, participants, sponsors, or combinations), whether the process is blinded to assigned treatment, and what definitions of adverse events they use (4).

Discontinuations and withdrawals due to adverse events are especially important because they reflect the ultimate decision of the participant and/or physician to discontinue treatment. Although treatment may occasionally be discontinued for mild or moderate adverse events, attributing discontinuation to a specific reason (to toxicity, lack of efficacy, other reasons, or combinations of reasons) may be difficult. For example, in psychopharmacology, dropouts may reflect treatment ineffectiveness as much as toxicity-related intolerance (36). Trial reports should specify who gave the reasons for discontinuation (participants or physicians) and whether attribution was blinded to the assigned treatment. For example, even in blinded trials, participants and their clinicians are often unblinded before they decide whether to discontinue the intervention. It is important to report participants who are nonadherent or lost to follow-up because their actions may reflect their inability to tolerate the intervention. Moreover, authors should specify how they handled withdrawals in the analyses of the data.

Randomized, controlled trials should report any plan for monitoring for harms and rules for stopping the trial because of harms (37). They should clarify whether stopping guidelines examine benefits and harms separately or evaluate a composite measure that reflects the trade-off between benefits and harms (38).

Statistical Methods

Recommendation 5. Describe plans for presenting and analyzing information on harms (including coding, handling of recurrent events, specification of timing issues, handling of continuous measures, and any statistical analyses).

Using only descriptive statistics to report harms is perfectly appropriate in most RCTs because most trials lack power to test harms-related hypotheses and indeed have no explicit prespecified harms-related hypotheses. If investigators combine data for different adverse events into 1 outcome measure, they should describe each combination, cite the dictionary that lists the definitions of the adverse events, and state whether they decided the grouping of events post hoc or a priori.

The distributions of adverse events over the follow-up period can pose problems for analysis of the data. When pertinent, authors should specify whether they count recurrent events (events that occur more than once in the same participant) as separate events or as 1 event. For trials with longitudinal follow-up, specifying the timing of the events may be important (for example, to separate early from late toxicity). Incidence rates, period prevalence rates, and point prevalence rates may provide complementary information about the occurrence of an adverse event. Kaplan–Meier curves showing cumulative incidence of important adverse events can be helpful. Simple summaries with person-time denominators (for example, median months after treatment) can be misleading if the event occurs only after extended treatment and long follow-up, and most participants had short follow-up and therefore no events.

For continuous variables (such as reported for most laboratory tests), means and SDs or medians and interquartile ranges may provide an aggregate picture, but they may not convey information on extreme values that correspond to severe toxicity. Means and medians may be useful in informing participants and clinicians about expected, relatively minor changes.

Scales are increasingly used for measuring quality of life in RCTs. These measures are composite outcomes that reflect both benefits and harms (39). Authors should describe the development of these instruments, their validity and sensitivity to detect change, and whether they assumed interval scaling in order to use the scale as a continuous variable.

When harms are major primary or secondary outcomes of a trial, the authors should describe plans to perform any formal statistical analyses and inferences. They should separate prespecified statistical analyses from post hoc analyses (40) and address common problems: low power for uncommon events, adjustment for multiple outcomes, composite outcomes, regression to the mean (for example, for laboratory tests that are also used for screening for study eligibility), and heterogeneity of treatment effects across prespecified subgroups (41).

Results

Participant Flow

Recommendation 6. Describe for each arm the participant withdrawals that are due to harms and the experience with the allocated treatment.

Authors should describe the reasons for discontinuations and reductions in dosage of the allocated treatment and withdrawals from the study. They should emphasize harms-related reasons and acknowledge the caveats noted under Recommendation 6. Authors should always report deaths in each study group during a trial, regardless of whether death is an end point and regardless of whether attribution to a specific cause is possible (42).

Randomized, controlled trials with prolonged follow-up should report the timing of allocated treatment received, dose reductions and discontinuations, and study withdrawals. The cause of early withdrawals may differ from that of late withdrawals; separate descriptions of each may enhance the accuracy of information on the tolerability profile of an intervention. Kaplan–Meier plots of the time-to-discontinuation or time-to-withdrawal for each study group may be useful, especially if the treatment duration is considerably different from the time of follow-up. These plots might reflect (and graphically show) differences in adverse events between treatment groups, which could suggest that harm is associated with the intervention.

Numbers Analyzed

Recommendation 7. Provide the denominators for analyses on harms.

There are many ways to report the number of participants included in analyzing harms. Randomized, controlled trials in which time-on-treatment differs from total follow-up should report the denominator for each analysis (that is, which participants and what follow-up time count toward total exposure to the allocated treatment. Terminology such as “intention-to-treat,” “modified intention-to-treat,” “available-case,” and “on-treatment” analyses, can be confusing and may provide different results, depending on the type of analyses conducted. Overall, intention-to-treat is usually the preferred analysis both for efficacy and harms because intention-to-treat is an analysis in which the original random participant assignment is maintained in the data analysis. Because differences in the use of the definitions of these types of analysis can be important, authors should state which analyses and definitions they use. Moreover, authors should state whether they use the same type of analysis for both efficacy end points and harms.

Rates of Outcomes and Ancillary Analyses for Adverse Events

Recommendation 8. Present the absolute risk of each adverse event (specifying type, grade, and seriousness per arm), and present appropriate metrics for recurrent events, continuous variables and scale variables, whenever pertinent.

Authors should present results separately for each study group of a trial. For each type of adverse event, they should offer appropriate metrics of absolute risk (for example, frequency or incidence), with separate information about the severity grade of the event, if relevant. Serious events should be reported separately for each type of event. Recurrent events and timing of events need appropriate reporting, as discussed in Recommendation 5. For events with many recurrences, it is useful to provide both the number of affected participants and the number of events for each study group and rate (events per unit of person-time). Occasionally, a graphical representation of the distribution of number of events per patient or time-to-event analyses may be informative.

Overall, the Results section should report on what the Methods section promises (43). Any break from this symmetry requires explanation. If no adverse events of a specific type and severity occurred, authors should so state in the Results section (44). Table 3 shows common reporting practices to avoid.

Table 3. Common Poor Reporting Practices for Harms-Related Data

Recommendation 9. Describe any subgroup analyses and exploratory analyses for harms.

Reporting of adverse events for different participant subgroups follows the same principles that govern the reporting of subgroup analyses for efficacy. Authors should avoid overstating the significance of false-positive subgroup findings (45). Authors should state how, why, and when they planned subgroup analyses (a priori or post hoc). Regulatory agencies increasingly require subgroup analyses by age, sex, and race for license applications. However, these variables rarely show any significant effect modification for efficacy outcomes (45, 46) and may be equally low-yield (or even misleading) for harms.

Discussion

Recommendation 10. Provide a balanced discussion of benefits and harms with emphasis on study limitations, generalizability, and other sources of information on harms.

The Discussion is typically the most poorly structured section of an RCT report and could be modified to gain space for reporting harms. Authors may curtail the length of the Discussion section to gain space for appropriate reporting of harms.

In summarizing the key findings of an RCT, the Discussion section should pay attention to the original trial objectives and provide a balanced view that puts the benefits and harms into perspective. Authors should avoid overinterpretation of the findings. Limitations are probably the most important part of the Discussion section. Common limitations in studies that report harms include inconclusive findings, lack of power, multiplicity of comparisons, post hoc analyses, and short duration of exposure to the allocated treatment, especially for treatments of chronic diseases. Generalizability is often a problem for harms. The frequency and severity of adverse events may depend on the clinical setting and participants. Often, clinical trials enroll participants who have the disease of interest but are otherwise healthy and do not have comorbid conditions. Once licensed, however, most approved interventions are used in individuals who have several comorbid conditions and who are taking several other drugs with potentially additive or synergistic toxicity.

The Discussion section should also appraise emerging data on benefits and harms. Authors should systematically integrate prior evidence on harms, whenever possible (47). If a systematic review of previous studies of harms is not possible, authors should so state, perhaps to stimulate someone to correct the deficiency in the future. Authors should contrast the trial results on harms with other sources of information on harms, including observational data from spontaneous reporting, automated databases, case–control studies, and case reports.

Manuscript Length

Improved reporting of harms need not lead to longer manuscripts. In the Methods section, adopting standard definitions with appropriate references may actually save space. Tables may summarize key results on harms. Graphs may convey important time-to-event outcomes or repeated measurements of adverse events. Finally, it is possible to write short Discussion sections by using an appropriate structure (48).

Occasionally, investigators present adverse events separately in another paper. This practice denies both author and reader the opportunity to formulate the balance between benefits and harms. Therefore, authors should report harms and benefits together in the same manuscript. Authors and journal editors should publish additional trial information on the Web, as an adjunct to the main results. For example, Web material could include enhanced graphical displays that present individual participants' experiences (49).

Investigators may sparingly use single patient reports embedded in the results of an RCT to describe severe, serious, and previously unreported adverse events. More comprehensive case reports may require a separate paper. In the RCT report, a single sentence may suffice to describe an unusual adverse event by succinctly summarizing the type of adverse event, when it occurred, the type of patient, the management of the adverse event, and the outcome of the adverse event.

Conclusions

This extension of the main CONSORT statement to include harms is a work in progress. We therefore invite readers to submit comments, critique, and suggestions for improvement through www.consort-statement.org. We also hope that journals and editorial groups will support our efforts to improve the reporting of harms. We ask journals that endorse the CONSORT reporting requirement to include a reference to this document in their Instructions to Authors section. Adherence to reporting standards for harms should help to inform readers and the public on the harms of interventions.

Glossary

Adverse events: Side effects that are harmful. However, side effects suggest causality (effects caused by the tested intervention). Some authors use the term “adverse effects” synonymously with “side effects”. In the typical randomized trial, it is difficult to know whether an observed event is partially or entirely due to the intervention or whether it is totally unrelated to the intervention (for example, a consequence of the underlying disease process). The purpose of a trial is to collect and appropriately report good and bad events and outcomes so that they may be compared across treatment groups. In this regard, the term “adverse events” is probably better to describe harmful events that occur during a trial.

Adverse reaction and adverse drug reaction (ADR): Events for which a causality link to the tested intervention is well established and strong enough (sensitive and specific) to warrant attribution of the event to the intervention (for details, see definitions proposed in references 4 and 5). Attribution of causality in the setting of clinical trials may be misleading.

Harms: The totality of possible adverse consequences of an intervention or therapy; they are the direct opposite of benefits, against which they must be compared.

Passive surveillance of harms: The recorded adverse events are those that the study participants spontaneously report on their own initiative. In active surveillance of harms, participants are asked about the occurrence of specific adverse events in structured questionnaires or interviews or predefined laboratory or other diagnostic tests are performed at prespecified time intervals.

Risk–benefit ratio: The most common expression for the comparison of harms and benefits. It is a technical term that assumes that a ratio can indeed be calculated. Because the benefits and harms of an intervention are often so different in character or are measured on different scales, the term “risk–benefit ratio” has no literal meaning. In addition, there may be several distinct benefits and harms. We advocate using “balance of benefits and harms” rather than “risk–benefit ratio.”

Safety: Substantive evidence of an absence of harm. The term is often misused when there is simply absence of evidence of harm.

Serious adverse events: As defined by the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, document E2A (available at www.ich.org): “During clinical investigations, adverse events may occur which, if suspected to be medicinal product-related (adverse drug reactions), might be significant enough to lead to important changes in the way the medicinal product is developed (e.g., change in dose, population, needed monitoring consent forms). This is particularly true for reactions which, in their most severe forms, threaten life or function. Such reactions should be reported promptly to regulators.”

Side effects: Unintended drug effects. The term, however, does not necessarily imply harm, as some side effects may be beneficial. Furthermore, it tends to understate the importance of harms because “side” may be perceived as denoting secondary importance.

Toxicity: Describes drug-related harms. The term may be most appropriate for laboratory-determined measurements, although it is also used in relation to clinical events. Abnormal laboratory values may be described as laboratory-determined toxicity. The disadvantage of the term “toxicity” is that it implies causality. If authors cannot prove causality, the terms “abnormal laboratory measurements” or “laboratory abnormalities” are more appropriate to use.

Appendix 1: Members of the Consort Group

Kamran Abbasi, MB (British Medical Journal); Douglas Altman, DSc (Centre for Statistics in Medicine, Oxford, United Kingdom); Matthias Egger, MD (Bristol University, Bristol, United Kingdom, and University of Bern, Bern, Switzerland; International Journal of Epidemiology); Diana Elbourne, PhD (London School of Hygiene and Tropical Medicine, London, United Kingdom); Stephen J.W. Evans, MSc (London School of Hygiene and Tropical Medicine, London, United Kingdom); Joel Gagnier, ND (University of Toronto, Toronto, Ontario, Canada); Peter Gøtzsche, MD (Nordic Cochrane Centre, Copenhagen, Denmark); David Grimes, MD (Family Health International, Research Triangle Park, North Carolina); John Ioannidis, MD (University of Ioannina, Ioannina, Greece); David Jaques, MD (Memorial Sloan-Kettering Cancer Center, New York, New York); Tom Lang, MA (Tom Lang Communications, Lakewood, Ohio), Joseph Lau, MD (Tufts-New England Medical Center, Boston, Massachusetts); Yoon Kong Loke, MD (University of Oxford, Oxford, United Kingdom); Faith McLellan, PhD (The Lancet); Siddika Mithani, MD (Health Canada–Clinical Trials, Ottawa, Ontario, Canada); David Moher, PhD (Chalmers Research Group and University of Ottawa, Ottawa, Ontario, Canada); Cynthia Mulrow, MD (Annals of Internal Medicine and University of Texas Health Sciences Center, San Antonio, Texas); Robert O'Neill, PhD (Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Rockville, MD); Gilda Piaggio, PhD (World Health Organization, Geneva, Switzerland); Drummond Rennie, MD (University of California at San Francisco, San Francisco, California; Journal of the American Medical Association); David Riley, MD (University of New Mexico Medical School, Santa Fe, New Mexico; EXPLORE: The Journal of Science and Healing); David Sackett, MD (Trout Research and Education Centre, Irish Lake, Ontario, Canada); Roberta Scherer, PhD (University of Maryland School of Medicine, Baltimore, Maryland); Kenneth Schulz, PhD (Family Health International, Chapel Hill, North Carolina); James Scott, MD (University of Utah, Salt Lake City, Utah; Obstetrics and Gynecology). Observers: Don Husereau, BScPharm, MSc (Canadian Coordinating Office for Health Technology, Ottawa, Ontario, Canada); Andra Morrison, BSc (Chalmers Research Group, Ottawa, Ontario, Canada); Isabelle French, BSc (Dalhousie University, Halifax, Nova Scotia, Canada); Gabriela Lewin, MD (Chalmers Research Group, Ottawa, Ontario, Canada).

Note: Laurence Hirsch, MD, and Beate Stych, MD (Merck), participated in the meetings as observers.

Appendix 2: Examples of Recommendations

Recommendation 1

Title with benefits and harms:

The safety and efficacy of prophylactic ondansetron in patients undergoing modified radical mastectomy (50).

Title of trial with primary harms outcome:

An assessment of the safety of pediatric ibuprofen. A practitioner-based randomized clinical trial (17).

Title with emphasis on harms:

No benefit, but increased harm from high dose (100 µg) misoprostol for induction of labour: a randomised trial of high vs. low (50 µg) dose misoprostol (51).

Abstract:

There were two uterine ruptures and four intrapartum stillbirths in the high misoprostol group. There was no difference in postpartum haemorrhage, 9.5% vs. 7.9% (P = 1.00) and admissions to the neonatal unit 18.8% vs. 17.0% (P = 0.980) in the 1ow- and high-groups) respectively. … the higher dose had an increased risk of serious complications (51).

Abstract conclusion:

CONCLUSION: In postmenopausal women with coronary disease, neither hormonal replacement therapy nor antioxidant vitamin supplements provide cardiovascular benefit. Instead, a potential for harm was suggested with each treatment (52).

Recommendation 2

Short courses of prednisone, 1 to 2 mg/kg daily for 3 to 10 days, are recommended for the management of acute exacerbations of asthma in children. … We could find no evidence in the literature that 2 mg/kg is more beneficial than 1 mg/kg in treating children. Furthermore, studies in asthmatic children have looked primarily at efficacy and parental management practices, rather than at the possible adverse effects. Because the parents of many of our patients have voiced concern about the behavioral changes in their children during short courses of treatment with oral steroids, we decided to conduct a prospective, randomized, blinded trial comparing the adverse effects and the benefits at the two dose levels (53).

Recommendation 3

Comprehensive list:

… assessments included minimum systolic blood pressure, minimum pressure of oxygen, maximum concentration of delivered nitrogen dioxide, and maximum concentration of methemoglobin (54).

Comprehensive list with definitions:

Bleeding complications and other adverse events were documented by interview or were reported by the patients during the study. Major bleeding was defined as any clinically apparent bleeding associated with a decrease of at least 2.0 g per dL in the hemoglobin level, requirement for transfusion of at least 2 units of packed red cells, or retroperitoneal or intracranial bleeding or other bleeding that the investigators decided required permanent discontinuation of treatment. Bleeding that did not meet this definition was considered minor. An adverse event was considered serious if it was fatal or life threatening, caused permanent disability, or required hospitalization or prolonged hospitalisation (55).

Definitions and grading (referral to established system):

… using the AIDS Clinical Trials Group adverse event grading scheme (56).

New definition:

In June 1998, the protocol was amended to include laboratory monitoring, toxicity management, and dose reduction of adefovir dipivoxil for proximal renal tubular dysfunction (PRTD), because of new information about the toxicity provided by Gilead Sciences. The protocol definition of PRTD was as follows: serum creatinine 0.5 mg/dL above baseline and serum phosphate < 2.0 mg/dL, or 1 of these abnormalities plus 2 of the following: proteinuria (2+), glycosuria (1+) in the absence of hyperglycemia, hypokalemia (<3.0 mEq/L), or serum bicarbonate <19 mEq/L (57).

Recommendation 4

Mode of data collection:

At each semiannual contact, a standardized interview collected information on designated symptoms and [harms] concerns, and initial reports of outcome events were obtained using a self-administered questionnaire (58).

Timing:

Adverse experiences and toxic effects were assessed every 4 weeks until 12 weeks after the discontinuation of the study drugs (59).

Attribution methods:

Causality was assessed by the investigator at the time of the event, using a modified version of Karch and Lasagna's 5-point scale (60).

Harms-related stopping rules:

Trial monitoring guidelines for early stopping considerations were based on O'Brien-Fleming boundaries using asymmetric upper and lower boundaries: a 1-sided, .025-level upper boundary for benefit and 1-sided, .05-level lower boundaries for adverse effects. The adverse-effect boundaries were further adjusted with a Bonferroni correction for the 7 major outcomes other than breast cancer that were specifically monitored (58).

Recommendation 5

Coding:

More than 200 distinct types of clinical, laboratory or ECG events were noted using … literal description, but the FDA coding symbols for the Thesaurus of Adverse Reaction Terms (COSTART), used to report adverse events, reduced that number to 110. Since some adverse events were similar, i.e. fatigue, asthenia, feeling unease, they were also regrouped, resulting in … 37 [descriptions] … (60)

Timing issues—early vs. late events:

Early reactions. Early reactions were recorded by the nursing staff in the cardiac catheter laboratory and on the cardiology ward after the patient left the catheter laboratory. … Late reactions. Each patient was asked to complete a simple questionnaire after discharge from the hospital, on which to record any adverse reactions occurring within 1 week of the cardiac catheterization. … The analysis of patients with late skin reactions was confined to patients with reactions that had clearly started after hospital discharge and therefore were not a continuation of, for example, an urticarial reaction occurring in the catheter laboratory. … (61) [Note: use of the term “reactions” may not be optimal, since it implies causality, but the example is appropriate for timing issues].

Continuous measures (mean estimates and serious extremes):

… assessed by changes in vital signs (summarized as a mean [SE] change from baseline) and by reports of adverse events with onset within 8 weeks of randomization. All reports of adverse events were included whether or not they were deemed by the investigator to be related to treatment. An adverse event was defined in the study protocol as serious if it was fatal or life-threatening, required or prolonged hospitalization, or resulted in persistent or significant disability or incapacity (62).

Statistical analyses:

Patients reported occurrence and severity of 18 side effects. … Between-group differences were compared by using the Wilcoxon test (63).

Recommendation 6

Withdrawals and harm-related reasons for withdrawals per arm:

Eight other patients (6 treated with fluconazole and 2 treated with itraconazole) were withdrawn from the study because of mild to moderate symptoms, such as rash (fluconazole group), dry skin (fluconazole group), nausea (fluconazole group), or difficulty concentrating (itraconazole group) (64).

No withdrawals due to adverse events:

No patients receiving trimethoprim−sulfamethoxazole or ciprofloxacin discontinued therapy with the drug because of side effects (65).

Treatment experience over time:

A substantial number of women had stopped taking study drugs at some time (42% of estrogen plus progestin and 38% of placebo) [Figure shows cumulative dropout rates over time] (58).

Treatment exposure:

The study drug exposure was as follows: 15% received 1 dose; 31% received 2 doses; 37% received 3 doses; and 17% received 4 doses (66).

Recommendation 7

All randomly assigned and treated patients (n = 68) were included in the … [harms] … analysis (67).

Recommendation 8

Absolute risks for binary events per arm and per type and grade (follow-up/exposure time approximately comparable for all participants):

See Appendix Table 1.

Appendix Table 1. Adverse Events among Human Immunodeficiency Virus-Infected Patients Whose Aphthous Ulcers Had Healed Previously When Treated with Thalidomide and Who Then Were Treated with Thalidomide or Placebo in a Maintenance-Phase Study

Absolute risks for binary events per arm and per type and grade (follow-up/exposure time is differential and not comparable for all participants):

45 patients (46 events) in the rofecoxib group and 20 patients (20 events) in the naproxen group were adjudicated to have serious thrombotic cardiovascular adverse events (myocardial infarction, unstable angina, cardiac thrombus, resuscitated cardiac arrest, sudden or unexplained death, ischemic stroke, and transient ischemic attacks). Event-free survival analysis of these 66 patients showed that the RR (95% confidence interval [CI]) of developing a cardiovascular event in the rofecoxib treatment group was 2.38 (1.39-4.00), P < .001 [Figure shows the Kaplan–Meier plots for time to cardiovascular adverse event in each arm] (69).

Recurrent events expressed with person-time denominator:

The frequency of hypoglycemia at 3:00 a.m. was greater in the mixed-treatment period than in the split-treatment period (0.28 [SD 0.04] episode/patient-day vs. 0.10 [SD 0.02] episode/patient-day, respectively; P = 0.002) [table also shown]. On average, each patient experienced 2.8 (95% CI, 1.9 to 3.7) fewer episodes of hypoglycemia with split than with mixed dosing [Figure also shown] (70).

Continuous measures presented with information on both averages and extremes:

There were small changes in mean (SE) systolic blood pressure (placebo group, −2.0 [0.5] mm Hg and carvedilol group, −3.6 [0.5] mm Hg) and in diastolic blood pressure (placebo group, −1.8 [0.3] mm Hg and carvedilol group, −2.7 [0.3] mm Hg) at the end of 8 weeks. …Patients in the carvedilol group were more likely than in the placebo group to report … hypotension … and bradycardia [see Appendix Table 2] (62).

Appendix Table 2. Adverse Events during the First 8 Weeks

Recommendation 9

Across the same subgroups shown in Figures 1-3, the relative risk of “any bleeding” with treatment compared with placebo ranged from 0.65 to 1.86 (data not shown). No statistically significant treatment-by-subgroup interactions were noted …. Using the same consistency criteria employed for mortality, “any bleeding” and “serious bleeding” results for all subgroups were consistent with the overall trial results. For bleeding event end points (any or serious bleeding), there was no statistically significant interaction with predicted risk of mortality (P = .55 and P = .21) (72).

Recommendation 10

Discussion of harms:

[In] our study pretreatment with a small dose of subcutaneous adrenaline significantly reduced the incidence of acute adverse reactions to polyvalent antivenom serum. … We did not encounter significant adverse effects attributable to it; there were no cases of acute neurological deficit suggestive of cerebrovascular accidents or patients in whom blood pressure rose significantly (73).

Limitations:

Even this large study was still too small to allow for a rigorous … assessment [of harms]. The data on drug related clinical adverse experiences show that etoricoxib may be better tolerated than indometacin [sic], but additional studies are needed to enable any definitive conclusions … (74).

Other sources of information:

Adefovir dipivoxil-related PRTD occurred in 1% of study patients during the first 16 weeks. In other studies, 35% of patients taking adefovir dipivoxil developed significant increases in serum creatinine levels, and 50% developed significant hypophosphatemia, laboratory abnormalities consistent with PRTD, by 48 weeks (57).