THE EVIDENCE www.evidencejournals.com
Meta-Analysis Cardiovascular Diseases
Cite this Article
Kumar V, Soni M, Dutt M, Assessing the incidence of myocarditis risk in mRNA COVID-19 vaccines: a systematic review and meta-analysis. The Evi. 2024:2(1):-1-. DOI:10.61505/evidence.2024.2.1.27
Available From
https://the.evidencejournals.com/index.php/j/article/view/27

Received: 2024-01-19
Accepted: 2024-01-28
Published: 2024-05-28

Evidence in Context

• Reviewed myocarditis risks from over 720 million doses of Pfizer and Moderna vaccines. • Found a slightly higher risk with Moderna, with a risk ratio of 1.62. • Noted increased myocarditis risk primarily after the second vaccine dose. • Concluded that myocarditis is rare; vaccine benefits outweigh risks. • Advocated for continued use of mRNA vaccines, recommending further research.

To view Article

Assessing the incidence of myocarditis risk in mRNA COVID-19 vaccines: a systematic review and meta-analysis

Vijay Kumar1*, Manya Soni1, Mehak Dutt1

1 Global Center for Evidence Synthesis, Chandigarh, Chandigarh, India.

*Correspondence: vijaykumar990708@gmail.com

Abstract

Objectives: This systematic review and meta-analysis aimed to comprehensively compare the risk of myocarditis associated with two mRNA COVID-19 vaccines: Pfizer (BNT162b2) and Moderna (mRNA-1273).

Methods: Under the PRISMA guidelines, this review incorporated observational cohort studies and case series that compared the risk of myocarditis in individuals who received either the Pfizer (BNT162b2) or Moderna (mRNA-1273) vaccines. The studies were discovered through a systematic exploration of PubMed, EMBASE, and Web of Science. The data extraction concentrated on the incidence of myocarditis, demographic details, type of vaccine, and dosage count. A random-effects model was used to conduct the meta-analysis. The JBI critical appraisal tools were used to evaluate the risk of bias.

Results: A total of nine studies were identified that fulfilled the selection criteria, which included four cohort studies and five case series. These studies collectively analyzed data from 294,731,021 doses of the Moderna vaccine and 426,526,128 doses of the Pfizer vaccine. The aggregated risk ratio (RR) for myocarditis was 1.62 (95% CI: 1.02, 2.56) when comparing Moderna to Pfizer. Furthermore, a subgroup analysis based on the first and second doses revealed varying risk levels. For the first dose, the RR was 1.14 (95% CI: 0.79, 1.64) for myocarditis in individuals who received the Moderna vaccine compared to those who received the Pfizer vaccine. For the second dose, the RR increased to 1.69 (95% CI: 0.79, 3.59).

Conclusion: Myocarditis following mRNA COVID-19 vaccination was infrequent, with a very slight increased risk observed in Moderna recipients compared to Pfizer. These findings suggested the use of mRNA COVID-19 vaccines in all eligible individuals as suggested by CDC and WHO previously.  

Keywords: mRNA COVID-19 vaccines, myocarditis risk, cardiac side effects, post-vaccination side effects, vaccine safety

Introduction

The emergence of the SARS-CoV-2 pandemic instigated the progression and implementation of vaccines. mRNA-based vaccines, notably Pfizer's BNT162b2 and Moderna's mRNA-1273, have played a crucial role in mitigating the transmission of COVID-19 [1,2]. However, post-marketing surveillance and subsequent research have raised concerns regarding rare cardiac side effects, notably myocarditis, associated with these vaccines [3].

© 2024 The author(s) and Published by the Evidence Journals. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

In the medical history, myocarditis has been identified as an infrequent complication associated with diverse vaccines and viral infections, inclusive of COVID-19. The condition involves inflammation of the heart muscle and can manifest with a spectrum of symptoms ranging from mild to severe cardiac dysfunction [4]. The introduction of mRNA COVID-19 vaccines brought myocarditis into focus due to reported post-vaccination cases.

Initial studies highlighted a higher incidence of myocarditis following COVID-19 vaccination compared to unvaccinated individuals [5,6]. A higher incidence of myopericarditis among adolescents was seen following mRNA COVID-19 vaccination but with a very low overall incidence rate [7,8]. The research trajectory saw a significant emphasis on comparing the myocarditis risk between the two prominent mRNA vaccines. Essential findings were found in this regard, showing that myocarditis cases were more frequent in individuals vaccinated with mRNA-1273 compared to BNT162b2 [9]. Further study indicated more than 2 times higher odds of developing myocarditis/pericarditis with mRNA-1273 compared to BNT162b2, especially among younger males [10].

Despite these findings, it is crucial to contextualize the absolute risk of myocarditis to the broader benefits of COVID-19 vaccination. The rarity of myopericarditis post-vaccination has been underscored, emphasizing that the benefits of vaccination in preventing severe COVID-19 far outweigh the risks of rare adverse events [11,12]. Furthermore, the incidence of myocarditis following vaccination is significantly lower than that observed following SARS-CoV-2 infection itself [13,14]. This systematic review and meta-analysis aim to provide a comprehensive comparison of the risk of myocarditis between the Moderna and Pfizer mRNA COVID-19 vaccines. It seeks to address the gaps in understanding the differential risks associated with each vaccine. Through this review, we aim to contribute to informed decision-making in vaccine administration and policy, particularly in the context of optimizing vaccine safety and public health strategies.

Methods

This review was conducted according to the PRISMA guidelines [15] (Table S1) and has been registered in PROSPERO with registration number CRD42023492119. The entire review process (database search, duplicate removal, abstract screening, full-text screening, tagging, and synthesis) was conducted using semi-automated Auto Lit software (Nested Knowledge, MN, USA). It is a software aiding systematic reviews and meta-analysis. Employing nested knowledge structures, streamlines data extraction and analysis, enhancing efficiency and rigor in evidence synthesis processes.

Eligibility Criteria

The inclusion of the studies was based on the following inclusion criteria: observational cohort studies, case-series comparing myocarditis risk among individuals receiving BNT162b2 or mRNA-1273 COVID-19 vaccines. Exclusion criteria included non-peer-reviewed studies, editorials, and studies not reporting specific data on myocarditis following Pfizer or Moderna vaccination.

Database Search

A comprehensive and systematic search was conducted across three major databases: PubMed, EMBASE, and Web of Science. This search encompassed all records available from each database's inception up to November 20, 2023, with the primary objective of identifying studies relevant to our research. To ensure the inclusion of the most current literature, the search was updated on December 13, 2023, allowing for the identification and inclusion of any newly published studies. In addition to this electronic search, we performed a manual examination of the reference lists from identified relevant studies, aiming to uncover any eligible article that has not been captured during an initial electronic database search.

Search Strategy

The search strategy included combinations of the following terms: "Pfizer", "BNT162b2", "Moderna", "mRNA-1273", "myocarditis", and “inflammatory cardiomyopathy". The complete search string for each database is available in the supplementary material (Table S1).

Selection Process

Two independent reviewers screened each record and reported for eligibility. Discrepancies were resolved through discussion or consultation with a third reviewer. The selection process did not involve automation tools.

Data Collection Process

Two reviewers independently extracted data on the incidence of myocarditis, demographic characteristics (age), vaccine types (Pfizer or Moderna), and the number of doses administered using the tagging function of the Nested-Knowledge software. In case of differences in opinion between two reviewers, third reviewer was consulted. No automation tools were used.

Study Risk of Bias Assessment

In each study, the risk of bias was evaluated using JBI critical appraisal instruments designed for cohort studies and case series [16,17]. Two reviewers independently assessed each study, with disagreements resolved through discussion or a third reviewer.

Effect Measures

The primary effect measure was the risk ratio (RR) for myocarditis associated with each vaccine.

Synthesis Methods

A random-effects model was utilized for the meta-analysis. The assessment of statistical heterogeneity was carried out with the I² statistic. Subgroup analyses were performed, focusing on the first and second doses. The compilation of this data was done using Revman software, version 5.4 and R software.

Publication bias assessment

The assessment of publication bias was done through the use of a funnel plot and the application of Egger's regression test to detect asymmetry.

evi-02-01-27-01.jpg

Figure 1 PRISMA flow diagram showing identification process of included studies

Results

A total of 1,134 articles were identified through database search. After the abstract screening, 108 duplicates were removed and one additional study was included based on

an expert recommendation. Further screening process led to the exclusion of articles based on relevance, as judged by their titles and abstracts, and those lacking necessary data in their full texts. Consequently, 20 articles were selected for full-text review (Figure 1) [18].

Of these 20 articles, 9 studies met our inclusion criteria. Selected studies comprised four cohort studies and five case series. The systematic review and meta-analysis incorporated 4 observational studies from the United States [6,19-21], 2 from Colombia [9,10], and 1 each from Denmark [22], Italy [23], and Canada [24]. These studies utilized diverse data sources, including national health records and surveillance systems, spanned from December 2020 to March 2022, across age groups (12-80 years). All studies focused on myocarditis cases following Pfizer and Moderna vaccinations. The case definition for myocarditis identification and diagnosis are summarized in Table 1. Included observational studies were assessed using JBI tools for Case series and Cohort studies and summarized in supplementary tables (table S2a and table S2b).

The analysis shown in Figure 2 integrated data from studies involving 294,731,021 Moderna vaccine doses (first and second) and 426,526,128 Pfizer doses (first and second). The incidence of myocarditis was 1,409 for Moderna vaccine doses (first and second) and 2,707 for Pfizer vaccine doses (first and second) among various studies. Using the Mantel-Haenszel random-effects model, a pooled risk ratio of 1.62 (95% Confidence Interval [CI]: 1.02, 2.56) was found. Significantly high heterogeneity (I² = 98%) and a Z-score for the overall effect of 1.70 (p = 0.09) was observed. Subgroup analyses were performed to assess differential risk profiles across different vaccine doses of Pfizer and Moderna. The analysis for the first dose presented in Figure 3, myocarditis events reported were 222 for Moderna and 347 for Pfizer, corresponding to the respective totals of 777,94453 and 10,930,2450 doses. The pooled RR for this subgroup was 1.14 (95% CI: 0.79, 1.64), with moderate heterogeneity (I² = 49%) and a Z-score for the overall effect of 0.70 (p = 0.48).

evi-02-01-27-02.jpg

Figure 2 Forest plot showing pooled RR of myocarditis among both the doses

Referring to Figure 5, the outcomes following the administration of the second dose showed 428 and 769 myocarditis events for Moderna and Pfizer respectively, against a backdrop of 67,150,511 and 93,768,883 total doses. This analysis yielded a pooled RR of 1.69 (95% CI: 0.79, 3.59), with a very high level of heterogeneity (I² = 94%) and a Z-score for the overall effect of 1.36 (p = 0.17).

evi-02-01-27-03.jpgFigure 3 Forest plot showing pooled RR of myocarditis among first dose

evi-02-01-27-04.jpg

Figure 4. Forest plot showing pooled RR of myocarditis among second dose

Discussion

Our meta-analysis is first to compare the risk of myocarditis in two different COVID-19 mRNA vaccines, specifically Pfizer (BNT162b2) and Moderna (mRNA-1273). The study was conducted across a wide range of populations covering adolescents, young adults, and the geriatric population. Study findings suggest a slightly higher risk of myocarditis among the recipients of Moderna compared to Pfizer. Subgroup analysis conducted for the first and second doses of these vaccines also suggests increased risk among recipients of Moderna. However, the difference in risk of these two vaccines has not come out to be statistically significant.

Many primary studies highlighted the increased risk of myocarditis to a very large extent among recipients of Moderna unlike ours [9,10,22,24]. One study has found that there was an increased risk of myocardial infarction, ischemic stroke, other thromboembolic incidents, and Renal damage in individuals who received the Pfizer vaccine versus those who were given the Moderna vaccine [25]. A study from USA indicated no significant difference in the risk of myocarditis between Moderna and Pfizer [21]. These results are in line with our analysis. There have been divergent results from various studies. These findings highlight the significance of carrying out comparative studies directly to consider possible differences among individuals who receive these mRNA vaccines.

Considering their future use. Decisions about risk management in public health practice must consider several intricate and occasionally contradictory elements [26]. The kind of population to whom we are providing the benefits should be considered when attempting to balance risks and benefits, especially if there are benefits that balance the risks and risk cannot be completely avoided. Given the demonstrated efficacy of mRNA COVID-19 vaccines in reducing instances of severe illness, hospital admissions, and fatalities associated with COVID-19 on an individual level, along with their role in decreasing community transmission and protecting immunocompromised persons, these vaccines also contribute to maintaining the operational capacity of the community healthcare system [27,28]. Moreover, it was found in one SRMA that there was a seven-fold higher risk of myocarditis among COVID-19 patients compared to mRNA-vaccinated individuals [29]. This indicates the ongoing recommendation for the use of mRNA COVID-19 vaccines in all eligible individuals, as advised by the CDC and WHO. Future studies should aim to precisely determine the incidence of myocarditis associated with mRNA COVID-19 vaccines, explore the underlying biological processes causing these rare cardiac events, and identify individuals who are most at risk. This will enable the creation of benefit-risk profiles tailored to various age groups. Future studies should also focus on the risk of such rare cardiac events from the new emerging variants and the newer vaccines should also be tested for their risk of causing such events.

The study presented several constraints. Primarily, the lack of comprehensive

data restricted our ability to assess potential variations in risk based on gender and the number of vaccine doses (first and second) administered. Our study does not report gender-specific risk of myocarditis among recipients of mRNA vaccines. Additionally, the observational nature of the study design inherently carries the potential for bias, stemming from the non-randomized approach to vaccination in real-world scenarios. It is important to acknowledge that the characteristics of the vaccinated population might differ significantly in key areas such as disease susceptibility, and availability of screening and healthcare services. These factors were not considered in our study. Discussing myocarditis that coincides with COVID-19 vaccinations does not definitively indicate a diagnosis of myocarditis caused by the vaccine, since it is difficult to differentiate it from myocarditis due to other reasons.

Although there are some limitations, the study has several significant strengths. First, the data for our quantitative analysis came from various well-executed observational studies, as indicated by the risk of bias assessment using the JBI tool for risk of bias in case series and cohort studies. we included population-based cohort studies, which exhibit diversity in race and ethnicity. The majority of these studies share a similar timeframe, enhancing their comparability. Additionally, all instances of myocarditis in the studies were evaluated according to established case definitions, reducing the potential for misclassification that often occurs when relying solely on diagnostic codes.

Conclusion

Myocarditis after receiving an mRNA vaccine is uncommon. While vaccination with mRNA-1273 shows a marginally increased risk of myocarditis compared to the BNT162b2 vaccine, though this difference has come out as statistically non-insignificant. This indicates the ongoing recommendation for the use of mRNA COVID-19 vaccines in all eligible individuals. Further, Differences in risk between products can contribute to risk-benefit discussions and better policy decisions, as countries vaccinate younger populations with additional doses. Thus, there is a need for direct comparison studies to understand more clearly the associated risks with different mRNA vaccines.

Acknowledgments

A special acknowledgment goes to the developers of the Nested Knowledge software. This tool was instrumental in our meta-analysis process, offering robust features for managing and analysing large datasets. It is a software aiding systematic reviews and meta-analysis. it streamlines data extraction and analysis, enhancing efficiency and rigor in evidence synthesis processes. Its role in facilitating our research was crucial and greatly appreciated.

Supporting information

Provided: Supplementary Tables S1, S2a, S2b. Download

Ethical Considerations

None

Acknowledgments

None

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author contribution statement

Vijay Kumar: Conceptualization (lead); writing – original draft (lead); formal analysis (lead); writing – review and editing(equal). Manya Soni: conceptualization, Software (lead); writing – review and editing (equal). Mehak Dutt: Methodology (lead); writing – review and editing (equal).

All authors attest they meet the ICMJE criteria for authorship and gave final approvalforsubmission.

Data availability statement

Data included in article/supp. material/referenced in article.

Additional information

No additional information is available for this paper.

Declaration of competing interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

1. Fang E, Liu X, Li M, Zhang Z, Song L, Zhu B, et al. Advances in COVID-19 mRNA vaccine development. Signal Transduction and Targeted Therapy. 2022;7(1) [Crossref][PubMed][Google Scholar]

2. Hussain A, Yang H, Zhang M, Liu Q, Alotaibi G, Irfan M, et al. mRNA vaccines for COVID-19 and diverse diseases. Journal of Controlled Release. 2022;345:314-33 [Crossref][PubMed][Google Scholar]

3. Hromić-Jahjefendić A, Sezer A, Aljabali AAA, Serrano-Aroca Á, Tambuwala MM, Uversky VN, et al. COVID-19 Vaccines and Myocarditis: An Overview of Current Evidence. Biomedicines. 2023;11(5) [Crossref][PubMed][Google Scholar]

4. Cooper Leslie T. Myocarditis. New England Journal of Medicine. 360(15):1526-38 [Crossref][PubMed][Google Scholar]

5. Ishisaka Y, Watanabe A, Aikawa T, Kanaoka K, Takagi H, Wiley J, et al. Overview of SARS-CoV-2 infection and vaccine associated myocarditis compared to non-COVID-19-associated myocarditis: A systematic review and meta-analysis. International Journal of Cardiology. 2024;395 [Crossref][PubMed][Google Scholar]

6. Alami A, Krewski D, Farhat N, Mattison D, Wilson K, Gravel CA, et al. Risk of myocarditis and pericarditis in mRNA COVID-19-vaccinated and unvaccinated populations: a systematic review and meta-analysis. BMJ Open. 2023;13(6) [Crossref][PubMed][Google Scholar]

7. Gao J, Feng L, Li Y, Lowe S, Guo Z, Bentley R, et al. A Systematic Review and Meta-analysis of the Association Between SARS-CoV-2 Vaccination and Myocarditis or Pericarditis. American Journal of Preventive Medicine. 2023;64(2):275-84 [Crossref][PubMed][Google Scholar]

8. Yasuhara J, Masuda K, Aikawa T, Shirasu T, Takagi H, Lee S, et al. Myopericarditis After COVID-19 mRNA Vaccination Among Adolescents and Young Adults. JAMA Pediatrics. 2023;177(1) [Crossref][PubMed][Google Scholar]

9. Naveed Z, Li J, Spencer M, Wilton J, Naus M, García HAV, et al. Observed versus expected rates of myocarditis after SARS-CoV-2 vaccination: a population-based cohort study. Canadian Medical Association Journal. 2022;194(45):E1529-E36 [Crossref][PubMed][Google Scholar]

10. Naveed Z, Li J, Wilton J, Spencer M, Naus M, Velásquez García HA, et al. Comparative Risk of Myocarditis/Pericarditis Following Second Doses of BNT162b2 and mRNA-1273 Coronavirus Vaccines. Journal of the American College of Cardiology. 2022;80(20):1900-8 [Crossref][PubMed][Google Scholar]

11. Ling RR, Ramanathan K, Tan FL, Tai BC, Somani J, Fisher D, et al. Myopericarditis following COVID-19 vaccination and non-COVID-19 vaccination: a systematic review and meta-analysis. The Lancet Respiratory Medicine. 2022;10(7):679-88 [Crossref][PubMed][Google Scholar]

12. Cordero A, Cazorla D, Escribano D, Quintanilla MA, López-Ayala JM, Berbel PP, et al. Myocarditis after RNA-based vaccines for coronavirus. International Journal of Cardiology. 2022;353:131-4 [Crossref][PubMed][Google Scholar]

13. Guo B-Q, Li H-B, Yang L-Q. Incidence of myopericarditis after mRNA COVID-19 vaccination: A meta-analysis with focus on adolescents aged 12–17 years. Vaccine. 2023;41(28):4067-80 [Crossref][PubMed][Google Scholar]

14. Chang Y, Lv G, Liu C, Huang E, Luo B. Cardiovascular safety of COVID-19 vaccines in real-world studies: a systematic review and meta-analysis. Expert Review of Vaccines. 2022;22(1):25-34 [Crossref][PubMed][Google Scholar]

15. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021 [Crossref][PubMed][Google Scholar]

16. Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evidence Synthesis. 2020(10):2127-2133 [Crossref][PubMed][Google Scholar]

17. Chapter 7: Systematic reviews of etiology and risk. JBI Manual for Evidence Synthesis 2020. . [Crossref][PubMed][Google Scholar]

18. Haddaway NR, Page MJ, Pritchard CC, McGuinness LA. PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews. 2022;18(2):e1230 [Crossref][PubMed][Google Scholar]

19. Klein NP, Lewis N, Goddard K, Fireman B, Zerbo O, Hanson KE, et al. Surveillance for Adverse Events After COVID-19 mRNA Vaccination. JAMA. 2021;326(14) [Crossref][PubMed][Google Scholar]

20. Oster ME, Shay DK, Su JR, Gee J, Creech CB, Broder KR, et al. Myocarditis Cases Reported After mRNA-Based COVID-19 Vaccination in the US From December 2020 to August 2021. JAMA. 2022;327(4) [Crossref][PubMed][Google Scholar]

21. Wong H-L, Hu M, Zhou CK, Lloyd PC, Amend KL, Beachler DC, et al. Risk of myocarditis and pericarditis after the COVID-19 mRNA vaccination in the USA: a cohort study in claims databases. The Lancet. 2022;399(10342):2191-9 [Crossref][PubMed][Google Scholar]

22. Husby A, Hansen JV, Fosbøl E, Thiesson EM, Madsen M, Thomsen RW, et al. SARS-CoV-2 vaccination and myocarditis or myopericarditis: population based cohort study. BMJ. 2021 [Crossref][PubMed][Google Scholar]

23. Massari M, Spila Alegiani S, Morciano C, Spuri M, Marchione P, Felicetti P, et al. Postmarketing active surveillance of myocarditis and pericarditis following vaccination with COVID-19 mRNA vaccines in persons aged 12 to 39 years in Italy: A multi-database, self-controlled case series study. PLOS Medicine. 2022;19(7):e1004056 [Crossref][PubMed][Google Scholar]

24. Abraham N, Spruin S, Rossi T, Fireman B, Zafack J, Blaser C, et al. Myocarditis and/or pericarditis risk after mRNA COVID-19 vaccination: A Canadian head to head comparison of BNT162b2 and mRNA-1273 vaccines. Vaccine. 2022;40(32):4663-71 [Crossref][PubMed][Google Scholar]

25. Dickerman BA, Madenci AL, Gerlovin H, Kurgansky KE, Wise JK, Figueroa Muñiz MJ, et al. Comparative Safety of BNT162b2 and mRNA-1273 Vaccines in a Nationwide Cohort of US Veterans. JAMA Internal Medicine. 2022;182(7):739-46 [Crossref][PubMed][Google Scholar]

26. Krewski D, Saunders-Hastings P, Larkin P, Westphal M, Tyshenko MG, Leiss W, et al. Principles of risk decision-making. Journal of Toxicology and Environmental Health, Part B. 2022;25(5):250-78 [Crossref][PubMed][Google Scholar]

27. Navar AM, Bonow RO. Communicating the Benefits of Vaccination in Light of Potential Risks. JAMA Cardiology. 2022;7(6) [Crossref][PubMed][Google Scholar]

28. Mazagatos C, Monge S, Olmedo C, Vega L, Gallego P, Martín-Merino E, et al. Effectiveness of mRNA COVID-19 vaccines in preventing SARS-CoV-2 infections and COVID-19 hospitalisations and deaths in elderly long-term care facility residents, Spain, weeks 53 2020 to 13 2021. Eurosurveillance. 2021;26(24) [Crossref][PubMed][Google Scholar]

29. Voleti N, Reddy SP, Ssentongo P. Myocarditis in SARS-CoV-2 infection vs. COVID-19 vaccination: A systematic review and meta-analysis. Frontiers in Cardiovascular Medicine. 2022;9 [Crossref][PubMed][Google Scholar]

Disclaimer / Publisher’s Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of Journals and/or the editor(s). Journals and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.