research limitations due to covid 19

• Open access
• Published: 25 January 2023

Limitations of COVID-19 testing and case data for evidence-informed health policy and practice

• Elizabeth Alvarez   ORCID: orcid.org/0000-0003-2333-0144 1 ,
• Iwona A. Bielska 1 ,
• Stephanie Hopkins 1 ,
• Ahmed A. Belal 1 ,
• Donna M. Goldstein 2 ,
• Jean Slick 3 ,
• Sureka Pavalagantharajah 4 ,
• Anna Wynfield 2 ,
• Shruthi Dakey 5 ,
• Marie-Carmel Gedeon 6 ,
• Edris Alam 7 &
• Katrina Bouzanis 8  

Health Research Policy and Systems volume  21 , Article number:  11 ( 2023 ) Cite this article

8723 Accesses

11 Citations

6 Altmetric

Metrics details

Coronavirus disease 2019 (COVID-19) became a pandemic within a matter of months. Analysing the first year of the pandemic, data and surveillance gaps have subsequently surfaced. Yet, policy decisions and public trust in their country’s strategies in combating COVID-19 rely on case numbers, death numbers and other unfamiliar metrics. There are many limitations on COVID-19 case counts internationally, which make cross-country comparisons of raw data and policy responses difficult.

Purpose and conclusions

This paper presents and describes steps in the testing and reporting process, with examples from a number of countries of barriers encountered in each step, all of which create an undercount of COVID-19 cases. This work raises factors to consider in COVID-19 data and provides recommendations to inform the current situation with COVID-19 as well as issues to be aware of in future pandemics.

Peer Review reports

Since the emergence of coronavirus disease 2019 (COVID-19) in Wuhan, China, the world has faced serious data issues, ranging from a lack of transparency on the emergence, spread and nature of the virus to an absence of grounded comparative analyses, with temporal differences considered, about emerging social and economic challenges [ 1 , 2 ]. Most critically, scientists have lacked data to conduct analyses on non-pharmaceutical interventions (NPIs), including policies and strategies that governments have engaged to mitigate the situation, and how these have varied across regions, presumably affecting both short- and long-term outcomes [ 1 , 2 ].

Out of all the strategies implemented to date, physical distancing policies have emerged as one of the more effective NPIs to battle COVID-19 [ 3 , 4 ]. While physical distancing policies have been the mainstay in the battle against COVID-19, there has been a call to understand which forms of physical distancing policies are effective so that targeted and less disruptive measures can be taken in further waves of this pandemic and future pandemics [ 1 , 2 , 5 , 6 ]. The best time to institute physical distancing policies and what happens when and how they are eased remain unclear. There are many aspects of distancing, such as recommendations for maintaining a physical distance in public, banning group gatherings (the maximum number and where they take place), or complete lockdowns, that complicate their assessment. Timing and synergies of policies and sociodemographic and political factors play a role in the effectiveness of these policies [ 7 , 8 , 9 , 10 , 11 , 12 , 13 ]. Some hypothesized sociodemographic factors for increased exposure and severity of COVID-19 include living in a long-term care facility or being institutionalized, age (older), gender (mixed findings), having comorbidities (including high blood pressure, diabetes, obesity, immunocompromised status, tobacco smoking) and social vulnerabilities including race or ethnicity. Also relevant is the carrying capacity and infrastructure of health systems. These factors pose challenges for comparison among countries. Comparison is a prime requisite for evaluating the effectiveness of implementation of various policies between countries. Policymakers and the public have been using metrics such as number of cases, number of deaths and testing capacity to make policy or programme decisions or to decide whether to trust the actions of their governments, respectively.

An international team of researchers has been collecting data on physical distancing policies and contextual factors, such as health and political systems and demographics, to expedite knowledge translation (which means applying high-quality research evidence to processes of decision making) on the effect of policies and their influence on the epidemiology of COVID-19 [ 14 , 15 , 16 ]. Through this work, we identified gaps in the accuracy of reported numbers of COVID-19 cases and deaths, which make cross-country comparisons of the raw data, indexes using the raw data, and policy outcomes challenging [ 7 , 17 ]. While the work of this team is ongoing, this paper limits the findings from the inception of the pandemic to the end of 2020. It is important to understand the limitations of available COVID-19 data in order to properly inform decision making, especially at the outset as a novel infectious disease. This paper focuses on the testing and reporting cycle (Fig.  1 ) and provides examples from a number of countries of possible barriers leading to inaccurate data on reported COVID-19 cases. It also describes other cross-cutting implications of COVID-19 data for policy, practice and research, including reported deaths, missing information, implementation of policy, and unpredictable population behaviour. Furthermore, it calls into question analyses performed to date, which do not account for a number of known data gaps.

COVID-19 testing and reporting cycle. *The icons in this figure are in the public domain (Creative Commons CC0 1.0 Universal Public Domain) and were obtained from Wikimedia Commons at: https://commons.wikimedia.org/wiki/File:Medical_Library_-_The_Noun_Project.svg ; https://commons.wikimedia.org/wiki/File:Home_(85251)_-_The_Noun_Project.svg ; https://commons.wikimedia.org/wiki/File:Laboratory_-_The_Noun_Project.svg ; https://commons.wikimedia.org/wiki/File:Noun_project_1063.svg ; https://commons.wikimedia.org/wiki/File:Analysis_-_The_Noun_Project.svg

It is important to note that Fig.  1 only represents the testing and reporting cycle, which leads to counting of cases, and it does not include COVID-19 contact tracing and case management; however, we recognize that testing, contact tracing and case management are intricately linked to each other in the spread of COVID-19 [ 18 , 19 ]. As ‘Our World in Data’ states, “Without testing there is no data.” [ 20 ]. Understanding the links between testing, data and action underlies country responses to the pandemic. Ultimately, this work serves to provide a basis to improve pandemic planning, surveillance and reporting systems, and communications.

In Fig.  1 , the first level of testing is at the healthcare recipient level (Sect. “ Healthcare recipient level ”), followed by sample collection and processing (Sect. “ Sample collection and processing ”) and surveillance and reporting (Sect. “ Surveillance and reporting ”). Each level will be further explained below and examples provided as to potential or actual barriers at each level. These descriptions are not exhaustive, and nuanced understanding of the context will be needed to evaluate these steps and potential barriers in different settings.

Healthcare recipient level

Testing starts with individuals getting tested. There may be times when it is predetermined who gets tested and when, such as health workers getting tested prior to starting work in a long-term care facility or travellers returning from overseas [ 21 , 22 ]. However, most individuals are tested in the community, where a number of steps predicate individuals’ decisions to seek out testing. First, case definition, testing criteria and referral for testing influence our understanding of what the disease entity is and whether people are encouraged or discouraged to get tested. Given the novel status of COVID-19, there were challenges at the onset of this pandemic in establishing a working case definition. In China, arguably the leader in COVID-19 knowledge at the time, the case definition for reporting changed over time and between places [ 23 ]. These definitions were not always consistent with one another. Between 22 January and 12 February 2020, China’s National Health Commission had revised the COVID-19 outbreak response guidelines at least six times, resulting in significant differences in the daily counts due to changes over time in the definition of a case [ 24 ]. Adding to the uncertainty, the World Health Organization did not publish case definition guidelines until 16 April 2020, long after many countries had created their own working case definitions [ 25 ]. Although changes in methodology are expected as we learn more about the disease and as new variants emerge, these changes have implications for case counts [ 23 , 25 ]. Yet, communication does need to be flexible during a crisis. For example, little was known about asymptomatic COVID-19 spread at the beginning of the pandemic. As more evidence was garnered on this topic, information about precautions and testing criteria needed to be flexible to keep up with what was known [ 26 ].

Not only have case definitions changed over time, criteria for testing have changed over time and across jurisdictions on the basis of a number of factors, such as better understanding of the disease process, availability and capacity for testing, and national and local strategies for addressing the pandemic [ 27 ]. In some places, testing criteria were narrow, which discouraged people from getting tested because they did not fit the criteria. In its early response, Canada only tested symptomatic people returning from specific countries known to have high numbers of cases of COVID-19 [ 18 ]. Given that there were no treatments and media reported that the hospitals were overwhelmed, people were also discouraged from seeking medical attention unless they warranted hospitalization. If people were feeling unwell, but not needing to be on a ventilator, testing might not have been deemed necessary. Shifting testing criteria and differences in referral channels for testing, such as going through public health or needing a physician referral versus self-referrals, could create additional barriers.

Depending on the testing strategy, whether based on specific criteria or population-based, will make a difference for number of COVID-19 cases identified. Changes in criteria for testing sometimes led to increased demand without a corresponding increase in the availability of testing resources, which then led to delays in accessing tests [ 28 ]. Additionally, as different sectors, such as schools, resumed in-person activity, there was an increased demand for testing within certain population groups. Again, testing capacity could not always keep up with demand, leading, in some instances, to further limitations of who could be tested to prioritize resources for testing [ 27 ].

In the case that an individual has a choice to get tested, once a person is determined to be eligible for testing, that person has to decide whether or not to get tested, following a decisional process for getting tested, which can be affected by factors such as the availability of education and decision supports, health literacy, health status, trade-offs between knowing their results and potential economic and social consequences, health system complexity, and personal costs, such as time and out-of-pocket expenses [ 29 , 30 ]. Availability of education and decision support is needed for people to understand that there is a pandemic, what that means, how it might impact them, how and where to get tested (if available) and why getting tested is important for them or their loved ones. This relies on accurate and timely information, which is discussed in more detail in Sect. “ Governance and knowledge translation ”.

Furthermore, health literacy can involve a general understanding of factors that affect health or it can be specific to a disease entity, such as the virus that causes COVID-19. Health status can decrease the number of people seeking testing if they have mild symptoms and decide it is not worthwhile to seek testing or care, or they may not fit the testing criteria. On the other hand, some people with severe symptoms may not have the physical resources to go to a testing centre.

Of course, even individual-level factors are affected by broader systematic determinants of health. As the gravity of the pandemic took hold, jurisdictions began implementing more robust isolation policies to prevent the spread of COVID-19. These policies included self-isolation or a quarantine period for those who tested positive or who had come in contact with a known case. In many countries, governments provided economic relief to support people who were unable to work [ 31 , 32 ]. However, in countries such as Brazil and Mexico there were limited social safety nets, and in many other countries such as the USA, COVID-19 exposed gaps in these nets [ 33 , 34 ]. This created an economic barrier for people to access testing, as a positive test would force them to stay home without adequate financial means to survive. On 28 April 2020, the French Prime Minister, Edouard Philippe, urged the population of France to “protect-test-isolate”; meanwhile, containment measures generated a “disaffiliation process” among migrants and asylum seekers. Absence of work, isolation from French society, and fear of being checked by the police brought individuals into a “disaffiliation zone” marked by social non-existence, in a context of global health crisis [ 35 ].

Health systems themselves created a barrier to testing through their slow response to testing requests, causing some individuals to abandon testing [ 36 ]. In some countries, testing was expensive and not offered in the poorest communities [ 37 ]. For those travelling, mandatory testing, with varying requirements between different countries and potential out-of-pocket costs, increased the complexity of getting tested. Furthermore, competing crises may have lowered the number of people seeking testing due to other, more immediate, priorities, such as floods or wildfires [ 38 , 39 , 40 , 41 ].

Sample collection and processing

Once a person decides to seek testing, tests must be available and accessible and there must also be sufficient test processing centres. While these factors are often lumped together, it is important to distinguish these two steps in the testing cycle as they often require different structural and/or operational components.

Tests and testing sites

For an individual to get tested, there must be availability of testing sites and accessibility to these testing sites. Testing sites may include already available clinic, hospital or community sites, or assessment centres which are created for the purpose of testing. Having separate assessment centres can ease conflicting burdens on already overwhelmed health systems, and they can allow for efficiency in the process of testing and in keeping potentially infectious individuals separate from those who are seen for other ailments. Not only do testing sites have to be available, they have to be accessible. Times of operation, parking and other accessibility considerations are important. Testing sites can be centralized in one or several locations, where people have to find transportation to the sites, or can be mobile sites, which can increase access to those in rural/remote areas or those with mobility or transportation issues. Drive-thru testing has been showcased in countries such as South Korea [ 42 ]. However, limitations also exist with drive-thru sites for those who do not own a vehicle, or those who have to drive long distances or endure long wait times [ 43 ]. In areas with poor health system infrastructure, lack of access can exacerbate inequities in testing.

Operational components include the need for adequate human resources and testing supplies. In Ontario, Canada, assessment centres were slow to set up and there was a lack of swabs and other testing supplies [ 18 ]. In France, laboratories struggled to keep up with testing demands due to delays in receiving chemicals and testing kits produced abroad, given France’s reliance on global supply chains [ 44 ]. Bangladesh had a very limited number of case testing capacity in the beginning of the outbreak. The country conducted fewer than 3000 tests in the first four weeks of the outbreak between 8 March and 5 April 2020 for its 164 million population as well as 155,898 overseas passengers, some arriving from hard-hit countries such as Italy, allowing for community transmission [ 45 ].

The method of specimen collection and specimen management for processing are also important considerations. Specimen collection has varied between contexts and over time [ 46 ]. Nasopharyngeal, nasal and throat swabs have been used in community settings. Saliva tests and blood samples, mainly for hospitalized patients, are other methods of obtaining specimens. Each of these testing modalities has different properties, but none is 100% sensitive or able to pick up all positive cases of COVID-19. There are reports of very ill patients testing negative on multiple occasions on nasopharyngeal samples but subsequently testing positive from lung samples [ 47 , 48 ]. Specimen management requires the proper labelling, storage and transportation of samples from the testing site to the laboratory for processing.

Laboratories

Laboratory preparedness and laboratory capacity played crucial roles in COVID-19 testing globally [ 27 , 49 ]. Issues with this preparedness and capacity, along with lack of testing supplies, resulted in “lack of testing” as a prime factor for not having accurate numbers of COVID-19 cases, especially at the beginning of the pandemic. Laboratory capacity includes human resources and specimen processing supplies, often called the testing kits, which require specific reagents and equipment. Over time, countries with low laboratory preparedness focused on improving their testing capacities [ 49 ]. Since the start of the pandemic, Germany was touted as testing widely and therefore having a robust ability to contact trace in order to find people who may transmit the virus causing COVID-19. However, other countries struggled to get testing in place. In the USA, initial tests developed were invalid, which delayed the ability to distribute and complete tests [ 50 ]. This was further exacerbated by bureaucratic/institutional red tape which centralized testing to the Centers for Disease Control and Prevention (CDC) and prohibited local public health and commercial laboratories from developing or administering more effective tests [ 51 ]. Supply chain management issues for swabs, transport media and reagents slowed down early testing in multiple countries [ 27 ].

Once testing methods have been established, there are a number of tests available for COVID-19 [ 52 ]. Test properties include the sensitivity and specificity of a test, among others, and these can vary by test. Therefore, the type of test used can also influence case counts. Recent studies have highlighted the need to validate laboratory tests and share the results during a pandemic. Evidence from a study in Alberta, Canada suggested that variations in test sensitivity for the virus causing COVID-19, particularly earlier in a pandemic, can result in “an undercounting of cases by nearly a factor of two” (p. 398) [ 53 ]. With rapid tests and home-approved testing kits available during the course of the pandemic, testing properties can vary even more greatly [ 52 , 54 , 55 ].

Surveillance and reporting

Once individuals have been tested and the results are processed, surveillance and reporting systems must be in place to communicate that information back to individuals, public health officials or others involved in case management or treatment, and to politicians and other stakeholders to act on this information and prevent further spread.

Data systems

Data management refers to the inputting and tracking of data. However, because of the need to quickly and accurately inform the public and decision makers in the time of a crisis, coordination of information technology is needed to align all the various data management systems within a jurisdiction and internationally. For example, each hospital system, clinic or laboratory may have separate electronic medical record or data management systems. Not many countries maintain a common database system for COVID-19-related management (testing, response, etc.). Even if database management systems are in place, lack of trained professionals, serious lags in updating data, challenges with interdepartmental coordination among various task force members, and new innovations such as artificial intelligence, health tracking apps, telemedicine and big data, which are suddenly in place, can lead to disrupted transparency. An exception is China, which developed a highly responsive national notifiable disease reporting system (NNDRS) in the aftermath of severe acute respiratory syndrome (SARS) [ 56 , 57 ]. The United Nations Department of Economic and Social Affairs statistics division launched a common website for improving the data capacities of countries [ 58 ]. This information has to be further coordinated to create larger and more robust surveillance and notification systems. Robust surveillance systems help decision makers know what is happening locally or how a disease is moving through populations. Notification systems are needed for sharing information between the testing site, laboratory and public health or local health agencies for case management and contact tracing and for letting people know their test results in a timely manner to help prevent further spread. The COVID Tracking Project has highlighted many discrepancies in USA reporting and surveillance, demonstrating unreliability of the data [ 59 ]. For example, hospitals were required to change how COVID-19 data were relayed to the federal government, and the switch from reporting through the CDC to the Health and Human Services (HHS) system resulted in misreporting of data and administrative lags across several states. Countries’ national-level CDCs collect information from state and local sources. The time lag can hence be one of the reasons for misleading the overall comprehensive pandemic impact. Lastly, with rapid, point-of-care and home tests available, keeping track of positive cases may be even more difficult, and COVID-19 case counts could be even further artificially decreased [ 60 ]. These tests could make contact tracing even more difficult if there is a lack of disclosure from the user end. It is important to note that, while there are many available sites for international COVID-19 data comparisons, including John’s Hopkins COVID-19 Dashboard [ 61 ], Worldometer [ 62 ], Our World in Data [ 20 ] and the World Health Organization (WHO) COVID-19 Dashboard [ 63 ], these all rely on locally-acquired data for their reporting, and therefore fall into and potentially augment the same fallacies discussed in this paper.

Governance and knowledge translation

Even with robust surveillance and notification systems, transparency and accountability are important for informing decision makers and the public. Decision makers need to know what the health and laboratory systems are finding so that evidence-informed policy and practice decisions can be made for the public good. At the same time, trust in government and government responses rely in part on perceived transparency of government by the public [ 64 , 65 ]. Accountability spans all through the spectrum discussed in the testing and reporting cycle, in a whole-of-society approach. Individuals are accountable for knowing when to get tested, getting tested and following public health guidelines and other policies. The public health and healthcare systems are accountable for planning testing and sharing information. Decision makers are accountable for transparency in sharing information, communicating appropriately with the public and relevant stakeholders, and making decisions for those they represent. In parts of Russia, there were two separate reports for those who died from COVID-19 and those who were positive but died from other causes [ 25 ]. In Florida, state officials instructed medical examiners to remove causes of death in their lists [ 66 ]. In China, despite having a highly responsive national data surveillance and reporting system, at the beginning of the pandemic, cases were only reported to the system once they had been approved by local members of government who only allowed cases with a direct connection to the original source of the outbreak, the seafood market, to be recorded [ 67 ].

Political will has been shown to be a barrier or facilitator in the fight against COVID-19. Examples of good leadership and political will can be found in places like New Zealand, where decisions were made early on, implemented, supported and continued to be informed by emerging evidence, or as described, following “science and empathy”[ 68 ]. Poor leadership has also come through clearly during this pandemic. Tanzania, Iran, the USA, Brazil and Egypt are only a handful of countries demonstrating the impact of political will on the course of the pandemic, in some cases resulting from subversion and corruption. Communication in these countries was often not transparent or mixed, and accountability for the lack of decision making or poor decision making was limited or non-existent in the pandemic’s outset. Tanzania stopped reporting cases due to political optics [ 30 , 69 ]. Iran’s Health Ministry reported 14,405 deaths due to COVID-19 through July 2020, which was a significant discrepancy from the 42,000 deaths recorded through government records [ 70 ]. The number of cases was also almost double those reported, 451,024 as compared with 278,827. One main reason for releasing underestimated information about the cases was considered to be upcoming parliamentary elections [ 70 , 71 ]. The former president of the USA, Donald Trump, often flouted public health and healthcare expert advice [ 72 ]. The Washington Post reported that Brazil was testing 12 times fewer people than Iran and 32 times fewer people than the USA, and hospitalized patients and some healthcare professionals were not tested in an effort to lower the case numbers [ 73 ]. Hiding numbers of deaths from COVID-19, whether intentionally or inadvertently, shored up far-right supporters of Brazil’s President Bolsonaro at a time when he was facing possible charges of impeachment for corruption and helped bolster the President’s messaging that the pandemic was under control. This further enabled a large swath of the population to call for less strict rules around COVID-19 and a quick reopening of the economy. Similarly, in July 2020, it was reported that at least eight doctors and six journalists had been arrested because they criticized the Egyptian government’s response to the pandemic [ 74 ].

Lastly, communication and information dissemination link to every piece of this process. Why, when and how people seek testing, how and where to set up testing sites, supply chain management, setting up and managing data systems, and policy decision making all work in a cycle. Good communication between systems and dissemination of information to the public and relevant stakeholders is imperative during a crisis, such as the COVID-19 pandemic. The amount of information available and rapid change in information creates an infodemic problem. ‘Infodemic’ is a term used by the WHO in the context of COVID-19 and refers to informational problems, such as misinformation and fake news, that accompany the pandemic [ 75 ]. Addressing the infodemic issue was highlighted as one of the prominent factors needed to improve future global mitigation efforts [ 76 ]. A report published in the second week of April 2020 by the Reuters Institute for the Study of Journalism at the University of Oxford found that roughly one-third of social media users across the USA, as well as Argentina, Germany, South Korea, Spain and the UK, reported seeing false or misleading information about COVID-19 [ 77 ]. The presidents of Brazil and the USA were themselves sources of misinformation, as they were seen in public without masks and touting the benefits of hydroxychloroquine after it was largely known that harms outweighed benefits of its use [ 72 , 78 , 79 ]. Having clear public health communications, from trusted sources, and breaking down silos between systems could be helpful in combating ever-changing information during a pandemic.

Other implications of COVID-19 data for policy, practice and research

There are several cross-cutting issues separate, but related, to the testing and reporting cycle which arose during this work. These issues also affect COVID-19 case counts and optimal timing of policies: how deaths are reported, missing information, implementation of policies, and unpredictable population behaviour.

Reported deaths

Deaths from COVID-19 tend to occur weeks after infection; therefore, assessments of policy changes using death counts need to account for this timing. However, reported death counts from COVID-19 carry many similar limitations given lack of testing for those who are deceased, attributing cause of death to COVID-19-related complications, processes for declaring deaths and causes of deaths, and lack of transparency [ 80 ]. In Brazil, hospitalized patients were not being tested, and deaths were attributed to respiratory ailments [ 73 ]. Further, COVID-19 deaths from the City of Rio de Janeiro’s dashboard were blacked out for 4 days in May (22–26 May 2020) [ 81 ]. When the dashboard was restarted, the death count was artificially lowered by changing the cause of death from COVID-19 to its comorbidities. Additional changes included requiring a confirmed COVID-19 test at the time of death in order for the death certificate to list COVID-19 as the cause; however, the results of the test often came after the death certificates were issued [ 81 ]. In Italy, the reverse occurred where only those in hospital were counted as COVID-19 deaths, while many people died at home or in care homes without being tested [ 82 , 83 ]. In Ireland, early discrepancies in reported deaths were noted between official government figures and an increase in deaths noted on the website Rip.ie, which has served as a public forum disclosing deaths and wake information in line with Irish funeral traditions. Information from this forum was used to re-assess mortality and in some cases aid epidemiological modelling [ 84 ].

Missing information

Given the lack of access to treatments at the beginning of the pandemic, understanding who was at highest risk of obtaining or dying from COVID-19 was important to know in order to develop appropriate policies that balanced health with social and economic impacts of the pandemic. Early data showed a sex and age gradient for COVID-19 cases and deaths. However, not all countries report data by sex and/or age. Race/ethnicity and sociodemographic findings were not collected or reported early in the pandemic [ 85 ]. France has been criticized for laws which prohibit the collection of race and ethnicity data, since they lack data which demonstrate whether certain groups are overrepresented in COVID-19 cases and deaths [ 86 , 87 ]. Another aspect of missing data early in the pandemic was that of asymptomatic spread. Due to limited testing early in the pandemic, asymptomatic cases were not picked up. Population-based studies are being conducted to better understand the role of asymptomatic and pre-symptomatic spread of COVID-19 in different population groups, such as children [ 26 ].

Implementation of policy

Population-level strategies since the start of the pandemic and reported findings in the literature go hand and hand. Cause and effect are difficult to attribute. For example, early literature looking at the role of children on the spread of COVID-19 found that children played a small role. This was to be expected given that many schools around the world closed, and children would not be exposed through transportation and workplaces as adults would be. Therefore, family spread would naturally flow from adults to children given these circumstances. In addition, many places were not testing mild to asymptomatic cases, which were more commonly found in children. Publications early on related to the few severe COVID-19 cases in children or to school-related cases in places that had low community transmission rates of COVID-19 and were following public health guidelines [ 88 ]. Limitations of these data have been described, yet findings have been used to justify specific policies in places that were dissimilar, with expected results ensuing, such as an increase in community transmission and school closures due to COVID-19 infections [ 89 ]. Therefore, it is even more important to understand the context of policies before applying them to various jurisdictions.

Unpredictable population behaviour

There is a difference between stated policy, implementation and enforcement. To understand which policies worked to combat COVID-19, it is important to consider the level of compliance with stated policies. Some people may follow recommended approaches for protective actions while others may not comply and see these recommendations as problematic [ 90 ]. For example, people may change their behaviours in anticipation of an announced change; for example, individuals may start working from home even before it is enforced or if it is never officially mandated, or people may go on a shopping spree prior to known closures [ 91 , 92 ]. Of course, people’s behaviours may also be dependent on a disconnect between policy messages at different levels of government and exacerbated by rapid updates in a fast-moving pandemic of unknown properties and the associated information overload. Therefore, communication management and clarity are of utmost importance during a crisis.

Discussion and recommendations

The need for cross-country comparison is necessary for understanding the effectiveness of policies in various countries. Policy decisions are being made and judged on the basis of case numbers, deaths and testing, among others. Understanding the steps and barriers in testing and reporting data related to COVID-19 case numbers can help address the limitations of data to strengthen these systems for future pandemics and can also help in the interpretation of findings across jurisdictions. Robust and timely public health measures are needed to decrease the health, social and economic ramifications of the pandemic. Even with available vaccines, it will still take time to have sufficient population coverage internationally.

There are a few assumptions considered in this paper. First, we assume that the reported numbers for each country are not inflated. There could be some cases that are counted more than once if repeated tests are taken and the person continues to test positive. Most data do not disclose how often this occurs, but it is likely not a significant issue for population reports, at least at from the beginning of the pandemic [ 20 ]. Next, ideally COVID-19 case counts are accurate. This is the assumption that is made by policymakers and the public in judging their decisions and their outcomes. We argue that the reported COVID-19 data are likely an undercount of actual cases. The reasons are highlighted in this paper.

Future global discussions will continue around who is most affected by COVID-19 and how to best prepare for pandemics, among others. COVID-19 case and death counts will be used in determining successful approaches. It is important to understand the context of COVID-19 data in these discussions, especially with respect to other global indicators that may look to COVID-19 data, such as the Sustainable Development Goals (SDGs) through improvement of early warning, risk reduction and management of national and global health risks [ 93 ]. Specifically, SDG 3 (good health and wellbeing) with an emphasis on highlighting the lacunas in informed data tying policy and epidemiology, SDG 10 to reduce inequalities within and among countries, and SDG 16 (peace, justice and strong institutions) with a goal to build effective and accountable institutions at all levels. This research also contributes to the Sendai Framework for Disaster Risk Reduction, specifically priority 2, strengthening disaster risk governance to manage disaster risk, and priority 3, investing in disaster risk reduction for resilience [ 94 ]. Unfortunately, there is little published on good governance in reporting systems during COVID-19, and our findings in this area are limited to media and news sources. Future research could focus on this critical aspect.

Decision makers could consider the following overarching recommendations, contextualized to their individual jurisdictions (i.e. regional, country, province, territory, state), to evaluate the testing and reporting cycle and improve accuracy and comparability of COVID-19 data:

Understand barriers to accurate testing and reporting —This paper lays out the steps in the testing and reporting process and components of these steps. Barriers are described at each of these steps, and examples are provided.

Address barriers to testing and reporting —Understanding barriers in the testing and reporting process can uncover facilitators. Each setting will deal with different barriers. Ultimately, political will, capacity building and robust information systems will be needed to address any of these barriers.

Transparency and accountability for surveillance and reporting —Any attempt to assign causality to these policies must take into account the timing and quality of surveillance data. Data quality issues, such as completeness, accuracy, timeliness, reliability, relevance and consistency, are important for surveillance and reporting [ 95 , 96 ].

Invest in health system strengthening, including surveillance and all-hazards emergency response plans —COVID-19, as this and past pandemics have shown, is not just a health issue, and instead requires community, health systems, social systems and policy approaches to mitigate its effects. Preparing for infectious disease outbreaks and other crises needs to incorporate all-hazards emergency response plans in order to have all the necessary resources in place at the time of the events.

Identify promising communication strategies —Research is needed to understand how messages conveyed at all stages of a pandemic are received and understood at the micro-level and used by the public [ 97 ]. Development of communication strategies aimed at promoting good understanding of information may defer inappropriate behaviours.

Invest in research to further understand data reporting systems and policy strategies and implementation . Research could compare global COVID-19 data reporting platforms mentioned in this article to see from where they obtained their raw data to further understand data reporting accuracy and comparability of data over time and whether any limitations of data were noted. Further research could address what policy and implementation strategies worked in a variety of settings to strengthen future recommendations for emerging pandemics.

The use and effectiveness of government responses, specifically pertaining to physical distancing policies in the COVID-19 pandemic, has been evolving constantly. Testing is a measure of response performance and becomes a focal point during an infectious disease pandemic as all countries are faced with a similar situation. COVID-19 represents a unique opportunity to evaluate and measure success by countries to control its spread and address social and economic impacts of interventions. Understanding limitations of COVID-19 case counts by addressing factors related to testing and reporting will strengthen country responses to this and future pandemics and increase the reliability of knowledge gained by cross-country comparisons. Alarmingly, with COVID-19 having asymptomatic spread, lack of testing can discredit the efforts of an entire community, not to say an entire population.

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Abbreviations

Centers for Disease Control and Prevention

Coronavirus disease 2019

Health and Human Services

National notifiable disease reporting system

Non-pharmaceutical interventions

Severe acute respiratory syndrome

Sustainable Development Goals

World Health Organization

Pueyo T. Coronavirus: the hammer and the dance. Medium. 2020. https://medium.com/@tomaspueyo/coronavirus-the-hammer-and-the-dance-be9337092b56 . Accessed 5 Sep 2020.

Ferguson N, Laydon D, Nedjati-Gilani G, et al. Report 9—impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand. WHO Collaborating Centre for Infectious Disease Modelling; MRC Centre for Global Infectious Disease Analysis; Abdul Latif Jameel Institute for Disease and Emergency Analytics; Imperial College London, UK; 2020. http://www.imperial.ac.uk/medicine/departments/school-public-health/infectious-disease-epidemiology/mrc-global-infectious-disease-analysis/covid-19/report-9-impact-of-npis-on-covid-19/ . Accessed 14 Dec 2020.

Chu DK, Akl EA, Duda S, Solo K, Yaacoub S, Schünemann HJ, et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. Lancet. 2020;395(10242):1973–87.

Article   CAS   Google Scholar  

Islam N, Sharp SJ, Chowell G, Shabnam S, Kawachi I, Lacey B, et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. BMJ. 2020;15(370): m2743.

Article   Google Scholar  

WHO. Archived: WHO Timeline—COVID-19. https://www.who.int/news/item/27-04-2020-who-timeline---covid-19 . Accessed 14 Dec 2020.

Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19). WHO; 2020. https://www.who.int/docs/default-source/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf . Accessed 7 May 2020.

Ho S. Breaking down the COVID-19 numbers: should we be comparing countries? CTV News. 2020. https://www.ctvnews.ca/health/coronavirus/breaking-down-the-covid-19-numbers-should-we-be-comparing-countries-1.4874552 . Accessed 14 Dec 2020.

D’Adamo H, Yoshikawa T, Ouslander JG. Coronavirus Disease 2019 in geriatrics and long-term care: the ABCDs of COVID-19. J Am Geriatr Soc. 2020;68(5):912–7.

Kluge HHP. Statement—older people are at highest risk from COVID-19, but all must act to prevent community spread. WHO, Regional Office for Europe. 2020. https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/statements/statement-older-people-are-at-highest-risk-from-covid-19,-but-all-must-act-to-prevent-community-spread . Accessed 14 Dec 2020.

Jin JM, Bai P, He W, Wu F, Liu XF, Han DM, et al. Gender differences in patients with COVID-19: focus on severity and mortality. Front Public Health. 2020. https://doi.org/10.3389/fpubh.2020.00152/full .

Canadian Institutes of Health Research. Why sex and gender need to be considered in COVID-19 research—CIHR. Government of Canada. 2020. https://cihr-irsc.gc.ca/e/51939.html . Accessed 14 Dec 2020.

Gaynor TS, Wilson ME. Social vulnerability and equity: the disproportionate impact of COVID-19. Public Adm Rev. 2020;80(5):832–8.

Karaye IM, Horney JA. The impact of social vulnerability on COVID-19 in the US: an analysis of spatially varying relationships. Am J Prev Med. 2020;59(3):317–25.

Alvarez E. Policy frameworks and impacts on the epidemiology of COVID-19. CONVERGE. 2020. https://converge.colorado.edu/resources/covid-19/working-groups/issues-impacts-recovery/policy-frameworks-and-impacts-on-the-epidemiology-of-covid-19 . Accessed 14 Dec 2020.

Alvarez E. Physical distancing policies and their effect on the epidemiology of COVID-19: a multi-national comparative study. World Pandemic Research Network. 2020. https://wprn.org/item/457852 . Accessed 14 Dec 2020.

COVID-19 Policies & Epidemiology Research Project. Home. https://covid19-policies.healthsci.mcmaster.ca/ . Accessed 20 Oct 2020.

CDC. Coronavirus Disease 2019 (COVID-19)—Transmission. Centers for Disease Control and Prevention. 2020. https://www.cdc.gov/coronavirus/2019-ncov/covid-data/faq-surveillance.html . Accessed 2 Sep 2020.

Crowe K. Why it’s so difficult to get tested for COVID-19 in Canada | CBC News. CBC News. 2020. https://www.cbc.ca/news/health/covid-testing-shortages-1.5503926 . Accessed 5 Sep 2020.

Confused about COVID-19 testing guidelines? Find out if you should get tested | CBC News. CBC News. 2020. https://www.cbc.ca/news/canada/toronto/covid-19-testing-ontario-1.5737683 . Accessed 14 Dec 2020.

Ritchie H, Ortiz-Ospina E, Beltekian D, Mathieu E, Hasell J, Macdonald B, et al. Coronavirus (COVID-19) Testing—our world in data. Our world in data. n.d. https://ourworldindata.org/coronavirus-testing . Accessed 14 Dec 2020.

Fox C. Workers at long-term care homes in COVID-19 hot spots will now be required to get tested weekly. CP24. 2020. https://www.cp24.com/news/workers-at-long-term-care-homes-in-covid-19-hot-spots-will-now-be-required-to-get-tested-weekly-1.5194380?cache=%3FclipId%3D89926%3FautoPlay%3DtrueSC . Accessed 8 Jan 2021.

Mzezewa T. Travel and coronavirus testing: your questions answered. The New York Times. 2020. https://www.nytimes.com/article/virus-covid-travel-questions.html . Accessed 8 Jan 2021.

Tsang TK, Wu P, Lin Y, Lau EHY, Leung GM, Cowling BJ. Effect of changing case definitions for COVID-19 on the epidemic curve and transmission parameters in mainland China: a modelling study. Lancet Public Health. 2020;5(5):e289–96.

Feuer W. “Confusion breeds distrust:” China keeps changing how it counts coronavirus cases. CNBC. 2020. https://www.cnbc.com/2020/02/26/confusion-breeds-distrust-china-keeps-changing-how-it-counts-coronavirus-cases.html . Accessed 14 Dec 2020.

Sauer P, Gershkovich E. Russia is boasting about low coronavirus deaths. The numbers are deceiving. The Moscow Times. 2020. https://www.themoscowtimes.com/2020/05/08/russia-is-boasting-about-low-coronavirus-deaths-the-numbers-are-deceiving-a70220 . Accessed 14 Dec 2020.

Gandhi M, Yokoe DS, Havlir DV. Asymptomatic transmission, the Achilles’ heel of current strategies to control COVID-19. N Engl J Med. 2020;382(22):2158–60.

Pabbaraju K, Wong AA, Douesnard M, Ma R, Gill K, Dieu P, et al. A public health laboratory response to the pandemic. J Clin Microbiol. 2020;58(8):e01110–20.

Sherriff-Scott I. Ontario is changing how it tests for COVID. Here’s what we know so far. iPolitics. 2020. https://ipolitics.ca/2020/10/05/ontario-is-changing-how-it-tests-for-covid-heres-what-we-know-so-far/ . Accessed 16 Dec 2020.

Cousins S. Bangladesh’s COVID-19 testing criticised. Lancet. 2020;396(10251):591.

Maclean R. COVID-19 outbreak in Nigeria is just one of Africa’s alarming hot spots. The New York Times. 2020. https://www.nytimes.com/2020/05/17/world/africa/coronavirus-kano-nigeria-hotspot.html . Accessed 30 Oct 2020.

Government of Singapore. Budget 2020. 2020. http://www.gov.sg/features/budget2020 . Accessed 8 Jan 2021.

Hapuarachchi P. President announces relief measures to the people amid COVID-19. 2020. https://www.newsfirst.lk/2020/03/23/president-grants-concessions-for-the-people-amid-covid-19/ . Accessed 8 Jan 2021.

Rauls L. COVID-19 is exposing the holes in Latin America’s safety nets. Americas Quarterly. 2020. https://americasquarterly.org/article/covid-19-is-exposing-the-holes-in-latin-americas-safety-nets/ . Accessed 8 Jan 2021.

Shafer P, Avila CJ. 4 ways COVID-19 has exposed gaps in the US social safety net. The Conversation. 2020. http://theconversation.com/4-ways-covid-19-has-exposed-gaps-in-the-us-social-safety-net-138233 . Accessed 16 Dec 2021.

Carillon S, Gosselin A, Coulibaly K, Ridde V, Desgrées du Loû A. Immigrants facing COVID 19 containment in France: an ordinary hardship of disaffiliation. J Migr Health. 2020;1–2: 100032.

Charbonneau D. Ottawa residents frustrated with multi-day wait for COVID-19 test results. CTV News. 2020. https://ottawa.ctvnews.ca/ottawa-residents-frustrated-with-multi-day-wait-for-covid-19-test-results-1.5122365 . Accessed 14 Dec 2020.

Faust L, Zimmer AJ, Kohli M, Saha S, Boffa J, Bayot ML, et al. SARS-CoV-2 testing in low- and middle-income countries: availability and affordability in the private health sector. Microbes Infect. 2020;22(10):511–4.

FloodList. India—over 60 dead after more rain and floods hit Telangana—FloodList. 2020. http://floodlist.com/asia/india-floods-telangana-october-2020 . Accessed 15 Dec 2020.

Al Jazeera. India: cyclone Nivar makes landfall bringing rains, flood | Weather News. 2020. https://www.aljazeera.com/news/2020/11/26/cyclone-nivar-makes-landfall-bringing-rains-flood . Accessed 15 Dec 2020.

Niles N, Hauck G, Aretakis R, Shannon J. Tropical storm sally: hurricane watch, evacuations for New Orleans; storm expected to strengthen. USA TODAY. 2020. https://www.usatoday.com/story/news/nation/2020/09/12/tropical-storm-update-florida-gulf-path-paulette-path-bermuda/5778321002/ . Accessed 15 Dec 2020.

Flaccus G, Cline S. Wildfires raging across Oregon, Washington may cause historic destruction: officials. Global News. 2020. https://globalnews.ca/news/7325865/oregon-washington-wildfires/ . Accessed 15 Dec 2020.

Watson I, Jeong S. South Korea pioneers coronavirus drive-through testing station—CNN. CNN . 2020. https://www.cnn.com/2020/03/02/asia/coronavirus-drive-through-south-korea-hnk-intl/index.html . Accessed 15 Dec 2020.

Herhalt C. “This is torture”: Ontario mom waits 6.5 hours for COVID-19 test. CTV News. 2020. https://toronto.ctvnews.ca/this-is-torture-ontario-mom-waits-6-5-hours-for-covid-19-test-1.5109045 . Accessed 15 Dec 2020.

Lough R. French labs show how global supply bottlenecks thwart effort to ramp up testing. Reuters. 2020. https://www.reuters.com/article/us-health-coronavirus-france-reagents-idINKCN26G2EP . Accessed 8 Jan 2021.

Huq S, Biswas RK. COVID-19 in Bangladesh: data deficiency to delayed decision. J Glob Health. 2020;10(1): 010342.

Alberta Precision Laboratories. Major changes in COVID-19 specimen collection recommendations. 2020. https://www.albertahealthservices.ca/assets/wf/lab/wf-lab-bulletin-major-changes-in-covid-19-specimen-collection-recommendations.pdf .

Ramos KJ, Kapnadak SG, Collins BF, Pottinger PS, Wall R, Mays JA, et al. Detection of SARS-CoV-2 by bronchoscopy after negative nasopharyngeal testing: stay vigilant for COVID-19. Respir Med Case Rep. 2020;30: 101120.

Google Scholar  

Chen LD, Li H, Ye YM, Wu Z, Huang YP, Zhang WL, et al. A COVID-19 patient with multiple negative results for PCR assays outside Wuhan, China: a case report. BMC Infect Dis. 2020;20(1):517.

Gupta N, Potdar V, Praharaj I, Giri S, Sapkal G, Yadav P, et al. Laboratory preparedness for SARS-CoV-2 testing in India: harnessing a network of virus research & diagnostic laboratories. Indian J Med Res. 2020;151(2 & 3):216–25.

CAS   Google Scholar  

Johnson CY, McGinley L. What went wrong with the coronavirus tests in the U.S. Washington Post. https://www.washingtonpost.com/health/what-went-wrong-with-the-coronavirus-tests/2020/03/07/915f5dea-5d82-11ea-b29b-9db42f7803a7_story.html . Accessed 5 Sep 2020.

Resnick B, Scott D. America’s shamefully slow coronavirus testing threatens all of us . Vox. 2020. https://www.vox.com/science-and-health/2020/3/12/21175034/coronavirus-covid-19-testing-usa . Accessed 14 Dec 2020.

Health Canada. Authorized medical devices for uses related to COVID-19: List of authorized testing devices. AEM. 2020. https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/medical-devices/authorized/list.html . Accessed 15 Dec 2020.

Burstyn I, Goldstein ND, Gustafson P. It can be dangerous to take epidemic curves of COVID-19 at face value. Can J Public Health. 2020;111(3):397–400.

Bachelet VC. Do we know the diagnostic properties of the tests used in COVID-19? A rapid review of recently published literature. Medwave. 2020;20(3): e7890.

Mina MJ, Parker R, Larremore DB. Rethinking COVID-19 test sensitivity— a strategy for containment. N Engl J Med. 2020;383(22): e120.

Jia P, Yang S. China needs a national intelligent syndromic surveillance system. Nat Med. 2020;26(7):990–990.

Jia P, Yang S. Early warning of epidemics: towards a national intelligent syndromic surveillance system (NISSS) in China. BMJ Glob Health. 2020;5(10): e002925.

United Nations Department of Economic and Social Affairs. Statistics, COVID-19 response. https://covid-19-response.unstatshub.org/https:/covid-19-response.unstatshub.org/ . Accessed 20 Oct 2022.

The Atlantic Monthly Group. The COVID tracking project. The COVID tracking project. https://covidtracking.com/ . Accessed 21 Oct 2021.

Erdman SL. FDA authorizes first rapid COVID-19 self-testing kit for at-home diagnosis. CNN. https://www.cnn.com/2020/11/18/health/covid-home-self-test/index.html . Accessed 15 Dec 2020.

Johns Hopkins Coronavirus Resource Center. COVID-19 Map. https://coronavirus.jhu.edu/map.html . Accessed 20 Oct 2022.

Worldometer. COVID Live—Coronavirus Statistics. https://www.worldometers.info/coronavirus/ . Accessed 20 Oct 2022.

World Health Organization. WHO Coronavirus (COVID-19) Dashboard. https://covid19.who.int . Accessed 20 Oct 2022.

Enria L, Waterlow N, Rogers NT, Brindle H, Lal S, Eggo RM, et al. Trust and transparency in times of crisis: results from an online survey during the first wave (April 2020) of the COVID-19 epidemic in the UK. PLoS ONE. 2021;16(2): e0239247.

OECD. Transparency, communication and trust: The role of public communication in responding to the wave of disinformation about the new Coronavirus. OECD. https://www.oecd.org/coronavirus/policy-responses/transparency-communication-and-trust-the-role-of-public-communication-in-responding-to-the-wave-of-disinformation-about-the-new-coronavirus-bef7ad6e/ . Accessed 21 Oct 2022.

McGrory K, Wollington R. Florida medical examiners were releasing coronavirus death data. The state made them stop. Tampa Bay Times. 2020. https://www.tampabay.com/news/health/2020/04/29/florida-medical-examiners-were-releasing-coronavirus-death-data-the-state-made-them-stop/ . Accessed 14 Dec 2020.

Myers SL. China created a fail-safe system to track contagions. It failed. The New York Times. 2020. https://www.nytimes.com/2020/03/29/world/asia/coronavirus-china.html . Accessed 16 Dec 2020.

Coronavirus: how New Zealand relied on science and empathy—BBC News. BBC News. 2020. https://www.bbc.com/news/world-asia-52344299 . Accessed 15 Dec 2020.

Wasike A. Tanzanian president claims “country free of COVID-19”. Anadolu Agency. 2020. https://www.aa.com.tr/en/africa/tanzanian-president-claims-country-free-of-covid-19/1869961 . Accessed 30 Oct 2020.

Coronavirus: Iran cover-up of deaths revealed by data leak. BBC News. 2020. https://www.bbc.com/news/world-middle-east-53598965 . Accessed 15 Dec 2020.

Hosseini K. Is Iran covering up its outbreak? BBC News. 2020. https://www.bbc.com/news/world-middle-east-51930856 . Accessed 15 Dec 2020.

Pazzanesse C. Calculating possible fallout of Trump’s face mask remarks. Harvard Gazette. 2020. https://news.harvard.edu/gazette/story/2020/10/possible-fallout-from-trumps-dismissal-of-face-masks/ . Accessed 15 Dec 2020.

McCoy T, Traiano H. Limits on coronavirus testing in Brazil are hiding the true dimensions of Latin America’s largest outbreak. Washington Post. https://www.washingtonpost.com/world/the_americas/coronavirus-brazil-testing-bolsonaro-cemetery-gravedigger/2020/04/22/fe757ee4-83cc-11ea-878a-86477a724bdb_story.html . Accessed 14 Dec 2020.

Dyer O. Covid-19: at least eight doctors in Egypt arrested for criticising government response. BMJ. 2020;370: m2850.

Joint statement by WHO, UN, UNICEF, UNDP, UNESCO, UNAIDS, ITU, UN Global Pulse, and IFRC. Managing the COVID-19 infodemic: Promoting healthy behaviours and mitigating the harm from misinformation and disinformation. WHO. https://www.who.int/news/item/23-09-2020-managing-the-covid-19-infodemic-promoting-healthy-behaviours-and-mitigating-the-harm-from-misinformation-and-disinformation . Accessed 30 Oct 2020.

Hua J, Shaw R. Corona Virus (COVID-19) “Infodemic” and emerging issues through a data lens: the case of China. Int J Environ Res Public Health. 2020;17(7):2309.

Nielsen RK, Fletcher R, Newman N, Brennen JS, Howard PN. Navigating the ‘Infodemic’: how people in six countries access and rate news and information about coronavirus. Reuters Institute for the Study of Journalism as part of the Oxford Martin Programme on Misinformation, Science and Media, a three-year research collaboration between the Reuters Institute, the Oxford Internet Institute, and the Oxford Martin School; 2020.

Gittleson B, Phelps J, Cathey L. Trump doubles down on defense of hydroxychloroquine to treat COVID-19 despite efficacy concerns. ABC News. 2020. https://abcnews.go.com/Politics/trump-doubles-defense-hydroxychloroquine-treat-covid-19-efficacy/story?id=72039824 . Accessed 15 Dec 2020.

De Sousa M, Biller D. Brazil’s president, infected with virus, touts malaria drug. CTVNews. 2020. https://www.ctvnews.ca/world/brazil-s-president-infected-with-virus-touts-malaria-drug-1.5015608 . Accessed 15 Dec 2020.

COVID-19 INED. Key issues. Ined—Institut national d’études démographiques. n.d. https://dc-covid.site.ined.fr/en/presentation/ . Accessed 14 Dec 2020.

Lima T. Following Data Blackout, Rio City Government’s “Updated” Covid-19 dashboard excludes 40% of deaths. RioOnWatch. 2020. https://www.rioonwatch.org/?p=59883 . Accessed 15 Dec 2020.

Ciminelli G, Garcia-Mandicó S. COVID-19 in Italy: an analysis of death registry data. J Public Health Oxf Engl. 2020;42(4):723–30.

Italy’s coronavirus death toll is likely much higher: “Most deaths simply aren’t counted.” CBS News . 2020. https://www.cbsnews.com/news/italy-coronavirus-deaths-likely-much-higher/ . Accessed 15 Dec 2020.

McCarthy G, Dempsey R, Parnell A, MacCarron. Just a “bad flu”? How death notices debunk that Covid-19 myth. 2020. https://www.rte.ie/brainstorm/2020/0928/1167878-covid-19-rip-death-notices-bad-flu-excess-mortality/ . Accessed 10 Jan 2021.

Osman L. Canada still only considering gathering race-based COVID-19 data. CTV News . 2020. https://www.ctvnews.ca/health/coronavirus/canada-still-only-considering-gathering-race-based-covid-19-data-1.4927648 . Accessed 15 Dec 2020.

Gilbert J, Keane D. How French law makes minorities invisible. The Independent . 2016. https://www.independent.co.uk/news/world/politics/how-french-law-makes-minorities-invisible-a7416656.html . Accessed 15 Dec 2020.

Timsit A. France’s data collection rules obscure the racial disparities of COVID-19. Quartz. 2020. https://qz.com/1864274/france-doesnt-track-how-race-affects-covid-19-outcomes/ . Accessed 15 Dec 2020.

Vogel L. Have we misjudged the role of children in spreading COVID-19? CMAJ. 2020;192(38):E1102–3.

Aguilar B. TDSB closes six more schools because of COVID-19. Toronto. 2020. https://toronto.ctvnews.ca/tdsb-closes-six-more-schools-because-of-covid-19-1.5229597 . Accessed 15 Dec 2020.

Kleitman S, Fullerton DJ, Zhang LM, Blanchard MD, Lee J, Stankov L, et al. To comply or not comply? A latent profile analysis of behaviours and attitudes during the COVID-19 pandemic. PLoS ONE. 2021;16(7): e0255268.

Savage M. Could the Swedish lifestyle help fight coronavirus? BBC . 2020. https://www.bbc.com/worklife/article/20200328-how-to-self-isolate-what-we-can-learn-from-sweden . Accessed 15 Dec 2020.

Green K. Worried about possible lockdown, Alberta shoppers flood the stores. CTV News . 2020. https://calgary.ctvnews.ca/worried-about-possible-lockdown-alberta-shoppers-flood-the-stores-1.5201682 . Accessed 15 Dec 2020.

Department of Economic and Social Affairs. The 17 GOALS | sustainable development. United Nations. n.d. https://sdgs.un.org/goals . Accessed 14 Dec 2020.

United Nations Office for Disaster Risk Reduction. Sendai Framework for Disaster Risk Reduction 2015–2030. 2015. https://www.undrr.org/publication/sendai-framework-disaster-risk-reduction-2015-2030 . Accessed 14 Dec 2020.

Chen H, Hailey D, Wang N, Yu P. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.

Rajan D, Koch K, Rohrer K, Bajnoczki C, Socha A, Voss M, et al. Governance of the Covid-19 response: a call for more inclusive and transparent decision-making. BMJ Glob Health. 2020;5(5):e002655.

Sherbrooke U de. Study by the Université de Sherbrooke on the psychosocial impacts of the pandemic. https://www.newswire.ca/news-releases/study-by-the-universite-de-sherbrooke-on-the-psychosocial-impacts-of-the-pandemic-886218205.html . Accessed 30 Jan 2021.

Download references

Acknowledgements

The authors acknowledge Dr. Neil Abernethy’s contributions to the COVID-19 Policies and Epidemiology Working Group.

No funding was received for this manuscript.

Author information

Authors and affiliations.

Department of Health Research Methods, Evidence and Impact, McMaster University, CRL 2nd Floor, 1280 Main Street West, Hamilton, ON, L8S4K1, Canada

Elizabeth Alvarez, Iwona A. Bielska, Stephanie Hopkins & Ahmed A. Belal

Department of Anthropology, University of Colorado Boulder, Boulder, CO, USA

Donna M. Goldstein & Anna Wynfield

Disaster and Emergency Management, Royal Roads University, British Columbia, Canada

School of Medicine, McMaster University, Hamilton, ON, Canada

Sureka Pavalagantharajah

Department of Architecture and Planning, Visvesvaraya National Institute of Technology, Nagpur, India

Shruthi Dakey

University Integrated Health Center of the Nord-de-l’île de Montréal (CIUSSS NIM), Montréal, QC, Canada

Marie-Carmel Gedeon

Faculty of Resilience, Rabdan Academy, Abu Dhabi, United Arab Emirates

Department of Global Health, McMaster University, Hamilton, ON, Canada

Katrina Bouzanis

You can also search for this author in PubMed   Google Scholar

Contributions

E.A. drafted the manuscript, and all other authors contributed to the content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Elizabeth Alvarez .

Ethics declarations

Ethics approval and consent to participate.

Not applicable as publicly available information was used.

Consent for publication

Not applicable as no individual-level data are used in this manuscript.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alvarez, E., Bielska, I.A., Hopkins, S. et al. Limitations of COVID-19 testing and case data for evidence-informed health policy and practice. Health Res Policy Sys 21 , 11 (2023). https://doi.org/10.1186/s12961-023-00963-1

Download citation

Received : 27 March 2021

Accepted : 15 January 2023

Published : 25 January 2023

DOI : https://doi.org/10.1186/s12961-023-00963-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Epidemiology
• Physical distancing policy

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• 05 May 2021

COVID research: a year of scientific milestones

For just over a year of the COVID-19 pandemic, Nature highlighted key papers and preprints to help readers keep up with the flood of coronavirus research. Those highlights are below. For continued coverage of important COVID-19 developments, go to Nature’s news section .

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-00502-w

Reprints and permissions

• Medical research
• Epidemiology

Could rats and dogs detect disease better than the finest lab equipment?

Outlook 19 JUN 24

Human neuroscience is entering a new era — it mustn’t forget its human dimension

Editorial 19 JUN 24

First encounter with SARS-CoV-2: immune portraits of COVID susceptibility

News & Views 19 JUN 24

Put people at the heart of schizophrenia research

World View 19 JUN 24

Cheaper versions of blockbuster obesity drugs are being created in India and China

News 19 JUN 24

What causes long COVID? Case builds for rogue antibodies

News 13 JUN 24

A virally encoded high-resolution screen of cytomegalovirus dependencies

Article 05 JUN 24

Spatial & Single-Cell Computational Immunogenomics for Cancer Diagnosis

(Level A) $75,888 to $102,040 per annum plus an employer contribution of 17% superannuation applies. Fixed term, full time position available for ...

Adelaide (Suburb), Metropolitan Adelaide (AU)

University of Adelaide

Software Engineer for R and High-Performance Computing

Ai large-language models of single-cell data for cancer immunogenomics.

(Level B) $107,276 to $126,894 per annum plus an employer contribution of 17% superannuation applies. Fixed term, full time position available for...

Senior Research Associate (m/f/d) - permanent position - Institute of Neuropathology

Main topics of scientific projects are: Kallikrein-8 as an early biomarker and therapeutic Target for Alzheimer’s Disease Epigenetic effects of e...

Essen, Nordrhein-Westfalen (DE)

Universitätsklinikum (AöR)

Department of Health and Human Services (DHHS), is seeking exceptional candidates for the position of Director

Bethesda, Maryland (US)

National Institutes of Health

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

COVID-19 Limitations Unique Opportunity for Researchers to Decrease Digital Divide

Researchers need to develop new ways to reach rural participants.

• by Karen Nikos-Rose
• April 29, 2020

The COVID-19 shelter-in-place orders and other limitations could offer researchers the chance to use technology to decrease the digital divide and disparities in academic research, suggests a University of California, Davis, professor in a new commentary.

“While I know many of my colleagues are frustrated with this pause in clinical research, it is actually a unique opportunity,” said Leigh Ann Simmons, chair of the Department of Human Ecology, whose research interests include increased equity in health care delivery and chronic disease prevention in rural areas. “People who live in rural areas are often left out of clinical trials that can benefit them, partly because they are not near large medical centers,” she said. This includes migrant workers, farmers and the general public who live in outlying areas.

She is co-author of the commentary , “Navigating Nonessential Research Trials During COVID 19: The Push We Needed for Using Digital Technology to Increase Access for Rural Participants?” published in The Journal of Rural Health earlier this month. Co-author is Devon Noonan, a researcher at Duke University.

Simmons said some research in which research subjects have to be contacted personally for interviews, testing or surveys has stopped since social distancing went into effect. This is a mistake, she said. “If we think creatively we can extend our reach.”

“We need to stop and think,” said Simmons, who is herself currently engaged in two rural health prevention studies that are being conducted solely using remote strategies. “How can we do our work remotely? Is there a way to get our data without human contact? And if we go this route, how can we include people who may not usually participate in our studies?”

It is well known, the authors said in their paper, that rural populations experience significant health disparities, especially in rates of common chronic diseases such as heart disease, diabetes, cancer and the associated health behaviors such as diet, physical activity, and tobacco and other substance use. “These disparities are in part due to rural residents’ lack of access to, knowledge about, and participation in clinical trials,” they said.

Participation in such trials is made more difficult in these areas too by lack of good internet access. Simmons said this could be augmented by researchers using community centers or regional facilities, or other community partners, to enable access for those in the study. Regional facilities could also be used to help with data and sample collections.

Further, state departments of heath “could replicate the partnership that the California Department of Education initiated with Google to distribute mobile hotspots to areas without broadband access so that K-12 education could continue amid school closures associated with shelter-in-place orders,” the authors suggest.

“Moving to remote clinical trials is not without its challenges, especially for studies that are well underway,” she emphasizes. “Importantly, the steps we take now to continue nonessential research remotely may provide the evidence we need to ensure that future studies target these hard-to-reach populations for study inclusion.”

Establishing remote access to clinical trials will serve to not only decrease rural clinical trial disparities, the authors said, but also to promote rural health equity into the next decade and beyond.

Media Resources

Karen Nikos-Rose, News and Media Relations, 530-219-5472, [email protected]

Primary Category

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Biases and limitations in observational studies of Long COVID prevalence and risk factors: A rapid systematic umbrella review

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected] (MJH); [email protected] (GB)

Affiliations Office of the Assistant Secretary of Planning and Evaluation, U.S. Department of Health and Human Services, Washington, DC, United States of America, Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, United States of America

Roles Conceptualization, Data curation, Methodology, Software

Affiliation National Institutes of Health Library, Office of Research Services, U.S. Department of Health and Human Services, Bethesda, MD, United States of America

Roles Investigation, Validation

Affiliation Office of the Assistant Secretary of Planning and Evaluation, U.S. Department of Health and Human Services, Washington, DC, United States of America

Roles Conceptualization, Investigation, Methodology, Project administration, Resources, Supervision, Writing – review & editing

• Miao Jenny Hua, 
• Gisela Butera, 
• Oluwaseun Akinyemi, 
• Deborah Porterfield

• Published: May 2, 2024
• https://doi.org/10.1371/journal.pone.0302408
• Peer Review
• Reader Comments

Observational studies form the foundation of Long COVID knowledge, however combining data from Long COVID observational studies has multiple methodological challenges. This umbrella review synthesizes estimates of Long COVID prevalence and risk factors as well as biases and limitations in the primary and review literatures.

Methods and findings

A systematic literature search was conducted using multiple electronic databases (PubMed, EMBASE, LitCOVID) from Jan 1, 2019 until June 9, 2023. Eligible studies were systematic reviews including adult populations assessed for at least one Long COVID symptom four weeks or more after SARS-CoV-2 infection. Overall and subgroup prevalence and risk factors as well as risk of bias (ROB) assessments were extracted and descriptively analyzed. The protocol was registered with PROSPERO (CRD42023434323). Fourteen reviews of 5–196 primary studies were included: 8 reported on Long COVID prevalence, 5 on risk/protective factors, and 1 on both. Prevalence of at least 1 Long COVID symptom ranged from 21% (IQR: 8.9%-35%) to 74.5% (95% CI: 55.6%-78.0%). Risk factor reviews found significant associations between vaccination status, sex, acute COVID-19 severity, and comorbidities. Both prevalence and risk factor reviews frequently identified selection and ascertainment biases. Using the AMSTAR 2 criteria, the quality of included reviews, particularly the prevalence reviews, were concerning for the adequacy of ROB assessments and justifications for conducting meta-analysis.

A high level of heterogeneity render the interpretation of pooled prevalence estimates of Long COVID challenging, further hampered by the lack of robust critical appraisals in the included reviews. Risk factor reviews were of higher quality overall and suggested consistent associations between Long COVID risk and patient characteristics.

Citation: Hua MJ, Butera G, Akinyemi O, Porterfield D (2024) Biases and limitations in observational studies of Long COVID prevalence and risk factors: A rapid systematic umbrella review. PLoS ONE 19(5): e0302408. https://doi.org/10.1371/journal.pone.0302408

Editor: Paulo Alexandre Azevedo Pereira Santos, University of Porto, Faculty of Medicine, PORTUGAL

Received: January 13, 2024; Accepted: April 3, 2024; Published: May 2, 2024

This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

As the COVID-19 pandemic enters its endemic phase, many questions remain regarding the prevalence and risks factors of Long COVID, which has also been called long-haul COVID, post-COVID-19 conditions and a subset of which are post-acute sequelae of COVID-19 [ 1 , 2 ]. One of the first systematic reviews published on Long COVID estimated that as many as 80% of COVID-19 survivors have at least one long-term post-COVID-19 condition [ 3 ]. While natural and vaccine-mediated immunity have reduced rates of hospitalization and death from acute COVID-19, the number of people who have been infected and reinfected with SARS-CoV-2 continues to grow, and with it, cases of Long COVID [ 4 ]. Three years into the pandemic, systematic reviews publish estimates of Long COVID prevalence as low as 6.2% [ 5 ] to as high as 50% [ 6 , 7 ]. Moreover, risk and protective factors such as vaccination and infection from different variants of concern remain underexplored areas of research [ 2 ]. Observational studies form the foundation of knowledge on Long COVID-19 prevalence and risks, but comparing and aggregating data poses multiple methodological challenges [ 8 , 9 ]. Lack of robustness in Long COVID observational studies has been remarked on through a recent systematic review of the pediatric population [ 10 ]. The aim of this study is to examine the more abundant research on Long COVID in adults that have already been synthesized in systematic reviews through the lens of an umbrella review. This is a useful method for revealing common biases and limitations in the field by synthesizing the critical appraisals that systematic reviews conduct [ 11 , 12 ].

The main questions of this review are 1) What are the prevalence and risk factors for Long COVID? 2) What kinds of biases and limitations affect the interpretation of observational studies of Long COVID prevalence and risk factors? Given the ongoing challenges to accurately measuring the burden of Long COVID, our goal is to provide guidance for future research to avoid common pitfalls that can impact the validity of observational and interventional studies.

We performed a rapid umbrella review of the evidence following the recommendations of the Cochrane Rapid Reviews Methods Group [ 13 ]. A rapid review is an evidence synthesis review which follows the systematic review process, and components of the methodology may be simplified or omitted [ 14 ]. This review omitted searches of grey literature and data extraction was performed by a single reviewer, which expedited the review process to under six months without compromising on other areas of a systematic review (e.g., critical appraisal) felt to be crucial to ensuring an unbiased protocol. The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) 2020 guidelines (see S6 Table ). We followed a review protocol pre-registered with the International Prospective Register of Systematic Reviews (PROSPERO) database, CRD42023434323, with no major deviations.

Eligibility criteria

Eligible study designs were systematic reviews (SR) with or without meta-analyses (MA), excluding narrative, scoping and non-systematic reviews. In terms of the PICO criteria, the study population included adults aged 18 years and older; reviews including children were eligible if outcomes were stratified by age. Exposure was defined as acute SARS-CoV-2 infection diagnosed 4 weeks or more prior to Long COVID ascertainment, in conformity with the U.S. federal working definition of Long COVID [ 15 ]. Comparators (i.e., controls) were defined according to the individual SR reviewed. We considered Long COVID as any or at-least one patient-reported, clinically presented, or administrative (e.g., ICD-10 codes) outcome associated with Long COVID. Studies that exclusively reviewed the prevalence of conditions with preexisting medical definitions (e.g., diabetes) arising post-COVID-19 were excluded, consistent with the WHO consensus definition of post-COVID-19 condition as a diagnosis of exclusion [ 16 ]. The relevant context was Long COVID diagnosed and treated in high income countries, thus only peer-reviewed articles in English were considered.

Search strategy and study selection

The following three databases were searched from January 1, 2019, through June 9, 2023: LitCOVID, PubMed, Embase. Full database search strategy can be found in S1 Table . In addition to database searches, secondary searches were performed within Web of Science to identify potential reviews that met the eligibility criteria. We manually screened the reference lists of systematic reviews and searched Google Scholar.

Database search results were imported into a reference manager (EndNote X20; Clarivate Analytics) for deduplication, then uploaded into Covidence (Covidence, Melbourne, Victoria, Australia) screening software to remove additional duplicates. An initial pilot was performed to screen title/abstract and full text articles, and any revisions to the search strategy were recorded. Dual screening of both title/abstract and full text was conducted by two reviewers (MH and OA) independently. Any disagreements were resolved by a third reviewer (DP).

Data extraction and quality assessment

The data extraction template was piloted on a subset of SRs by two independent reviewers (MH and OA). Data extraction was conducted by a single reviewer (MH) into an excel spreadsheet; the extracted data from two SRs were then randomly selected and reviewed by one adjudicator (DP). The SRs’ corresponding authors were contacted no more than two times over the course of two weeks to obtain missing data. Data collected included article identifying information, study type, design characteristics of primary studies, setting, participant characteristics, relevant outcomes and the ROB and limitations. Where possible, characteristics specific to studies/populations informing the subset of relevant outcomes were extracted.

Risk of bias assessment

The AMSTAR 2 critical appraisal tool for SRs that include non-randomized studies was used to determine risk of bias [ 17 ]. AMSTAR 2 evaluates SRs through sixteen domains, seven of which critically impact the validity of the review, including protocol registration before commencement of the review (item 2), adequacy of the literature search (item 4), justification for excluding individual studies (item 7), ROB from individual studies included in the review (item 9), appropriateness of meta-analytical methods (item 11), consideration of ROB when interpreting the results of the review (item 13) and assessment of presence and likely impact of publication bias (item 15). Results from studies that have one or more critical weakness will be considered to have low or critically low overall confidence. Studies were assessed by one reviewer (MH) with blinded validation by a second reviewer on one randomly selected study (DP).

Data synthesis

A meta-analysis was not undertaken as the included SRs were not sufficiently homogenous in population characteristics and design. For meta-analyses that reported relevant outcomes of Long COVID, we reported prevalence as a percentage with 95% CI and risk factors as and odds ratios (OR) or hazard ratios (HR) with 95% CI. The Higgins I 2 estimate of heterogeneity was reported for all outcomes where available. If a meta-analysis was not performed, outcomes were reported as median and interquartile range (IQR) if there were at least 5 studies.

We reported pooled estimates or manually calculated the median and IQR of prevalence for each category of 1) hospitalization status (hospitalized, non-hospitalized, mixed); 2) duration of follow-up (<3 months or ≥ 3 months); 3) use of a COVID-negative control group in the primary studies; 4) vaccination status (completed primary series vs. did not complete primary series); 5) COVID variant (wild-type, alpha/beta/delta, omicron). Risk factor outcomes were reported in accordance with the respective SRs. Due to the large number of risk factors investigated, we only reported pooled outcomes or which included at least 5 studies to calculate median/IQR.

We also conducted a narrative synthesis of the ROB identified by the review of the SR. Summary of the ROB comprised the ROB tool used, the number of included primary studies with high or critical ROB, and most frequent ROBs identified.

The database search resulted in 3,534 references. The reference list from Web of Science and Google Scholar searches yielded one additional article. The title/abstract screening excluded 2,285 articles and the full-text screen excluded 60 articles, most frequently for not including a relevant outcome (see S2 Table ). Fourteen SRs were ultimately deemed eligible (see Fig 1 ). Eight of 14 SRs reported on the prevalence or cumulative incidence of Long COVID (hereafter, prevalence SRs) and five reported on risk/protective factors (hereafter, risk factor SRs). One reported on both prevalence and risk factors in relation to different COVID-19 variants of concern but did not conduct meta-analyses for either outcome. For the sake of simplicity, it will hereafter be counted among the prevalence SRs.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

N, number of studies.

https://doi.org/10.1371/journal.pone.0302408.g001

Study and participant characteristics

All SRs were published between May 26, 2021 to June 8, 2023, with the most recent primary literature search inclusive of February 10, 2023 [ 18 ]. The prevalence SRs included 6 SR/MA, 2 SR, and 1 umbrella review with evidence synthesis of a selection of the primary literature. Many SRs did not provide a complete list of citations of the included studies, so we were unable to disambiguate a unique set of primary studies even after contacting the corresponding authors.

Prevalence SRs included 5 to 196 studies and 1643 to 1,289,044 participants. Among the five risk factor SRs, 6–41 studies of 7170 to 860,783 participants were included. Four conducted meta-analyses for relevant outcomes. The most common study designs of primary studies were cohort studies, followed by cross-sectional, and case-control studies. More information on publication and study design can be found in Table 1 and S3 Table .

https://doi.org/10.1371/journal.pone.0302408.t001

Among the prevalence SRs, 3 included adults only [ 18 – 20 ], 5 presented age-stratified outcomes [ 6 , 7 , 21 – 23 ], and 1 reported median or mean ages ≥47 for all relevant primary studies [ 24 ]. Diagnosis of SARS-CoV-2 infection was laboratory-based in most SRs that reported exposure ascertainment, but self-reported COVID-19 diagnosis was considered eligible by at least one SR [ 6 ]. Timing of follow-up ranged from 28 to 730 days from time zero, which varied as the point of COVID-19 diagnosis, symptom onset, hospital admission or hospital discharge [ 20 , 24 ]. Of the risk factor SRs, one was restricted to adults [ 25 ]; the rest reported mean/median ages of 40–69. Most primary studies in 4 out of 5 risk factor studies confirmed COVID-19 exposure by laboratory methods. For more details on participant characteristics, see Table 1 .

Prevalence outcomes

Long COVID prevalence or cumulative incidence in the SRs ranged from 21% (IQR: 8.9%-35%) to 74.5% (95% CI: 55.6%-78.0%). Hospitalization status was reported by 6 SRs, 3 at the study level (i.e., whether a study included hospitalized patients) and 3 at the individual level for at least one relevant outcome. Of the latter, the percentage of hospitalized patients ranged from 17.4% to 98.2% (see S4 Table ). Two SRs conducted random-effects meta-analysis by hospitalization status at the study level [ 6 , 7 ]. Four provided outcomes that could be stratified by duration of follow-up at 3 months, two of which calculated the pooled prevalence of Long COVID by duration of follow-up [ 7 , 20 ]. For the rest, we calculated the median prevalence and inter-quartile ranges stratified by 3 months/12 weeks.

The inclusion of COVID-19 negative comparator groups was noted in three SRs; only one estimated the prevalence of Long COVID stratified by the use of control groups [ 21 ]. Three SRs reported the SARS-CoV-2 variant of concern assumed responsible for most infections [ 19 , 22 , 23 ], one of which reported Long COVID prevalence by variant of concern without conducting a meta-analysis [ 19 ]. Two SRs commented on vaccination status reported in primary studies [ 7 , 19 ], neither of which reported prevalence estimates by vaccination status. See S5 Table for more information on comparator groups and other subgroups.

Outcomes with pooled estimates are shown in Fig 2 . The I 2 measure of heterogeneity was over 90% in all reporting pooled prevalence estimates but an I 2 was not reported for 50% of pooled estimates. Outcomes with median and IQR are shown in Fig 3 .

N, number of studies; CI, confidence interval; NR, not reported.

https://doi.org/10.1371/journal.pone.0302408.g002

N, number of studies; IQR, interquartile range.

https://doi.org/10.1371/journal.pone.0302408.g003

Risk factor outcomes

Up to fourteen different risk/protective factors were extracted from the five risk factor SRs reviewed. Three examined the associations between vaccination and Long COVID risk and found that two or more pre-infection doses of COVID-19 vaccine significantly decreased the risk of Long COVID compared to no or 1 vaccine, but a single dose did not significantly mitigate Long COVID risk [ 25 – 27 ]. Three SRs reported on risks associated with age, sex, acute COVID-19 severity and other sociodemographic and clinical risk factors [ 25 , 28 , 29 ]. All three found female sex to be a significant risk factor for Long COVID. One SR found that categorical age of 40 or greater posed increased risk of Long COVID (OR: 1.21, 95% CI: 1.11–1.33) [ 25 ], while another did not find evidence of age-associated risks [ 28 ]. The latter also found elevated risks of non-recovery from severe or critical acute COVID-19 with moderate certainty, although associations with hospitalization status were non-significant [ 28 ]. Both SRs investigated associations between Long COVID and comorbidities, finding significant risk associations. Full statistical outcomes are summarized in S4 Table .

Summary of ROB assessments of primary studies

A majority of SRs (6 out of 9 prevalence SRs, 2 out of 5 risk factor SRs used the Newcastle-Ottawa Scale (NOS) [ 30 ]. Other tools used in the prevalence SRs included the NIH tool (2 studies) [ 31 ], Hoy et al. (1 study) [ 32 ] and the AHRQ tool (1 study) for cross-sectional studies [ 33 ] in a study that also used the NOS for cohort studies. Risk factor SRs also employed ROBINS-I (1 study) [ 34 ], JBI checklist for cohort studies (1 study) [ 35 ] and QUIPS (1 study) [ 36 ].

Six prevalence SRs gave an overall ROB score. Regardless of the tool used, the majority of primary studies were scored as having low or moderate ROB, with only 0–9.9% of studies rated has having a high ROB (e.g., less than five out of nine on the NOS scale). The risk factor SRs reported higher ROB overall, with 0–56% of primary studies rated as having a high or critical ROB.

Out of fourteen SRs, two prevalence SRs [ 7 , 22 ] and one risk factor SR [ 25 ] reported only the overall ROB of the primary studies without a score breakdown. From the 11 SRs that did report a score breakdown, the quality of outcome ascertainment and selection bias were the most frequently top-ranked ROBs; adjustment for confounders, attrition, representative sampling and outcome ascertainment were areas of deficiency in at least two risk factors SRs.

Summary of limitations as described in the SRs

High heterogeneity was identified as a limitation in 8 prevalence SRs, with the exception of the SR on asymptomatic cases [ 23 ]. Lack of control/comparator group or representative sampling was noted in 5 prevalence SRs. Lack of standardization in case definition and symptom measurement was noted in 3 prevalence SRs and 4 risk factor SRs. Variable follow-up time, lack of data on age, race/ethnicity, disability and overrepresentation of people hospitalized with COVID-19 were also identified as limitations.

ROB assessment of SRs with AMSTAR 2

All prevalence SRs had weaknesses in at least two critical domains (see Table 2 ) [ 17 ]. Among these, deficiencies on items 9 and 11 are particularly concerning for our aim of identifying biases and limitations in the Long COVID evidence base.

https://doi.org/10.1371/journal.pone.0302408.t002

An adequate score on item 11 required the SR to explicitly justify the decision to perform a meta-analysis based on the compatibility of included studies. None of the prevalence SRs that conducted a meta-analysis included such a statement, while all of the SRs which did not pursue a meta-analysis cited the high degree of heterogeneity in the primary literature as deterrent. We considered this item satisfied if meta-analysis was primarily undertaken by pre-determined subgroups that may create comparable cohorts, such as by hospitalization status or follow-up duration. Nevertheless, the subgroup outcomes in prevalence SRs all reported I 2 statistics greater than 90%.

ROB assessments conducted by the SRs were often deficient, hence the large number of partial or complete deficiencies on item 9. For a “yes,” the SR had to evaluate primary studies on at least four sources of bias: confounding, selection, measurement of exposures and outcomes, and selective reporting of analyses or outcomes. The NOS rates ROB across the three domains of selection, comparability and outcome assessment without any criteria for selective reporting [ 30 ], so SRs that used this tool without modification received a “partial yes” at best on this criterion. On assessing confounding, NOS requires pre-specification of the two most important factors to control to satisfy the criteria of “comparability,” which 6 out of the 8 SRs using the tool failed to specify [ 7 , 20 , 22 , 23 , 25 , 27 ]. Selection bias as well as confounding were equivocally evaluated by most SRs due to ambiguity around the definition of control groups (see Table 1 and S5 Table ). Most of the seven prevalence SRs that included the general adult population did not define a control group; one defined the control group as COVID-positive without post-discharge symptoms [ 20 ]; one defined it as people without COVID-19 but did not report the number of studies that used a control group or any associated outcomes [ 22 ]. This exposes interpretative challenges in how SRs applied the NOS criteria on “selection of non-exposed cohort” and “comparability.” [ 30 ].

Risk factor SRs had lower ROB overall, with two SRs with 0–1 deficiency. All had clearly defined non-exposed comparator groups by the PICO criteria (lacking in the risk factor rather than COVID-19 exposure). Nevertheless, a majority were deficient on items 7 and 10, and 40% were deficient on items 9 and 11 (see Table 2 ).

This umbrella review found a wide range in the prevalence estimates of Long COVID primary studies, yielding pooled prevalence estimates that cluster around 50% ( Fig 2 ), which needs to be interpreted in light of a major limitation. The presence of high heterogeneity demands the use of random-effects meta-analysis as was done in all the SRs reviewed [ 37 ]. But when between-study variance greatly exceeds within-study variance, as is the case when the I 2 statistic exceeds 90%, each primary study is given similar weight and the pooled estimate approximates the arithmetic mean [ 38 ]. It is thus no surprise that pooled prevalence estimates cluster around 50% when prevalence estimates in the primary literature spans nearly the entire range of numerical possibility ( Fig 3 ). This may also explain why, with few exceptions, meta-analytic estimates of Long COVID prevalence consistently exceeds estimates from population-based samples [ 3 , 39 – 41 ]. For instance, the June 7–19, 2023 wave of the U.S. Household Pulse Survey, which periodically samples a representative group of U.S. adults, suggested that Long COVID prevalence was 11.0% (95% CI: 10.4–11.6%) among U.S. adults reporting previous COVID-19, lower than any of the pooled prevalence estimates we reviewed [ 42 ].

Nevertheless, random-effects meta-analysis may be fruitfully applied to subgroups prespecified by study design and population characteristics. For instance, one SR observed a difference in prevalence estimates by hospitalization status, with lower prevalence estimates in studies of exclusively non-hospitalized compared to post-hospitalization cohorts [ 6 ]. The inclusion of control group and population sampling also generated lower prevalence estimates, although no meta-analysis was conducted in the only SR we included which reported outcomes by the use of these methods [ 21 ]. A recent SR not included in the date-range of our search estimated Long COVID absolute risk difference in community-based samples using control groups to be 10.1% (95% PI: -12.7%-32.8%) compared a pooled prevalence of 42.1% (95% PI: 6.8–87.9%) for all studies with more than 12 weeks of follow-up [ 41 ]. Timing of follow-up did not appear to significantly modify prevalence estimates in the four studies that reported on prevalence before and after 3 months of follow-up [ 6 , 7 , 20 , 24 ], although overlaps in follow-up durations and inconsistent reporting in the primary literature may confound these findings. No SR specified enough subgroups to adequately address the range of factors likely contributing to high heterogeneity.

The risk factor SRs did not suffer as much from high heterogeneity. More than one SR discerned significant associations between increased COVID-19 risk and less than 2 pre-infection vaccinations, female sex, and multiple comorbidities. The association between Long COVID risk and acute-COVID-19 hospitalization and severity were also significant in at least one SR. However, hospitalization and acute COVID-19 severity are strongly associated with selection into Long COVID studies, so one should be wary of spurious associations that emerge from collider bias, as has been demonstrated in other risk associations derived from test-positive or hospitalization-based COVID-19 cohorts [ 43 ].

Considering that SRs, coupled with meta-analyses, form the “capstone” of evidence-based medicine and public health [ 44 ], it is troubling that this review exposed a high level of ROB among prevalence SRs. Selection and measurement biases were reported across SRs. In prevalence SRs, bias towards hospitalized patients and survey respondents likely led to an over-estimation of Long COVID prevalence. Measurement bias, particularly the use of self-reporting without a standardized scale or blind independent assessment, was another recurrent ROB. Different approaches to Long COVID outcome assessment have been shown to produce prevalence estimates that vary from 3.0% based on tracking specific symptoms to 11.7% based on self-classification within the same sample population [ 45 ]. Our recommendations to address these and other sources of bias are elaborated in Table 3 .

https://doi.org/10.1371/journal.pone.0302408.t003

A major limitation of this review is that the reporting of study and population characteristics of the primary literature in the SRs reviewed, including the ROB assessments, lacked sufficient consistency, granularity, and methodological transparency. Aggregating across a kitchen-sink metric like “at least one symptom” when what counts as a Long COVID symptom differs across the primary studies obviously hinders measurement and interpretation. Yet, we had little choice but to use this outcome as it is a widely accepted operational definition of Long COVID. An overall high ROB of the SRs corresponds to low certainty in our outcome estimates. Nevertheless, consistent themes emerged in the ROB assessments and limitations reviewed.

Our review highlights four major areas of limitation and bias in Long COVID observational studies: 1) few primary studies used techniques of representative sampling or included non-exposed comparator cohorts; 2) both primary studies and SRs lacked uniformity and consistency in reporting potential confounders, including factors that may now be impossible to prospectively measure (e.g., pre-vaccination SARS-CoV-2 exposure); 3) a high overall ROB in the SRs, including inadequate ROB assessment of the primary studies; 4) primary studies and SRs selected a wide variety of outcomes to measure, contributing to high heterogeneity when aggregating across studies. A clear and consistent research definition of Long COVID with corresponding protocols for measurement would be an important intervention to reduce heterogeneity across Long COVID studies. The National Academies of Science, Engineering and Medicine have been tasked to examine the current U.S. government working definition of Long COVID, the culmination of which could bring much-needed standardization in Long COVID research [ 46 ]. However, this would only mitigate the fourth limitation, while the first three depend on improving study quality independent of heterogeneity stemming from an inconsistent case definition. In Table 3 , we augment an existing set of recommendations [ 24 ] for improving uniformity in the Long COVID primary literature and address sources of bias in the review literature. The effort to develop and maintain quality standards for measuring and monitoring Long COVID is not only important for understanding the long shadow of COVID-19, but in preparation for tracking post-infective conditions of future novel pathogens.

Supporting information

S1 table. database search strategies..

https://doi.org/10.1371/journal.pone.0302408.s001

S2 Table. Articles excluded after full-text review.

https://doi.org/10.1371/journal.pone.0302408.s002

S3 Table. Study publication and design information.

https://doi.org/10.1371/journal.pone.0302408.s003

S4 Table. Summary of statistical outcomes, hospitalization status and follow-up time points.

https://doi.org/10.1371/journal.pone.0302408.s004

S5 Table. Comparator groups and subgroups defined by vaccination status and variants of concern.

https://doi.org/10.1371/journal.pone.0302408.s005

S6 Table. PRISMA 2020 checklist.

https://doi.org/10.1371/journal.pone.0302408.s006

• View Article
• PubMed/NCBI
• Google Scholar
• 11. Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI2020.
• 15. Department of Health and Human Services OotASfH. National Research Action Plan on Long COVID. 2022.
• 31. National Heart Lung and Blood Institute. Study quality assessment tools 2021 [11/20/2023]. Available from: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools .
• 35. Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L. Chapter 3: Systematic reviews of effectiveness. 2020. In: JBI Manual for Evidence Synthesis [Internet]. JBI. Available from: https://synthesismanual.jbi.global .
• 45. Office of National Statistics. Technical article: Updated estimates of the prevalence of post-acute symptoms among people with coronavirus (COVID-19) in the UK: 26 April 2020 to 1 August 2021 2021 [25 October 2023]. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/articles/technicalarticleupdatedestimatesoftheprevalenceofpostacutesymptomsamongpeoplewithcoronaviruscovid19intheuk/26april2020to1august2021 .
• 46. National Academies of Science, Engineering, and Medicine. Examining the Working Definition for Long COVID n.d. [12/13/2023]. Available from: https://www.nationalacademies.org/our-work/examining-the-working-definition-for-long-covid .

Jump to navigation

• Bahasa Malaysia

What are the benefits and risks of vaccines for preventing COVID-19?

Key messages

– Most vaccines reduce, or probably reduce, the number of people who get COVID-19 disease and severe COVID-19 disease.

– Many vaccines likely increase number of people experiencing events such as fever or headache compared to placebo (sham vaccine that contains no medicine but looks identical to the vaccine being tested). This is expected because these events are mainly due to the body's response to the vaccine; they are usually mild and short-term.

– Many vaccines have little or no difference in the incidence of serious adverse events compared to placebo. 

– There is insufficient evidence to determine whether there was a difference between the vaccine and placebo in terms of death because the numbers of deaths were low in the trials.

– Most trials assessed vaccine efficacy over a short time, and did not evaluate efficacy to the COVID variants of concern. 

What is SARS-CoV-2 and COVID-19?

SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) is the virus that causes COVID-19 disease. Not everyone infected with SARS-CoV-2 will develop symptoms of COVID-19. Symptoms can be mild (e.g. fever and headaches) to life-threatening (e.g. difficulty breathing), or death.

How do vaccines prevent COVID-19?

While vaccines work slightly differently, they all prepare the body's immune system to prevent people from getting infected with SARS-CoV-2 or, if they do get infected, to prevent severe disease.

What did we want to find out?

We wanted to find out how well each vaccine works in reducing SARS-CoV-2 infection, COVID-19 disease with symptoms, severe COVID-19 disease, and total number of deaths (including any death, not only those related to COVID-19).

We wanted to find out about serious adverse events that might require hospitalization, be life-threatening, or both; systemic reactogenicity events (immediate short-term reactions to vaccines mainly due to immunological responses; e.g. fever, headache, body aches, fatigue); and any adverse events (which include non-serious adverse events).

What did we do? 

We searched for studies that examined any COVID-19 vaccine compared to placebo, no vaccine, or another COVID-19 vaccine.

We selected only randomized trials (a study design that provides the most robust evidence because they evaluate interventions under ideal conditions among participants assigned by chance to one of two or more groups). We compared and summarized the results of the studies, and rated our confidence in the evidence based on factors such as how the study was conducted.

What did we find? 

We found 41 worldwide studies involving 433,838 people assessing 12 different vaccines. Thirty-five studies included only healthy people who had never had COVID-19. Thirty-six studies included only adults, two only adolescents, two children and adolescents, and one included adolescents and adults. Three studied people with weakened immune systems, and none studied pregnant women.

Most cases assessed results less than six months after the primary vaccination. Most received co-funding from academic institutions and pharmaceutical companies. Most studies compared a COVID-19 vaccine with placebo. Five evaluated the addition of a 'mix and match' booster dose.

Main results 

We report below results for three main outcomes and for 10 World Health Organization (WHO)-approved vaccines (for the remaining outcomes and vaccines, see main text). There is insufficient evidence regarding deaths between vaccines and placebo (mainly because the number of deaths was low), except for the Janssen vaccine, which probably reduces the risk of all-cause deaths. 

People with symptoms

The Pfizer, Moderna, AstraZeneca, Sinopharm-Beijing, and Bharat vaccines produce a large reduction in the number of people with symptomatic COVID-19.

The Janssen vaccine reduces the number of people with symptomatic COVID-19.

The Novavax vaccine probably has a large reduction in the number of people with symptomatic COVID-19.

There is insufficient evidence to determine whether CoronaVac vaccine affects the number of people with symptomatic COVID-19 because results differed between the two studies (one involved only healthcare workers with a higher risk of exposure).

Severe disease

The Pfizer, Moderna, Janssen, and Bharat vaccines produce a large reduction in the number of people with severe disease.

There is insufficient evidence about CoronaVac vaccine on severe disease because results differed between the two studies (one involved only healthcare workers with a higher risk of exposure).

Serious adverse events

For the Pfizer, CoronaVac, Sinopharm-Beijing, and Novavax vaccines, there is insufficient evidence to determine whether there was a difference between the vaccine and placebo mainly because the number of serious adverse events was low.

Moderna, AstraZeneca, Janssen, and Bharat vaccines probably result in no or little difference in the number of serious adverse events. 

What are the limitations of the evidence?

Most studies assessed the vaccine for a short time after injection, and it is unclear if and how vaccine protection wanes over time. Due to the exclusion criteria of COVID-19 vaccine trials, results cannot be generalized to pregnant women, people with a history of SARS-CoV-2 infection, or people with weakened immune systems. More research is needed comparing vaccines and vaccine schedules, and effectiveness and safety in specific populations and outcomes (e.g. preventing long COVID-19). Further, most studies were conducted before the emergence of variants of concerns.

How up to date is this evidence?

The evidence is up to date to November 2021. This is a living systematic review. Our results are available and updated bi-weekly on the COVID-NMA platform at covid-nma.com.

Compared to placebo, most vaccines reduce, or likely reduce, the proportion of participants with confirmed symptomatic COVID-19, and for some, there is high-certainty evidence that they reduce severe or critical disease. There is probably little or no difference between most vaccines and placebo for serious adverse events. Over 300 registered RCTs are evaluating the efficacy of COVID-19 vaccines, and this review is updated regularly on the COVID-NMA platform ( covid-nma.com ).

Implications for practice

Due to the trial exclusions, these results cannot be generalized to pregnant women, individuals with a history of SARS-CoV-2 infection, or immunocompromized people. Most trials had a short follow-up and were conducted before the emergence of variants of concern.

Implications for research

Future research should evaluate the long-term effect of vaccines, compare different vaccines and vaccine schedules, assess vaccine efficacy and safety in specific populations, and include outcomes such as preventing long COVID-19. Ongoing evaluation of vaccine efficacy and effectiveness against emerging variants of concern is also vital. 

Different forms of vaccines have been developed to prevent the SARS-CoV-2 virus and subsequent COVID-19 disease. Several are in widespread use globally. 

To assess the efficacy and safety of COVID-19 vaccines (as a full primary vaccination series or a booster dose) against SARS-CoV-2.

We searched the Cochrane COVID-19 Study Register and the COVID-19 L·OVE platform (last search date 5 November 2021). We also searched the WHO International Clinical Trials Registry Platform, regulatory agency websites, and Retraction Watch.

We included randomized controlled trials (RCTs) comparing COVID-19 vaccines to placebo, no vaccine, other active vaccines, or other vaccine schedules.

We used standard Cochrane methods. We used GRADE to assess the certainty of evidence for all except immunogenicity outcomes. 

We synthesized data for each vaccine separately and presented summary effect estimates with 95% confidence intervals (CIs). 

We included and analyzed 41 RCTs assessing 12 different vaccines, including homologous and heterologous vaccine schedules and the effect of booster doses. Thirty-two RCTs were multicentre and five were multinational. The sample sizes of RCTs were 60 to 44,325 participants. Participants were aged: 18 years or older in 36 RCTs; 12 years or older in one RCT; 12 to 17 years in two RCTs; and three to 17 years in two RCTs. Twenty-nine RCTs provided results for individuals aged over 60 years, and three RCTs included immunocompromized patients. No trials included pregnant women. Sixteen RCTs had two-month follow-up or less, 20 RCTs had two to six months, and five RCTs had greater than six to 12 months or less. Eighteen reports were based on preplanned interim analyses.

Overall risk of bias was low for all outcomes in eight RCTs, while 33 had concerns for at least one outcome.

We identified 343 registered RCTs with results not yet available. 

This abstract reports results for the critical outcomes of confirmed symptomatic COVID-19, severe and critical COVID-19, and serious adverse events only for the 10 WHO-approved vaccines. For remaining outcomes and vaccines, see main text. The evidence for mortality was generally sparse and of low or very low certainty for all WHO-approved vaccines, except AD26.COV2.S (Janssen), which probably reduces the risk of all-cause mortality (risk ratio (RR) 0.25, 95% CI 0.09 to 0.67; 1 RCT, 43,783 participants; high-certainty evidence).

Confirmed symptomatic COVID-19

High-certainty evidence found that BNT162b2 (BioNtech/Fosun Pharma/Pfizer), mRNA-1273 (ModernaTx), ChAdOx1 (Oxford/AstraZeneca), Ad26.COV2.S, BBIBP-CorV (Sinopharm-Beijing), and BBV152 (Bharat Biotect) reduce the incidence of symptomatic COVID-19 compared to placebo (vaccine efficacy (VE): BNT162b2: 97.84%, 95% CI 44.25% to 99.92%; 2 RCTs, 44,077 participants; mRNA-1273: 93.20%, 95% CI 91.06% to 94.83%; 2 RCTs, 31,632 participants; ChAdOx1: 70.23%, 95% CI 62.10% to 76.62%; 2 RCTs, 43,390 participants; Ad26.COV2.S: 66.90%, 95% CI 59.10% to 73.40%; 1 RCT, 39,058 participants; BBIBP-CorV: 78.10%, 95% CI 64.80% to 86.30%; 1 RCT, 25,463 participants; BBV152: 77.80%, 95% CI 65.20% to 86.40%; 1 RCT, 16,973 participants).

Moderate-certainty evidence found that NVX-CoV2373 (Novavax) probably reduces the incidence of symptomatic COVID-19 compared to placebo (VE 82.91%, 95% CI 50.49% to 94.10%; 3 RCTs, 42,175 participants).

There is low-certainty evidence for CoronaVac (Sinovac) for this outcome (VE 69.81%, 95% CI 12.27% to 89.61%; 2 RCTs, 19,852 participants).

Severe or critical COVID-19

High-certainty evidence found that BNT162b2, mRNA-1273, Ad26.COV2.S, and BBV152 result in a large reduction in incidence of severe or critical disease due to COVID-19 compared to placebo (VE: BNT162b2: 95.70%, 95% CI 73.90% to 99.90%; 1 RCT, 46,077 participants; mRNA-1273: 98.20%, 95% CI 92.80% to 99.60%; 1 RCT, 28,451 participants; AD26.COV2.S: 76.30%, 95% CI 57.90% to 87.50%; 1 RCT, 39,058 participants; BBV152: 93.40%, 95% CI 57.10% to 99.80%; 1 RCT, 16,976 participants).

Moderate-certainty evidence found that NVX-CoV2373 probably reduces the incidence of severe or critical COVID-19 (VE 100.00%, 95% CI 86.99% to 100.00%; 1 RCT, 25,452 participants).

Two trials reported high efficacy of CoronaVac for severe or critical disease with wide CIs, but these results could not be pooled.

Serious adverse events (SAEs)

mRNA-1273, ChAdOx1 (Oxford-AstraZeneca)/SII-ChAdOx1 (Serum Institute of India), Ad26.COV2.S, and BBV152 probably result in little or no difference in SAEs compared to placebo (RR: mRNA-1273: 0.92, 95% CI 0.78 to 1.08; 2 RCTs, 34,072 participants; ChAdOx1/SII-ChAdOx1: 0.88, 95% CI 0.72 to 1.07; 7 RCTs, 58,182 participants; Ad26.COV2.S: 0.92, 95% CI 0.69 to 1.22; 1 RCT, 43,783 participants); BBV152: 0.65, 95% CI 0.43 to 0.97; 1 RCT, 25,928 participants). In each of these, the likely absolute difference in effects was fewer than 5/1000 participants.

Evidence for SAEs is uncertain for BNT162b2, CoronaVac, BBIBP-CorV, and NVX-CoV2373 compared to placebo (RR: BNT162b2: 1.30, 95% CI 0.55 to 3.07; 2 RCTs, 46,107 participants; CoronaVac: 0.97, 95% CI 0.62 to 1.51; 4 RCTs, 23,139 participants; BBIBP-CorV: 0.76, 95% CI 0.54 to 1.06; 1 RCT, 26,924 participants; NVX-CoV2373: 0.92, 95% CI 0.74 to 1.14; 4 RCTs, 38,802 participants).

For the evaluation of heterologous schedules, booster doses, and efficacy against variants of concern, see main text of review.

• Policy & Compliance
• NIH Extramural Response To Natural Disasters and Other Emergencies
• The Impact of COVID-19 On The Research Community

The Impact of COVID-19 on the Research Community

• 55% of respondents said the pandemic will have a negative impact on their career trajectory
• 68% of respondents said societal/political events negatively affected their mental health, more than other factor
• 78% of respondents reported lower levels of productivity since the pandemic began 

Click here for PDF  

Career trajectory.

• 61% of lab-based researchers agreed that the pandemic will harm their career trajectory
• Asian respondents were more likely than other groups to anticipate a negative career trajectory (65%), with a decline in research activities and lab-based research driving opinions
• Black or African American respondents were least likely to anticipate a negative career trajectory (39%), with relatively fewer lab researchers and more public health researchers driving a more optimistic outlook 

A Closer Look

• The strongest predictor of a negative career trajectory perception is researchers’ ability to apply for grants 

Top career stages that anticipate negatively impacted career trajectories due to COVID-19:

• Postdoctoral Fellow/ Resident
• Faculty (0-6 Years)
• Faculty (7-14 Years) 

MENTAL Health

• 42%of respondents said their mental/physical health had a substantially negative impact on productivity.
• Women and respondents identifying as “other” genders were consistently more negatively impacted than men across top factors affecting mental health
• Early career investigators were consistently more negatively impacted across top factors affecting mental health
• Asian researchers cited visa considerations as negatively affecting their mental health at twice the rate than the average 

Top factors that negatively impacted researchers’ mental health include:

• Societal and/or political events
• Physical and/or social isolation
• Disruption of promotion/ tenure timeline 

Did You Know?

• Survey findings indicated mental and physical health is the #1 factor negatively impacting the productivity of early career investigators, Hispanics, and African American respondents 

RESEARCH Productivity

• Early-(80%) and mid-career investigators (81%) reported lower levels of productivity due to COVID-19, with faculty members reporting a more negative impact than non-faculty researchers
• 53%of Hispanics indicated their mental/physical health has negatively impacted research productivity since the pandemic began 

The Bottom Line:

• The less institutional support provided to researchers leads to a greater impact on productivity 

Top factors that negatively impacted researchers’ overall productivity include:

• 53% Virtual instead of in-person interactions with trainees, mentors, or supervisors
• 50% Cancellation of in-person regional, national, and/or international conferences
• 49% Changes in laboratory and/or animal facility access 

AT A GLANCE: COVID-19 IMPACTS ON EXTRAMURAL institutions

• 83% of respondents indicated that COVID-19 had a moderate or major impact on overall research productivity at their institution
• 41% of respondents said it is likely the financial repercussions of COVID-19 will jeopardize their institution’s ability to maintain research functions
• 2 in 3 respondents were very or extremely concerned about the pandemic’s impact on the financial status of their institution
• 77% of Doctorate-granting universities reported as very or extremely concerned
• 33% of Independent research institutions reported as very or extremely concerned

This page last updated on: March 23, 2021

• Bookmark & Share
• E-mail Updates
• Privacy Notice
• Accessibility
• National Institutes of Health (NIH), 9000 Rockville Pike, Bethesda, Maryland 20892
• NIH... Turning Discovery Into Health

COVID-19’s impact felt by researchers

Scientists, graduate students talk about conducting research during a pandemic.

• Conducting Research

While a single virion of SARS-CoV2 is only an approximately 50-200 nanometers in diameter (Chen et al., 2020; Xu et al., 2020), the impact of the COVID-19 pandemic on psychological science has been large, disrupting the conduct of psychological research and forcing psychological scientists to adapt their work to continue its progress.

Several important theoretical and practical challenges have emerged along the way.

How has research been affected by the COVID-19 pandemic?

The COVID-19 pandemic has affected a variety of researchers, students and academics. As institutions of higher education have limited in-person activities, research and training have been disrupted. Many graduate students have faced new barriers as a result (Chenneville and Schwartz-Mette, 2020; Thompson, 2020).

“It is ironic that I study social barriers [to diabetes care], as the COVID-19 pandemic has exposed many social and systemic barriers that affect individuals on an intimate level, regardless of profession,” says Avia Gray, MA, a doctoral student at the University of California, Merced. “Access to lab equipment and programs has been a personal challenge for me. Our university does not provide student access to some of the statistical software my lab uses, but they had strict COVID policies that prevented me from being on campus.”

Achieving graduate program milestones in a timely manner has been a priority for students and graduate programs long before the emergence of COVID-19. But the COVID-19 pandemic has introduced additional barriers that graduate students must navigate, resulting in increased stress.

“Despite [finding out that I was] pregnant before the lockdowns, I had planned to begin data collection for my new study involving eye-tracking, requiring in-person sessions,” says Candice Stanfield-Wiswell, MA, a doctoral student at George Mason University. “Of course, the initial eye-tracking study had to be postponed. Fortunately for everyone’s safety, my university followed the strict CDC guidelines from the onset. Unfortunately, this also meant that all university-affiliated research came to an immediate halt. My time-sensitive—I was due to give birth in early November—and ambitious study idea had to be scrapped for a new one that did not require in-person sessions.”

These challenges are not unique to students. Faculty face their own difficulties in navigating new barriers introduced by COVID-19.

“My research has been significantly stalled due to COVID-19,” says Erlanger Turner, PhD, assistant professor at Pepperdine University. “I have put several data collection projects on hold and have had to delay some writing projects due to the demands of online teaching. Most of my focus has been on helping students navigate the pandemic and continue progress on their dissertation research.”

Some faculty have focused on writing up existing data and submitting those manuscripts. But even the peer review process has been affected.

“One primary challenge for my research during the pandemic is the delay in manuscripts getting reviewed,” says Krista Howard, PhD, associate professor in the department of psychology at Texas State University. “This includes manuscripts that were submitted in the months prior to the pandemic and during the pandemic. I've been asked to complete more peer reviews during this period than usual, and I've made an effort to comply with the requests for reviews in my area of expertise because I know this is a problem right now.”

Howard was able to quickly put together a multidisciplinary team of psychologists and public health researchers at three universities to study the effects of the pandemic.

“We were able to get our study approved quickly by the IRB, launched, analyzed and written up for multiple publications,” she said. “Then everything stalled once we tried submitting manuscripts for publication.”

Research methodology likely plays a large role in the degree of impact individuals may experience in their research due to COVID-19. The heterogeneity in psychological methodology may have helped some researchers adapt more readily to pandemic-related challenges than others. However, even among those who can continue their research, the generalizability of data obtained during these unprecedented times may be an issue (Lourenco & Tasimi, 2020; Wolkewitz and Puljak, 2020).

“My current research uses more web-based surveys or secondary data analysis, so I haven’t had any problems related to data collection that I know many, many researchers have had,” says Ty Schepis, PhD, professor in the department of psychology at Texas State University.

Most individuals in academia have transitioned to working from home. For many, however, there are additional challenges in maintaining a research program without the clear-cut boundaries between work and home. Many academics face the increased stressor of balancing home and work-related demands simultaneously.

“The biggest challenge has been that I am the primary at-home elementary and middle school supervisor for my two children. I have a more flexible job than my wife does, so most of the supervision falls to me,” says Schepis. “It has been a serious challenge to fit work around supervising a first grader’s work. I’ve had to work most of the weekend to make up for the time I miss. It’s not ideal, but it is working for now.”

Current circumstances reshape the research landscape moving forward

COVID-19 disproportionately affects communities of color (CDC, 2021). This COVID-19 racial inequity, as well as recent highly publicized racial injustices, have highlighted the marginalization of certain communities in the current climate—prompting the need for research and intervention efforts for these groups. Such examples include the SIOP Antiracism Grant and the APF EnVISION Ending Racism Grants.

“As a result of COVID-19 and racial injustice, I have shifted my focus a little to explore mental health among Black activists,” says Turner. “A group of my colleagues and I have recently submitted an article on the topic, and I hope to further explore ways to facilitate healing among this group. Furthermore, COVID-19 has provided opportunities for me to engage in public education around coping with the pandemic and helping families manage stress among youth. In my role as president of APA Div. 37 (Society for Child and Family Policy and Practice), we partnered with APA, Mental Health America and other organizations to offer webinars and resources to help families navigate the pandemic.” 

Some of the impacts of the pandemic have been positive, providing opportunities for growth for individuals and the scientific community. For example, many conferences pivoted to a virtual format and drastically reduced the cost of registration and attendance, allowing for greater and more inclusive participation. Many journals removed financial barriers to access articles about COVID-19 so the information would reach a wider audience. Scientists have made greater use of alternative platforms for disseminating research findings such as PsyArXiv and of online data collection systems, which have responded by increasing research capacity.

“Many aspects of the system I used to create my study for online distribution were rigorously improved by the amazing scientists who maintain these systems,” says Stanfield-Wiswell. “The international research psychology community rallied together to create reasonable solutions to work around in-person data collection halts. This experience has taught me how to develop more thorough instructions for participants and it has given me the valuable opportunity to improve my coding ability.”

The COVID-19 pandemic has highlighted the compounding stress graduate students experience.

“As a student teaching assistant, you realize the lack of communication there is from the administrative level to graduate students helping to navigate that weird position of trying to meet program qualifications while also serving as a university employee or being involved in campus committees,” says Gray. “I think it opens up an interesting conversation about the value of graduate students and what universities are willing to do to take responsibility for things like burnout from having to balance these multiple roles or the financial impact that may be larger than we may currently realize.”

Despite these challenges, psychological researchers have shown their ability to adapt during these trying times. These experiences will likely change the way we continue to approach research post-COVID-19.

“Our scientific community was already strong, yet made stronger through these difficult times. Via social media, message boards and Zoom, we developed and nurtured professional relationships that often led to collaborations with others from across the globe, not just within our local circles,” says Stanfield-Wiswell. “The way we approach research in 2021 and beyond will be forever marked by our experiences during the COVID-19 global pandemic.”

About the author

Centers for Disease Control and Prevention. (2021). Health Equity Considerations and Racial and Ethnic Minority Groups . Retrieved from https://www.cdc.gov/coronavirus/2019-ncov/community/health-equity/race-ethnicity.html

Chen, N., Zhou, M., Dong, X., Qu, J., Gong, F., Han, Y., ... and Zhang, L. (2020). Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The lancet, 395 (10223), 507-513.

Chenneville, T. and Schwartz-Mette, R. (2020). Ethical considerations for psychologists in the time of COVID-19. American Psychologist .

Lourenco, S. F. and Tasimi, A. (2020). No participant left behind: conducting science during COVID-19. Trends in Cognitive Sciences, 24 (8), 583-584.

Thompson, K. J. (2020). The perils of practicum in the time of COVID-19: A graduate student’s perspective. Psychological Trauma: Theory, Research, Practice, and Policy, 12 (S1), S151.

Wolkewitz, M. and Puljak, L. (2020). Methodological challenges of analysing COVID-19 data during the pandemic. BMC Medical Research Methodology, 20 (1). 

Xu, X., Chen, P., Wang, J., Feng, J., Zhou, H., Li, X., ... and Hao, P. (2020). Evolution of the novel coronavirus from the ongoing Wuhan outbreak and modeling of its spike protein for risk of human transmission. Science China Life Sciences, 63 (3), 457-460

Contact Science Directorate

You may also like.

The independent source for health policy research, polling, and news.

Global COVID-19 Tracker

Published: Jun 17, 2024

• Cases and Deaths
• Policy Actions

This tracker provides the cumulative number of confirmed COVID-19 cases and deaths, as well as the rate of daily COVID-19 cases and deaths by country, income, region, and globally. It will be updated weekly, as new data are released. As of March 7, 2023, all data on COVID-19 cases and deaths are drawn from the World Health Organization’s (WHO) Coronavirus (COVID-19) Dashboard . Prior to March 7, 2023, this tracker relied on data provided by the Johns Hopkins University (JHU) Coronavirus Resource Center’s COVID-19 Map, which ended on March 10, 2023. Please see the Methods tab for more detailed information on data sources and notes. To prevent slow load times, the tracker only contains data from the last 200 days. However, the full data set can be downloaded from our GitHub page .

Note: The data in this tool were corrected on March 18, 2024, to clarify that they represent new cases and deaths over a full week rather than the average per day over a seven-day period.

• Coronavirus (COVID-19)
• Coronavirus

Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Household Pulse Survey

As part of an ongoing partnership with the Census Bureau, the National Center for Health Statistics (NCHS) recently added questions to assess the prevalence of post-COVID-19 conditions (long COVID), on the experimental Household Pulse Survey. This 20-minute online survey was designed to complement the ability of the federal statistical system to rapidly respond and provide relevant information about the impact of the coronavirus pandemic in the U.S. Data collection began on April 23, 2020.

Beginning in Phase 3.5 (on June 1, 2022), NCHS included questions about the presence of symptoms of COVID that lasted three months or longer.  Beginning in Phase 3.6 (on September 14, 2022), NCHS included a question about whether long-term symptoms among those reporting symptoms lasting three months or longer reduced the ability to carry out day-to-day activities compared with the time before having COVID-19.  Phase 3.6 will continue with a two-weeks on, two-weeks off collection and dissemination approach.

Estimates on this page are derived from the Household Pulse Survey and show the following outcomes for adults aged 18 and over:

• The percentage of all U.S. adults who EVER experienced post-COVID conditions (long COVID). These adults had COVID and had some symptoms that lasted three months or longer.
• The percentage of adults who EVER experienced post-COVID conditions (long COVID) among those who ever had COVID .
• The percentage of all U.S. adults who are CURRENTLY experiencing post-COVID conditions (long COVID). These adults had COVID, had long-term symptoms, and are still experiencing symptoms.
• The percentage of adults who are CURRENTLY experiencing post-COVID conditions (long COVID) among those who ever had COVID.

Beginning in Phase 3.6:

• The percentage of any activity limitations (either ‘yes, a little’ or ‘yes, a lot’ responses) from long COVID, among adults who are currently experiencing long COVID and among all adults
• The percentage of significant activity limitations (‘yes, a lot’ response) from long COVID, among adults who are currently experiencing long COVID and among all adults

The percentage of all U.S. adults who ever said they had COVID is also included to provide context for the other percentages. It should be noted that the percentage of adults who said they ever had COVID based on the Household Pulse Survey is lower than other estimates based on seroprevalence studies .

See the technical notes  for more information on these measures.

Questions on post-COVID conditions (long COVID) were also included on the National Health Interview Survey (NHIS) in 2022.  The NHIS, conducted by NCHS, is the major source for high-quality data used to monitor the nation’s health. NHIS data collection will continue through December 2024.

• Anxiety and Depression
• Health Insurance Coverage
• Lack of Social Connection
• Functioning and Disability
• Telemedicine Use
• Mental Health Care
• Reduced Access to Care

Long COVID or Post COVID Conditions

Use the drop-down menus to show data for selected indicators or categories. Select the buttons at the bottom of the dashboard to view national and state estimates. The data table may be scrolled horizontally and vertically to view additional estimates.

Access Dataset on Data.CDC.gov (Export to CSV, JSON, XLS, XML) [?]

Beginning in Phase 4.1 (April 2, 2024) of data collection and reporting, the answer choices for the third question in the series changed to Yes, my symptoms lasted between 3 and 6 months; Yes, my symptoms lasted 6 months to a year; Yes, my symptoms lasted more than a year; No. The definitions of ever and currently have Long COVID remain the same.

Technical Notes

Survey questions.

Have you ever tested positive for COVID-19 (using a rapid point-of-care test, self-test, or laboratory test) or been told by a doctor or other health care provider that you have or had COVID-19?

Answer Choices:  yes, no

How would you describe your coronavirus symptoms when they were at their worst?

Answer choices: I had no symptoms, I had mild symptoms, I had moderate symptoms, I had severe symptoms.

Did you have any symptoms lasting 3 months or longer that you did not have prior to having coronavirus or COVID-19?

Long term symptoms may include: Tiredness or fatigue, difficulty thinking, concentrating, forgetfulness, or memory problems (sometimes referred to as “brain fog”), difficulty breathing or shortness of breath, joint or muscle pain, fast-beating or pounding heart (also known as heart palpitations), chest pain, dizziness on standing, menstrual changes, changes to taste/smell, or inability to exercise.

Answer choices: yes, no

Do you have symptoms now?

Do these long-term symptoms reduce your ability to carry out day-to-day activities compared with the time before you had COVID-19?

Answer choices: Yes, a lot; Yes, a little; Not at all

Data Source

The U.S. Census Bureau, in collaboration with multiple federal agencies, launched the Household Pulse Survey to produce data on the social and economic impacts of COVID-19 on American households.  The Household Pulse Survey was designed to gauge the impact of the pandemic on employment status, consumer spending, food security, housing, education disruptions, and dimensions of physical and mental wellness.

The survey was designed to meet the goal of accurate and timely weekly estimates. It was conducted by an internet questionnaire, with invitations to participate sent by email and text message. The sample frame is the Census Bureau Master Address File Data. Housing units linked to one or more email addresses or cell phone numbers were randomly selected to participate, and one respondent from each housing unit was selected to respond for him or herself. Estimates are weighted to adjust for nonresponse and to match Census Bureau estimates of the population by age, sex, race and ethnicity, and educational attainment. All estimates shown meet the NCHS Data Presentation Standards for Proportions .

Limitations

The Household Pulse Survey is different from other surveys.  NCHS, the Census Bureau, and other federal statistical agencies are considered the preeminent source of the nation’s most important benchmark surveys.  Many of these surveys have been in production for decades and provide valuable insight on health, social, and economic trends. However, the production of benchmark data requires a relatively long lead time, and personal interviews (face-to-face or telephone) require additional time.  While efforts are underway to introduce COVID-19 questions into these surveys, that process can take months, sometimes years, before data are made available.

The Household Pulse Survey is different:  It was designed to go into the field quickly, to be administered via the web, and to disseminate data in near real-time, providing data users with information they can use now to help ease the burden on American households and expedite post-pandemic recovery.  The Census Bureau is fielding the Household Pulse Survey as a demonstration project, with data released as part of its Experimental Statistical Products Series.

Confidence intervals included in the tables on this page only reflect the potential for sampling error.  Nonsampling errors can also occur and are more likely for surveys that are implemented quickly, achieve low response rates, and rely on online response.  Nonsampling errors for the Household Pulse Survey may include:

• Measurement error: The respondent provides incorrect information, or an unclear survey question is misunderstood by the respondent. The Household Pulse Survey schedule offered only limited time for testing questions.
• Coverage error: Individuals who otherwise would have been included in the survey frame were missed. The Household Pulse Survey only recruited households for which an email address or cell phone number could be identified.
• Nonresponse error: Responses are not collected from all those in the sample or the respondent is unwilling to provide information. The response rate for the Household Pulse Survey was substantially lower than most federally sponsored surveys.
• Processing error: Forms may be lost, data may be incorrectly keyed, coded, or recoded. The real-time dissemination of the Household Pulse Survey provided limited time to identify and fix processing errors.

For more information on nonresponse bias for the Household Pulse Survey, please visit  https://www2.census.gov/programs-surveys/demo/technical-documentation/hhp/2020_HPS_NR_Bias_Report-final.pdf .

For more information on the Household Pulse Survey, please visit  https://www.census.gov/data/experimental-data-products/household-pulse-survey.html .

Suggested Citation

National Center for Health Statistics. U.S. Census Bureau, Household Pulse Survey, 2022–2024. Long COVID. Generated interactively: from https://www.cdc.gov/nchs/covid19/pulse/long-covid.htm

Exit Notification / Disclaimer Policy

• The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
• Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
• You will be subject to the destination website's privacy policy when you follow the link.
• CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

• Alzheimer's disease & dementia
• Arthritis & Rheumatism
• Attention deficit disorders
• Autism spectrum disorders
• Biomedical technology
• Diseases, Conditions, Syndromes
• Endocrinology & Metabolism
• Gastroenterology
• Gerontology & Geriatrics
• Health informatics
• Inflammatory disorders
• Medical economics
• Medical research
• Medications
• Neuroscience
• Obstetrics & gynaecology
• Oncology & Cancer
• Ophthalmology
• Overweight & Obesity
• Parkinson's & Movement disorders
• Psychology & Psychiatry
• Radiology & Imaging
• Sleep disorders
• Sports medicine & Kinesiology
• Vaccination
• Breast cancer
• Cardiovascular disease
• Chronic obstructive pulmonary disease
• Colon cancer
• Coronary artery disease
• Heart attack
• Heart disease
• High blood pressure
• Kidney disease
• Lung cancer
• Multiple sclerosis
• Myocardial infarction
• Ovarian cancer
• Post traumatic stress disorder
• Rheumatoid arthritis
• Schizophrenia
• Skin cancer
• Type 2 diabetes
• Full List »

share this!

June 17, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

Study sheds light on factors that may predispose some COVID patients to recover more slowly

by Columbia University Irving Medical Center

Early in the pandemic, many people who had SARS-CoV-2 infection or COVID-19 began to report that they couldn't shake off their symptoms even after a month or more—unusually long for a viral infection of the upper respiratory tract—or developed new, persistent symptoms soon after the infection cleared.

Although it's still not clear what causes post-COVID-19 conditions or "long COVID" (symptoms and conditions that develop, linger, or reoccur weeks or months after SARS-CoV-2 infection), a new study by researchers at Columbia University Vagelos College of Physicians and Surgeons confirms the high burden of long COVID and sheds light on who's at greatest risk.

The study is titled, "Epidemiologic Features of Recovery from SARS-CoV-2 Infection." It was published online June 17 in JAMA Network Open .

The study found that people with a milder infection—including those who were vaccinated against SARS-CoV-2 and those who were infected with an omicron variant—were more likely to recover quickly.

Recovery time was similar for subsequent infections.

"Our study underscores the important role that vaccination against COVID has played, not just in reducing the severity of an infection but also in reducing the risk of long COVID," says Elizabeth C. Oelsner, the study's lead author and the Herbert Irving Associate Professor of Medicine.

The study involved over 4,700 participants from the Collaborative Cohort of Cohorts for COVID 19 Research (C4R), who were asked to report their time to recovery after infection with SARS-CoV-2.

The study found that, between 2020 and early 2023, the median recovery time after SARS-CoV2-infection was 20 days, and more than one in five adults did not recover within three months.

Women and adults with pre-pandemic cardiovascular disease were less likely to recover within three months. Other pre-pandemic health conditions—including chronic kidney disease , diabetes, asthma, chronic lung disease , depressive symptoms , and a history of smoking—were linked to longer recovery times, but these associations were no longer significant after accounting for sex, cardiovascular disease, vaccination, and variant exposure.

"Although studies have suggested that many patients with long COVID experience mental health challenges, we did not find that depressive symptoms prior to SARS-CoV-2 infection were a major risk factor for long COVID."

Other groups disproportionately affected by long COVID were American Indian and Alaska Native participants, in whom severe infections and longer recovery times were more common.

"Our study clearly establishes that long COVID posed a substantial personal and societal burden," says Oelsner. "By identifying who was likely to have experienced a lengthy recovery, we have a better understanding of who should be involved in ongoing studies of how to lessen or prevent the long-term effects of SARS-CoV-2 infection."

Explore further

Feedback to editors

Study suggests AI may soon be able to detect cancer

3 hours ago

Walking brings huge benefits for low back pain, study finds

9 hours ago

Combination targeted treatment produces lasting remissions in people with resistant aggressive B-cell lymphoma

11 hours ago

Study finds one copy of protective genetic variant helps stave off early-onset Alzheimer's disease

Drugs for enlarged prostate may also protect against dementia with Lewy bodies

12 hours ago

New study establishes best practices for supervised psilocybin

During a heat wave, high indoor temperatures can also prove dangerous

Scientists reveal how an unstructured protein traps cancer-promoting molecules

Addressing maternal prenatal depression can lead to longer gestation, researchers say

14 hours ago

Researchers develop portable serological test for rapid COVID-19 immunity monitoring

Related stories.

People with prior illness more likely to report longer symptoms after COVID-19 infection, research finds

Nov 7, 2023

Do allergic conditions increase the risk of developing long COVID after SARS-CoV-2 infection?

Nov 8, 2023

New study reveals long-term mental health risks after COVID-19

Mar 21, 2024

Post-COVID-19 conditions tied to higher health care utilization

Jan 20, 2023

Study provides insights into impact of COVID-19 and vaccination on mother and child health

Nov 16, 2023

More than 1 in 6 unvaccinated people report health effects of COVID two years after confirmed infection

May 31, 2023

Recommended for you

Immune response study explains why some people don't get COVID-19

17 hours ago

Researchers find bigger immune response to flu variants in people who were exposed in childhood

16 hours ago

Researchers discover a potential vaccine to prevent tuberculosis in people of all ages

Open-source project maps the biology of spinal cord injury in unprecedented detail

New tuberculosis vaccine candidate shows promise with post-translational modifications

Customizable AI tool helps pathologists identify diseased cells

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Medical Xpress in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

COVID-19 and the workplace: Implications, issues, and insights for future research and action

Affiliations.

• 1 Dyson School of Applied Economics and Management, S. C. Johnson College of Business, Cornell University.
• 2 Department of Management and Organizations, National University of Singapore.
• 3 Business School, University of New South Wales Sydney.
• 4 Department of Organizational Behavior, University of Lausanne.
• 5 Stephen M. Ross School of Business, University of Michigan.
• 6 Center of Excellence for Positive Organizational Psychology, Erasmus University Rotterdam.
• 7 Coller School of Management, Tel Aviv University.
• 8 Department of Management and Marketing, University of Melbourne.
• 9 Department of Organizational Behaviour and Human Resources, Singapore Management University.
• 10 Department of Psychology, University of Maryland, College Park.
• 11 The Wharton School, University of Pennsylvania.
• 12 Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology.
• 13 Graduate School of Business, Stanford University.
• 14 John Molson School of Business, Concordia University.
• 15 Department of Organizational Behaviour, London Business School.
• 16 Hankamer School of Business, Baylor University.
• 17 School of Management, University College London.
• 18 College of Business Administration, California State University, Sacramento.
• 20 Department of Psychology, Saint Louis University.
• 21 Nanyang Business School, Nanyang Technology University.
• 22 Lee Kong Chian School of Business, Singapore Management University.
• 23 Department of Work and Organizations, University of Minnesota.
• 24 Harvard Business School, Harvard University.
• 25 Sam M. Walton College of Business, University of Arkansas.
• 26 Department of Organizational Psychology, Vrije Universiteit Amsterdam.
• PMID: 32772537
• DOI: 10.1037/amp0000716

The impacts of COVID-19 on workers and workplaces across the globe have been dramatic. This broad review of prior research rooted in work and organizational psychology, and related fields, is intended to make sense of the implications for employees, teams, and work organizations. This review and preview of relevant literatures focuses on (a) emergent changes in work practices (e.g., working from home, virtual teamwork) and (b) emergent changes for workers (e.g., social distancing, stress, and unemployment). In addition, potential moderating factors (demographic characteristics, individual differences, and organizational norms) are examined given the likelihood that COVID-19 will generate disparate effects. This broad-scope overview provides an integrative approach for considering the implications of COVID-19 for work, workers, and organizations while also identifying issues for future research and insights to inform solutions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

PubMed Disclaimer

Similar articles

• Consequences of COVID-19 on Employees in Remote Working: Challenges, Risks and Opportunities An Evidence-Based Literature Review. De Vincenzi C, Pansini M, Ferrara B, Buonomo I, Benevene P. De Vincenzi C, et al. Int J Environ Res Public Health. 2022 Sep 16;19(18):11672. doi: 10.3390/ijerph191811672. Int J Environ Res Public Health. 2022. PMID: 36141948 Free PMC article. Review.
• COVID-19, Physical Distancing in the Workplace and Employees' Mental Health: Implications and Insights for Organizational Interventions - Narrative Review. Hamouche S. Hamouche S. Psychiatr Danub. 2021 Summer;33(2):202-208. doi: 10.24869/psyd.2021.202. Psychiatr Danub. 2021. PMID: 34185751 Review.
• Making daily decisions to work from home or to work in the office: The impacts of daily work- and COVID-related stressors on next-day work location. Shao Y, Fang Y, Wang M, Chang CD, Wang L. Shao Y, et al. J Appl Psychol. 2021 Jun;106(6):825-838. doi: 10.1037/apl0000929. J Appl Psychol. 2021. PMID: 34138589
• Dormant tie reactivation as an affiliative coping response to stressors during the COVID-19 crisis. Yang SW, Soltis SM, Ross JR, Labianca GJ. Yang SW, et al. J Appl Psychol. 2021 Apr;106(4):489-500. doi: 10.1037/apl0000909. J Appl Psychol. 2021. PMID: 34014705
• Videoconference fatigue? Exploring changes in fatigue after videoconference meetings during COVID-19. Bennett AA, Campion ED, Keeler KR, Keener SK. Bennett AA, et al. J Appl Psychol. 2021 Mar;106(3):330-344. doi: 10.1037/apl0000906. J Appl Psychol. 2021. PMID: 33871270
• Work engagement and sense of coherence as predictors of psychological distress during the first phase of the COVID-19 pandemic in Chile. Gómez-Salgado J, Delgado-García D, Ortega-Moreno M, Fagundo-Rivera J, El Khoury-Moreno L, Vilches-Arenas Á, Ruiz-Frutos C. Gómez-Salgado J, et al. Heliyon. 2024 May 15;10(10):e31327. doi: 10.1016/j.heliyon.2024.e31327. eCollection 2024 May 30. Heliyon. 2024. PMID: 38803934 Free PMC article.
• Rise of remote work across borders: opportunities and implications for migrant-sending countries. Mieriņa I, Šūpule I. Mieriņa I, et al. Front Sociol. 2024 May 7;9:1290629. doi: 10.3389/fsoc.2024.1290629. eCollection 2024. Front Sociol. 2024. PMID: 38774032 Free PMC article.
• What is the volume, quality and characteristics of evidence relating to the effectiveness and cost-effectiveness of multi-disciplinary occupational health interventions aiming to improve work-related outcomes for employed adults? An evidence and gap map of systematic reviews. Shaw E, Nunns M, Spicer SG, Lawal H, Briscoe S, Melendez-Torres GJ, Garside R, Liabo K, Coon JT. Shaw E, et al. Campbell Syst Rev. 2024 May 14;20(2):e1412. doi: 10.1002/cl2.1412. eCollection 2024 Jun. Campbell Syst Rev. 2024. PMID: 38751859 Free PMC article.
• Living the employer brand during a crisis? A qualitative study on internal employer branding in times of the COVID-19 pandemic. Rys M, Schollaert E, Van Hoye G. Rys M, et al. PLoS One. 2024 May 13;19(5):e0303361. doi: 10.1371/journal.pone.0303361. eCollection 2024. PLoS One. 2024. PMID: 38739576 Free PMC article.
• The Impact of the Covid-19 Pandemic on Older Workers: The Role of Self-Regulation and Organizations. Kooij DTAM. Kooij DTAM. Work Aging Retire. 2020 Sep 18:waaa018. doi: 10.1093/workar/waaa018. Work Aging Retire. 2020. PMID: 38626227 Free PMC article.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• American Psychological Association
• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations
• MedlinePlus Health Information
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Advertisement

Supported by

New Report Underscores the Seriousness of Long Covid

The National Academies said the condition could involve up to 200 symptoms, make it difficult for people to work and last for months or years.

• Share full article

By Pam Belluck

One of the nation’s premier medical advisory organizations has weighed in on long Covid with a 265-page report that recognizes the seriousness and persistence of the condition for millions of Americans.

More than four years since the start of the coronavirus pandemic, long Covid continues to damage many people’s ability to function, according to the National Academies of Sciences, Engineering and Medicine, a nongovernmental institution that advises federal agencies on science and medicine.

“Long Covid can impact people across the life span, from children to older adults, as well as across sex, gender, racial, ethnic and other demographic groups,” it said, concluding that “long Covid is associated with a wide range of new or worsening health conditions and encompasses more than 200 symptoms involving nearly every organ system.”

Here are some of the National Academies’ findings, drafted by a committee of 14 doctors and researchers:

How many people have long Covid?

The report cited data from 2022 suggesting that nearly 18 million adults and nearly a million children in the United States have had long Covid at some point. At the time of that survey, about 8.9 million adults and 362,000 children had the condition.

Surveys showed that the prevalence of long Covid decreased in 2023 but, for unclear reasons, has risen this year. As of January, data showed nearly 7 percent of adults in the United States had long Covid.

Diagnosis and consequences

There is still no standardized way to diagnose the condition and no definitive treatments to cure it. “There is no one-size-fits-all approach to rehabilitation, and each individual will need a program tailored to their complex needs,” the National Academies said, advising that doctors should not require patients to have a positive coronavirus test to be diagnosed with long Covid.

We are having trouble retrieving the article content.

Please enable JavaScript in your browser settings.

Thank you for your patience while we verify access. If you are in Reader mode please exit and  log into  your Times account, or  subscribe  for all of The Times.

Thank you for your patience while we verify access.

Already a subscriber?  Log in .

Want all of The Times?  Subscribe .

TechRepublic

Artificial intelligence.

The 10 Best AI Courses in 2024

Today’s options for best AI courses offer a wide variety of hands-on experience with generative AI, machine learning and AI algorithms.

ChatGPT Cheat Sheet: A Complete Guide for 2024

Get up and running with ChatGPT with this comprehensive cheat sheet. Learn everything from how to sign up for free to enterprise use cases, and start using ChatGPT quickly and effectively.

Microsoft Copilot Cheat Sheet: Price, Versions & Benefits

The Surface Laptop 7 and Surface Pro 11, which ship in June, will be able to run Copilot features on-device.

Llama 3 Cheat Sheet: A Complete Guide for 2024

Learn how to access Meta’s new AI model Llama 3, which sets itself apart by being open to use under a license agreement.

OpenAI’s GPT-4 Can Autonomously Exploit 87% of One-Day Vulnerabilities, Study Finds

Researchers from the University of Illinois Urbana-Champaign found that OpenAI’s GPT-4 is able to exploit 87% of a list of vulnerabilities when provided with their NIST descriptions.

Latest Articles

macOS 15 Sequoia Cheat Sheet: Release Date, Name, Features and More

Learn about the new features available with macOS 15, as well as how to download and install the latest version of Apple’s flagship operating system.

AWS Offers Free AI and Machine Learning Courses and Certifications

Two certifications that verify skills in building AI and ML on AWS are coming in August.

Microsoft Delays Recall Launch, Seeking Community Feedback First

An upcoming blog post for members of the Windows Insider Program will explain how to get the AI-powered Recall feature.

The 5 Best Laptop Deals for June 2024

The best laptop deals can help you get more done during the workday without emptying your bank account.

6 Best CRMs for Small Business in 2024

Find the perfect CRM for your small business in our detailed review. Explore features, pricing and more.

iPadOS 18 Cheat Sheet: Beta, Release Date, New Features Including Calculator

This is a complete guide for Apple's iPadOS, which includes iPadOS 18's supported devices, release dates and key features such as Calculator.

Apple iOS 18 Cheat Sheet: Release Date, RCS Integration and More

iOS 18 includes artificial intelligence features, new apps and much more. Learn how registered developers can install the iOS 18 beta.

Apple’s Siri Cheat Sheet: How to Use Siri and its Apple Intelligence Capabilities for Business

During WWDC 2024, Siri was given its largest overhaul since it was introduced back in 2011. Learn how to make the most of all the new features, including Apple Intelligence.

UK Trails Behind Europe in Technical Skills Proficiency, Coursera Report Finds

The U.K. is the 25th most technically proficient country in Europe, with Switzerland, Germany and the Netherlands taking the top three places.

WWDC: Apple Intelligence Brings Generative AI to Mail, Messaging and More

Apple Intelligence will let Siri use generative AI to navigate your phone and check your calendar. It can reach out to ChatGPT for help.

Apple WWDC Keynote: iOS 18, iPad OS 18 and macOS 15 Sequoia Coming In Fall

At WWDC 2024, Apple showed how macOS 15 Sequoia will let you control your iPhone from your laptop for a quick productivity boost.

OpenAI, Anthropic Research Reveals More About How LLMs Affect Security and Bias

Anthropic opened a window into the ‘black box’ where ‘features’ steer a large language model’s output. OpenAI dug into the same concept two weeks later with a deep dive into sparse autoencoders.

AWS Custom Silicon Chips Range a Sign of What’s Coming to APAC Cloud Computing

Rampant demand for AI computing is driving cloud providers like Microsoft, Google and AWS to build custom silicon chips. Soon, APAC customers will have more choice based on use case and cost.

Get an Introduction to AI Services Like ChatGPT for Just $50

This three-course bundle teaches you how to use ChatGPT and Midjourney to full effect to help your business. It's on sale for $100 off now.

Create a TechRepublic Account

Get the web's best business technology news, tutorials, reviews, trends, and analysis—in your inbox. Let's start with the basics.

* - indicates required fields

Sign in to TechRepublic

Lost your password? Request a new password

Reset Password

Please enter your email adress. You will receive an email message with instructions on how to reset your password.

Check your email for a password reset link. If you didn't receive an email don't forgot to check your spam folder, otherwise contact support .

Welcome. Tell us a little bit about you.

This will help us provide you with customized content.

Want to receive more TechRepublic news?

You're all set.

Thanks for signing up! Keep an eye out for a confirmation email from our team. To ensure any newsletters you subscribed to hit your inbox, make sure to add [email protected] to your contacts list.

• Our Program Divisions
• Our Three Academies
• Government Affairs
• Statement on Diversity and Inclusion
• Our Study Process
• Conflict of Interest Policies and Procedures
• Project Comments and Information
• Read Our Expert Reports and Published Proceedings
• Explore PNAS, the Flagship Scientific Journal of NAS
• Access Transportation Research Board Publications
• Coronavirus Disease 2019 (COVID-19)
• Diversity, Equity, and Inclusion
• Economic Recovery
• Fellowships and Grants
• Publications by Division
• Division of Behavioral and Social Sciences and Education
• Division on Earth and Life Studies
• Division on Engineering and Physical Sciences
• Gulf Research Program
• Health and Medicine Division
• Policy and Global Affairs Division
• Transportation Research Board
• National Academy of Sciences
• National Academy of Engineering
• National Academy of Medicine
• Publications by Topic
• Agriculture
• Behavioral and Social Sciences
• Biography and Autobiography
• Biology and Life Sciences
• Computers and Information Technology
• Conflict and Security Issues
• Earth Sciences
• Energy and Energy Conservation
• Engineering and Technology
• Environment and Environmental Studies
• Food and Nutrition
• Health and Medicine
• Industry and Labor
• Math, Chemistry, and Physics
• Policy for Science and Technology
• Space and Aeronautics
• Surveys and Statistics
• Transportation and Infrastructure
• Searchable Collections
• New Releases

VIEW LARGER COVER

Long-Term Health Effects of COVID-19

Disability and function following sars-cov-2 infection.

Since the onset of the coronavirus disease 2019 (COVID-19) pandemic in early 2020, many individuals infected with the virus that causes COVID-19, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), have continued to experience lingering symptoms for months or even years following infection. Some symptoms can affect a person's ability to work or attend school for an extended period of time. Consequently, in 2022, the Social Security Administration requested that the National Academies convene a committee of relevant experts to investigate and provide an overview of the current status of diagnosis, treatment, and prognosis of long-term health effects related to Long COVID. This report presents the committee conclusions.

RESOURCES AT A GLANCE

• Press Release
• Health and Medicine — Infectious Disease

Suggested Citation

National Academies of Sciences, Engineering, and Medicine. 2024. Long-Term Health Effects of COVID-19: Disability and Function Following SARS-CoV-2 Infection . Washington, DC: The National Academies Press. https://doi.org/10.17226/27756. Import this citation to: Bibtex EndNote Reference Manager

Publication Info

• Prepublication:  978-0-309-72082-3
• Paperback (forthcoming):  978-0-309-71860-8

What is skim?

The Chapter Skim search tool presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter. You may select key terms to highlight them within pages of each chapter.

Copyright Information

The National Academies Press (NAP) has partnered with Copyright Clearance Center's Marketplace service to offer you a variety of options for reusing NAP content. Through Marketplace, you may request permission to reprint NAP content in another publication, course pack, secure website, or other media. Marketplace allows you to instantly obtain permission, pay related fees, and print a license directly from the NAP website. The complete terms and conditions of your reuse license can be found in the license agreement that will be made available to you during the online order process. To request permission through Marketplace you are required to create an account by filling out a simple online form. The following list describes license reuses offered by the NAP through Marketplace:

• Republish text, tables, figures, or images in print
• Post on a secure Intranet/Extranet website
• Use in a PowerPoint Presentation
• Distribute via CD-ROM

Click here to obtain permission for the above reuses. If you have questions or comments concerning the Marketplace service, please contact:

Marketplace Support International +1.978.646.2600 US Toll Free +1.855.239.3415 E-mail: [email protected] marketplace.copyright.com

To request permission to distribute a PDF, please contact our Customer Service Department at [email protected] .

What is a prepublication?

An uncorrected copy, or prepublication, is an uncorrected proof of the book. We publish prepublications to facilitate timely access to the committee's findings.

You can purchase the uncorrected copy now, or pre-order the final version of the book. Prepublication sales are final. Due to the nature of this type of publication, it is not returnable.

What happens when I pre-order?

The final version of this book has not been published yet. You can pre-order a copy of the book and we will send it to you when it becomes available. We will not charge you for the book until it ships. Pricing for a pre-ordered book is estimated and subject to change. All backorders will be released at the final established price. As a courtesy, if the price increases by more than $3.00 we will notify you. If the price decreases, we will simply charge the lower price. Applicable discounts will be extended.

Downloading and Using eBooks from NAP

What is an ebook.

An ebook is one of two file formats that are intended to be used with e-reader devices and apps such as Amazon Kindle or Apple iBooks.

Why is an eBook better than a PDF?

A PDF is a digital representation of the print book, so while it can be loaded into most e-reader programs, it doesn't allow for resizable text or advanced, interactive functionality. The eBook is optimized for e-reader devices and apps, which means that it offers a much better digital reading experience than a PDF, including resizable text and interactive features (when available).

Where do I get eBook files?

eBook files are now available for a large number of reports on the NAP.edu website. If an eBook is available, you'll see the option to purchase it on the book page.

View more FAQ's about Ebooks

Types of Publications

Consensus Study Report: Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Pak J Med Sci
• v.36(COVID19-S4); 2020 May

Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era

Khadijah mukhtar.

1 Khadijah Mukhtar, BDS, MME. Assistant Professor, DME. University College of Medicine and Dentistry, The University of Lahore, Lahore, Pakistan

Kainat Javed

2 Kainat Javed, MBBS, MME. Assistant Professor, DME. University College of Medicine and Dentistry, The University of Lahore, Lahore, Pakistan

Mahwish Arooj

3 Mahwish Arooj, MBBS, M. Phil, MME, PhD Physiology. Associate Professor, Physiology and Director DME, University College of Medicine and Dentistry, The University of Lahore, Lahore, Pakistan

Ahsan Sethi

4 Ahsan Sethi, BDS, MPH, MMEd, FHEA, MAcadMEd, FDTFEd, PhD Medical Education Assistant Professor, Institute of Health Professions Education and Research, Khyber Medical University, Peshawar, Pakistan

During COVID-19 pandemic, the institutions in Pakistan have started online learning. This study explores the perception of teachers and students regarding its advantages, limitations and recommendations.

This qualitative case study was conducted from March to April 2020. Using maximum variation sampling, 12 faculty members and 12 students from University College of Medicine and University College of Dentistry, Lahore were invited to participate. Four focus group interviews, two each with the faculty and students of medicine and dentistry were carried out. Data were transcribed verbatim and thematically analyzed using Atlas Ti.

The advantages included remote learning, comfort, accessibility, while the limitations involved inefficiency and difficulty in maintaining academic integrity. The recommendations were to train faculty on using online modalities and developing lesson plan with reduced cognitive load and increased interactivities.

Conclusion:

The current study supports the use of online learning in medical and dental institutes, considering its various advantages. Online learning modalities encourage student-centered learning and they are easily manageable during this lockdown situation.

INTRODUCTION

The spread of COVID-19 has led to the closure of educational institutions all over the world. This tested the preparedness of universities to deal with a crisis that requires the help of advanced technology including hardware and software to enable effective online learning. Such closure accelerated the development of the online learning environments so that learning would not be disrupted. 1 Many institutions have become interested in how to best deliver course content online, engage learners and conduct assessments. Hence, COVID-19 while being a hazard to humanity, has evolved institutions to invest in online learning.

Online learning systems are web-based software for distributing, tracking, and managing courses over the Internet. 2 It involves the implementation of advancements in technology to direct, design and deliver the learning content, and to facilitate two-way communication between students and faculty. 3 They contain features such as whiteboards, chat rooms, polls, quizzes, discussion forums and surveys that allow instructors and students to communicate online and share course content side by side. These can offer productive and convenient ways to achieve learning goals. In Pakistan, the institutions are using Microsoft Teams, Google meet, Edmodo and Moodle as learning management systems along with their applications for video conferencing. 4 Other commonly used video conferencing solutions include Zoom, Skype for business, WebEx and Adobe connect etc.

According to our literature review, three previous studies were found, 5 - 7 supporting online learning from Pakistan. The two studies at Dow University of Health Sciences, Karachi and Lahore Medical and Dental College, Lahore reported high student satisfaction with online learning modalities. The study from Khyber Pakhtunkhwa assessed the feasibility of online learning among students, trainees and faculty members. They reported good technology access, online skills, and preparedness for online discussions among participants across the medical education continuum.

With the increase in use of online modalities during COVID-19, it is necessary to assess their effectiveness with regards to teaching and learning from various stakeholders. 8 Therefore, the current study explores the perception of faculty members and students regarding the advantages, limitations and recommendations for online learning in Pakistan. The study is timely as Higher Education Commission (HEC) is in the process of implementing online learning across all the universities in Pakistan. The findings will help identify the required changes on priority basis to make it more practical and worthwhile.

This qualitative case study was conducted from March to April 2020 in two medical and dental institutes. Ethical approval for this study was taken from ethical review board of University of Lahore (Ref No. ERC/02/20/02, dated February 25, 2020). Using maximum variation sampling 12 faculty members and 12 students from University College of Medicine and University College of Dentistry, Lahore were invited to participate. In addition to learning management system ‘Moodle’, these colleges have recently adopted ‘Zoom’ for interactive teaching in small and large group formats. The participants were also involved in online Problem-Based Learning sessions, along with regular online assessments during COVID-19 pandemic.

An interview guide was developed to explore faculty and students’ perception about online learning modalities, its advantages, limitations and recommendations. The interview guide was piloted to ensure comprehensiveness and then also validated by two medical education experts. 9 Two interviewers who were not involved in teaching and assessment of students conducted four focus group interviews (n=6 in each group) with faculty members (n=12) and students (n=12) of medicine and dentistry. The faculty and students were from both basic sciences (1 st and 2 nd year) and clinical sciences (3 rd , 4 th and final year). All interviews were recorded through ‘Zoom’ and subsequently transcribed verbatim. The data were thematically analyzed: compiling, disassembling, reassembling and interpretation by all the authors independently and then corroborated to ensure analytical triangulation.

The faculty members were predominantly females from both basic and clinical sciences with age range from 30-64 years. The students were from all professional years of MBBS and BDS program ( Table-I ).

Participant characteristics.

Total six themes, two each for advantages, limitations and recommendations were extracted from the transcribed data after qualitative analysis ( Table-II ).

E-learning advantages, limitations and recommendations by Students and Faculty.

Faculty opined that online learning helped ensure remote learning, it was manageable, and students could conveniently access teachers and teaching materials. It also reduced use of traveling resources and other expenses. It eased administrative tasks such as recording of lectures and marking attendance. Both the students and teachers had an opinion that online learning modalities had encouraged student-centeredness during this lockdown situation. The student had become self-directed learners and they learnt asynchronously at any time in a day.

Limitations

Faculty members and students said that through online learning modalities they were unable to teach and learn practical and clinical work. They could only teach and assess knowledge component. Due to lack of immediate feedback, teachers were unable to assess students’ understanding during online lecturing. The students also reported limited attention span and resource intensive nature of online learning as a limitation. Some teachers also mentioned that during online study, students misbehaved and tried to access online resources during assessments.

Recommendations

Teachers and students suggested continuous faculty development. They recommended a reduction in cognitive load and increased interactivities during online teaching. Those in clinical years suggested ways to start online Case Based Learning. However, some were also of the opinion that there should be revision classes along with psychomotor hands on teaching after the COVID-19 pandemic is under control. To enhance quality, they suggested buying premium software and other proctoring software to detect cheating and plagiarism.

The current study reported advantages, limitations and recommendations to improve online learning during lockdown of institutions due to COVID-19 pandemic. This study interprets perspectives of medical/dental students and faculty members, which showed that online learning modalities are flexible and effective source of teaching and learning along with some pitfalls. According to the teachers and students, online learning is a flexible and effective source of teaching and learning as most of them agreed upon the fact that this helps in distant learning with easy administration and accessibility along with less use resource and time. Regardless of time limit, students can easily access the learning material. This flexibility over face to face teaching has been reported in the literature as well. 2 The students also become self-directed learners, which is an important competency for encouraging lifelong learning among health professionals. 10 , 11

Both the faculty members and students viewed inefficiency to teach psychomotor skills, resource intensiveness and mismanaged decorum during sessions as limitations of online learning. Even though, hands-on sessions such as laboratory and clinical skills teaching have been disrupted during COVID-19 pandemic, we believe that online simulated patients or role plays can be used teach history taking, clinical reasoning and communication skills. Sharing recorded videos of laboratory and clinical skills demonstration is also worthwhile. Faculty members also complained about lack of students’ feedback regarding understanding of subject. Research showed that regular two-way feedback helps enhance self-efficacy and motivation. 12 The interaction between facilitator, learner and study material along with emotional and social support are essential ingredients for effective learning. 13 , 14 Internet connectivity issues also adversely impacted learning through online modalities, however, simply improving internet package/speed would help resolve this. Government should also take immediate measures and telecommunication companies should invest in expanding its 4G services across the country.

Recommendations reflect that decorum can be maintained by thorough supervision of students, setting ground rules for online interaction, counselling and disciplinary actions. 15 According to students, the attention span during online learning was even shorter than face to face sessions as also supported by the literature. 16 This can be managed by using flipped classroom learning modalities, giving shorter lectures and increasing teacher-student interaction. As ‘assessment drives learning’, so online formative assessments can be conducted through Socrative and Kahoot etc. Faculty needs training and students orientation in using online learning tools. 17 Investment in buying premium software packages will also help overcome many limitations and is therefore recommended.

Limitations of the Study

As the study participants belonged to the medical and dental college from a single private-sector university of Punjab, therefore the findings are only applicable to similar contexts. For generalizability, a survey based on our findings should be conducted across the province or country. Despite the limitations, the findings offer an understanding of the advantages, limitations and recommendations for improvement in online learning, which is the need of the day.

The current study supports the use of online learning in medical and dental institutes, considering its various advantages. E-learning modalities encourage student-centered learning and they are easily manageable during this lockdown situation. It is worth considering here that currently online learning is at a nascent stage in Pakistan. It started as ‘emergency remote learning’, and with further investments we can overcome any limitations. There is a need to train faculty on the use of online modalities and developing lesson plan with reduced cognitive load and increased interactivities.

Author’s Contribution

AS and MA conceived the idea , designed the study and are responsible for integrity of research.

KM and KJ collected the data.

All the authors contributed towards data analysis and writing the manuscript and approved the final version.

Acknowledgements

The authors would like to acknowledge the participants for their time and contributions.

Conflict of interest: None.