The logo of peer-reviewed academic journal of the American Association for the Advancement of ... [+] Science (AAAS) Science displayed on a computer's screen. (Photo by LOIC VENANCE / AFP) (Photo credit should read LOIC VENANCE/AFP via Getty Images)
Except for the crisis in the Ukraine, the most serious cloud overhanging the global economy right now is the slowdown in China, driven (in part) by the re-imposition of lockdowns and other severe countermeasures in response to the latest Covid outbreaks in several Chinese cities.
This has raised the question of whether China’s Zero-Covid policy is really working, and at what cost – which in turn has called attention to obvious gaps, anomalies and discrepancies in Beijing’s reported figures for Covid infection and mortality.
The media has mostly ignored the data issue until recently, but this week two new reports in the leading international scientific journals, Science and Nature, have spotlighted the problem.
The latest issue of Science reports on a study of Covid mortality statistics in India, which concludes that the official Covid death statistics vastly under-estimate India’s actual death rate.
BENGALURU, INDIA - APRIL 30: Men wearing PPE (Personal Protective Equipment) perform the last rites ... [+] of a deceased relative in a disused granite quarry repurposed to cremate the dead due to COVID-19 on April 30, 2021 in Bengaluru, India. With recorded cases crossing 380,000 a day and 3000 deaths in the last 24 hours, India has more than 2 million active cases of COVID-19, the second-highest number in the world after the U.S. A new wave of the pandemic has totally overwhelmed the country's healthcare services and has caused crematoriums to operate day and night as the number of victims continues to spiral out of control. (Photo by Abhishek Chinnappa/Getty Images)
The study estimates that in 2020 and 2021 there were approximately 3.25 million excess deaths – deaths above the historical trend line – a number 6 to 7 times higher than the official Covid death count (480,000) reported by the Indian government.
The article cites a number of previous studies that also found significant excess mortality for India. A model developed by The Economist magazine (described in my previous columns) gives an even higher number, which Nature magazine calls “sadly plausible”:
Another recent study by the National Bureau of Economic Research puts the estimate for India’s excess deaths at “roughly 6.3 million” — which is higher than the current official Covid death toll for the entire world (~ 5.6 million).
Estimates of India's Covid Deaths
Although the article does not address China specifically, it bears on the general question of the reliability of official Covid data, and the causes of under-reporting that need to be controlled for in any statistical study. The data-quality and completeness issues that confront India also confront China, and although the scale of the discrepancies may be different, they must still be acknowledged. Under-reporting is real and pervasive almost everywhere, and efforts to waive it away because of supposed Chinese exceptionalism do not measure up to proper scientific standards.
The authors of the Science article attribute the under-reporting in India to three factors:
Certification of “cause of death” is one step in the Covid data pipeline where information-loss can occur.
The criterion for deciding if someone has died from Covid is not standardized across the world.
In the case of India, “medical certification remains uncommon.”
The Economist points out that “Many people who die while infected with SARS-CoV-2 are never tested for it, and do not enter the official totals.”
The problem of underreporting in India is apparently worse in the poorer, rural areas of the country. The same is true in the United States.
Do similar problems affect rural China? Because of Beijing’s data suppression, we don’t know. But that is a huge “Don’t Know.” There are more people living in the rural China than in the entire European Union. The presumption (until proven otherwise) must be that certification deficits would likely exist. Because of the enormous numbers involved, any flaws in the cause-of-death certification process in rural China would have a huge potential impact on the numbers.
The Indian study cites “misattribution” of Covid deaths to other causes. How serious is this problem, seen from a statistical and data collection perspective?
Covid elevates the death rate for people with pre-existing conditions or risk factors, like diabetes or heart disease. The effect is very significant. As The Economist’s authors write, “those with other illnesses (“comorbidities”)—die at alarming rates.”
When someone with, say, kidney disease, falls victim to Covid and dies, how should be reported? Is Covid the actual cause of death?
A related problem is created by Covid’s ability to express in forms similar to other illnesses.
How serious is the impact of this ambiguity on the data? The Economist’s researchers were able to access an impressive database describing comorbidity patterns in the U.S.
Covid boosted the mortality rates for all groups defined by the various comorbidity factors, as shown here for a sample of elderly women:
The death rate for all of these medical conditions increased with Covid. For people with no pre-existing conditions in this age/gender group, the death rate for those who are infected with Covid is about 4.5 per 1000 people. For those with hypertension, Covid raises the death rate to about 6 per 1000. For people with Type 1 diabetes or heart disease, Covid almost doubles the death rate, to more than 8 per 1000.
If a person with hypertension or diabetes becomes infected with Covid, and dies, what should be recorded as the cause of death? Some with those conditions would have died in any case. How can we capture statistically the extra mortality contributed by Covid?
There is obviously significant ambiguity – which can be exploited to manipulate the Covid statistics. If the authorities wish to see a reduced Covid mortality rate, it is easy to attribute these deaths to the other causes. This is another significant source of information loss in the collation of Covid mortality statistics.
Are local Chinese authorities actually exploiting this ambiguity to meet the zero-covid mandates of Beijing? It would be hard to detect. It is certainly possible, and even “sadly plausible.”
The authors of the Science article also point to government interference in India in the data collection and reporting processes.
Is it possible that Chinese authorities may also have manipulated the data on Covid?
It is more than possible. Given (1) the history of similar cover-ups in China in the past, and (2) the absurdity of the official Covid death count (zero deaths since April 2020), one would have to say that political manipulation by Beijing is a virtual certainty.
The scientific magazine Nature. AFP PHOTO LOIC VENANCE (Photo credit should read LOIC VENANCE/AFP ... [+] via Getty Images)
Also this week, Nature magazine published a review of excess mortality methodologies and studies.
The public health establishment needs this metric, with all it flaws. The World Health Organization (WHO) says “we are likely facing a significant undercount of total deaths directly and indirectly attributed to COVID-19.”
The Nature article examines excess mortality calculations for past influenza outbreaks, as well as the models built to estimate the excess mortality associated with Covid, including models developed by the Institute for Health Metrics and Evaluation (IMHE) in Seattle, and by The Economist (both of which are updated daily). It also reviews “the most comprehensive of these excess-mortality estimates” developed by an economist at the Hebrew University of Jerusalem in Israel, and a data scientist at the University of Tübingen, Germany - the World Mortality Dataset (WMD), a leading database of all-cause mortality before and during the pandemic (2015–21) assembled from many sources covering 116 countries and territories.
One of the implications of the Nature article is this: All these models are hampered by the incompleteness of Chinese Covid data.
For example, the WMD (an unfortunate acronym?) is less than “most comprehensive” in at least one important respect: it “lacks excess-death estimates for China…They either do not collect them or do not publish them…” (The WMD authors themselves cite the disappointing response they received from Chinese authorities, i.e., “We are sorry to inform you that we do not have the data you requested.”)
(Another important database, the Human Mortality Database (HMD) developed and managed by the University of California at Berkeley and the Man Planck Institute in Germany, is apparently also forced to exclude China because of the lack of data.)
The IMHE model does not have updated information on China, and can cite only the official, ever-unchanging figure of 4636 Covid deaths in all of mainland China, frozen since April 2020.
The Economist’s team (also stuck with the spurious 4636 datapoint) used a Machine Learning model to try to interpolate the missing data. Their approach is sound and careful, but they acknowledge that the absence of reliable Chinese data is a serious handicap.
Optimism is good. Maybe some day more data will show up.
[Note: As described in my previous column, my own admittedly rough analysis of excess mortality based on anomalous changes in China’s Crude Death Rate arrived at an estimate of about 800,000 unexplained excess deaths in 2020 and 2021 – not far off from the mid-range of The Economist’s calculation.]
Much has been made of China’s refusal to cooperate with investigations into the origin of the virus. The refusal to provide information about ongoing Covid infections and deaths is just as problematic – even more so, because it hampers our understanding of the effectiveness of alternative countermeasures.
WASHINGTON, DC - SEPTEMBER 22: Chris Duncan, whose 75 year old mother Constance died from COVID on ... [+] her birthday, photographs a COVID Memorial Project installation of 20,000 American flags on the National Mall as the United States crosses the 200,000 lives lost in the COVID-19 pandemic September 22, 2020 in Washington, DC. The flags are displayed on the grounds of the Washington Monument facing the White House. (Photo by Win McNamee/Getty Images)
The very idea of excess mortality has been seen by some as controversial. It has the flavor of a “hidden variable.” It is often not directly observable. It is inferred from analysis of information that is available, such as crude death rates and official Covid mortality reports. The estimates for the missing data are imprecise, and may cover a wide range. Some have tried to cast doubt on these efforts, as though “excess mortality” were a mere statistical contrivance.
This is misguided. Divining “hidden variables” is what statistics is all about. Sometimes the missing information can be confirmed by additional direct observation. This is what the Science study has attempted, by examining survey data and government numbers to establish better Covid mortality numbers. In other cases (e.g. China), the hidden variable cannot be verified directly, but it can be quantified within a range of probabilities, sufficient for policy design and assessment. As WHO has said –
The UN is on board. They call excess mortality the “preferred measure” for assessing Covid impact.
So, too, the Journal of the American Medical Association:
And so, too, the World health Organization. WHO’s assistant Director General (a data analytics expert) said this of the Indian study:
(It will be interesting to see if WHO’s update addresses the obvious discrepancies in Chinese excess mortality figures.)
The inherent uncertainty of the estimation process – which statisticians measure with confidence factors and probabilities – is not a valid reason to reject this approach.
The whole world pays the price of bad Covid data.
Or – for instance – the nonsense theory that Beijing’s zero-covid policy has actually eliminated all Covid mortality from China since April 2020.
As the authors of the Science study point out