Measuring Soup: Why the US Military Doesn't Know How Effective Its Covid-19 Measures Are

or, when you're only wrong 24 times out of 25

It’s a story that was barely noticed, appearing only in the military-focused press: The Department of Defense knows close to nothing about its encounter with Covid-19, and so knows close to nothing about the value of any of the steps it has taken to control infection with or illness from SARS-CoV-2. There are no data to use for a reliable before and after.

On July 13, 2020, the Assistant Secretary of Defense for Health Affairs ordered the Defense Health Agency (DHA) to open a Covid-19 patient registry, capturing detailed information about all servicemembers infected during the pandemic. An office within the DHA, the Joint Trauma System, was tasked with gathering Covid-19 data and reporting that information to military leaders, consistent with that office’s longstanding responsibility “to improve trauma readiness and casualty care for Service members and maximize their survivability and recovery.” The DHA executed a series of contracts, ultimately worth about $7 million, for a data entry process that would be overseen by nurses trained in quality control for medical records. Remarkably, the performance standard defined by the data entry contracts requires that the DOD’s contractors “are to maintain an accuracy rate of at least 90 percent for the data entered into the COVID-19 Registry.” A ten percent failure rate is success.

But a July 7 report from the Office of Inspector General at the Department of Defense concludes that the contractor “did not enter accurate data into the COVID-19 Registry.” As a result of those data entry errors, “any data from the COVID-19 Registry that JTS officials provided to the DoD and other stakeholders during the COVID-19 pandemic are inaccurate and potentially misleading.”

Among other errors, the DHA’s patient registry simply missed thousands of servicemembers diagnosed with Covid-19. But records for servicemembers who made it into the registry were so inaccurate as to be useless:

To determine whether the patient health data in the COVID-19 Registry were accurate, we compared 39 data fields in 25 patient registry records to the source data in the patient’s electronic health record. Of the 25 records we reviewed, 24 records had errors, such as errors in the demographics, symptoms, and pre‑existing conditions data fields. For example, we identified that the data in ethnicity field were incorrect for 6 of 25 records, and the data in the symptoms field were incorrect or incomplete for 15 of 25 records.

And about those specially trained quality control nurses: “We reviewed 10 records that the quality compliance nurses validated as accurate in the JTSM, and we identified errors in all 10 records.”

Not having accurate information about servicemembers who were infected, not reliably knowing their symptoms or the severity of their illnesses, and having no accurate picture of the pre-existing illnesses or other medical conditions of military patients reported as being ill with Covid-19, military officials can’t possibly begin to measure the effect of mitigation measures like masking, quarantine, and compulsory vaccination. What would they be measuring against?

You can read the full report from the DOD Office of Inspector General at this link, or by clicking on the PDF file below:

Dodig 2023 093

11.1MB ∙ PDF file

Download

Download

Redaction produces some interesting holes in the picture the report produces, as with this description of the total number of servicemembers hospitalized with Covid-19:

Figure 2, you’ll be shocked to hear, is also a block of black ink.

They can’t measure how they were doing before compulsory vaccination, so they can’t measure the subsequent effects of compulsory vaccination. That helps to explain why the DOD’s Covid-19 vaccination mandate was unsustainable. They were standing on a foundation of garbage data, and arguing that an intervention in the middle of all the unmeasurable garbage was meaningful.