Value and Uncertainty in Pandemic Metrics
New York Governor Andrew Cuomo’s daily briefings have become a mainstay of support for many during the COVID-19 pandemic, especially with New York being the initial epicenter of the disease in the U.S. It is clear that Cuomo’s polished slide presentations have been developed by consultants with strong management science, computer modelling and contingency planning backgrounds. It is refreshing to see a politician base his understanding and decisions on fact-based data and proven analytical approaches.
I was particularly struck by Cuomo’s presentations of May 4 and 5, 2020, which addressed uncertainty and value respectively. I was so attracted because I had published an article in 2008 on value and uncertainty in security metrics.[i]
The gist of my article is that the most common and easily-obtained security data and metrics tend to be the least useful for decision-making, whereas those data and metrics that are most valuable for decision-making can be difficult, expensive and time-consuming to collect and analyze, yet they yield substantially better results in terms of responding appropriately to given situations.
In the article, I describe various categories of metrics, including: existence, ordinal, score, cardinal, percentage, holistic, value and uncertainty, and list the pros and cons of each.
When it comes to value, Governor Cuomo asks what the value of a life is … his answer is that each life is “priceless.” Yet there are some for whom the value of lives of other people (not their own lives) appears to be relatively low. Insurance companies do place dollar values on wrongful death, pain and suffering in order to settle claims. In reality, the value of a life is highly subjective and will vary from one person to another depending on their character, upbringing, background, life experience, mental health, nationality, social group, ethnicity, religion, etc. While the value of a life is not strictly measurable, decisions and actions are based on some underlying measure of the value of our own lives and those of others.
While the number of infections, hospitalizations, and deaths from COVID-19 clearly feeds into any analysis, the objective (apart from primary goal of saving lives) is to reduce the number of hospitalizations and patients needing intensive-care to a level that the health-care systems can accommodate. Governor Cuomo described this process as closing a valve to control the input, then releasing the valve slowly and monitoring the results. If cases continue to drop, the valve may be opened further; if not, the valve should be tightened.
On one hand, we have the pain, suffering and deaths from the disease itself, and, on the other hand are the economic and emotional costs of the lockdowns. The key is to find a balance between life and livelihood that is acceptable to the majority. But doing so is incredibly difficult since the benefits and costs are highly subjective and fall upon some groups much harder than on others. Yet, how do you make these life-and-death decisions without having some basis for analysis and decision-making?
The dynamics of the situation are also key. Medical workers are learning how to take care of patients more effectively and researchers are getting a better idea of the causes and physiological impact of the disease so as to come up with therapies and vaccines. But, unless we fully adopt the precautions of physical distancing, wearing protective equipment, and following sanitary practices, such as hand-washing and the santizing of surfaces, the pandemic will be prolonged and the health and economic consequences even more difficult to bear. Pushing hard on the medical front to mitigate the impact of the virus and researching ways to treat and prevent the disease, and providing people with resources to sustain themselves through this tortuous process seem to be the right things to do. But, implicit in these decisions are dynamic value judgements that should be understood, described and explained in order that the best decisions can be made.
Currently, pandemic statistics are fraught with uncertainty. The number of actual cases could be undercounted by one or two orders of magnitude since they depend upon the number of diagnostic tests and the incidence of false negatives.[ii] Also, there is concern that some serologic (antibody) tests on those who have contracted the virus yield false positives.[iii] Furthermore, the number of COVID-19 deaths is thought to be well above those reported, because some of those who died were not tested for the virus, especially if they died at home. There is also the possibility of intentional under-reporting of both cases and fatalities by leaders and governments wishing to downplay the impact of the virus in their countries. Comparing increases in year-to-year deaths to reported COVID-19 fatalities suggests that the numbers of the latter are greater than reported for some U.S. states by from 50 percent to as much as 600 percent.[iv]
So, how do we cut through all the uncertainty and deal with the difficulties in ascertaining values? With respect to value determination, it is understood to be extremely difficult to measure, highly subjective and often misleading. Values are also dynamic, varying as circumstances and people’s perceptions change. The best we can hope to do is to understand the underlying values that are shaping our personal and communal judgment, and confirm to ourselves that we are making the “best” decisions at any particular point in time based upon the information available.
With respect to uncertainty, we need to focus on actionable metrics. Which statistics are appropriate in order to monitor and control the spread of the virus? What does it take to convince people to stay on lockdown or open up certain segments of commerce and society? A significant concern is to avoid overwhelming health care systems. Hospital admissions are key here. Since hospitalizations depend upon the spread of the virus, it is important to know how prevalent are infections, which comes from extensive testing. However, the value of testing depends on the accuracy and timeliness of the tests and on the population selected for testing. Those being voluntarily tested are generally individuals who suspect that they have been exposed or are showing symptoms. This creates biased results. It is interesting that in Wuhan, China, which is generally considered to be where the virus originated and was thought to have been eliminated, every one of the 11 million inhabitants are to be tested after new cases have been discovered.[v] Such exhaustive testing would suggest that sampling of the general population or testing of selected (or self-selecting) individuals does not pass muster. Chinese authorities want to remove uncertainty from sampling, although the issues of false positives and negatives may well remain.
However we look at it, we are confronted with life-and-death decisions based on metrics that are very difficult to obtain accurately. However, it is well worth the effort to acquire such metrics than not to do so. Critical decisions should be based on significant, accurate and timely data and honest interpretations of those data, Such life-and-death decisions are only as good as the quality of data and metrics and it behooves us to make the effort to ensure that quality.
[i] C. Warren Axelrod, “Accounting for Value and Uncertainty in Security Metrics,” ISACA Journal, November 2008. Available at https://www.researchgate.net/publication/268059625_Accounting_for_Value_and_Uncertainty_in_Security_Metrics ,
[ii] Josh Salman and Dan Keemahill, “Coronavirus cases are likely artificially low in some states thanks to flawed testing,” USA Today Network, May 4, 2020. Available in online version at https://www.usatoday.com/story/news/investigations/2020/05/04/covid-19-numbers-along-state-lines-show-problems-reopening-plans/3050921001/
[iii] Sangmi Cha and Josh Smith, “Explainer: South Korean findings suggest ‘reinfected’ coronavirus cases are false positives,” US News and World Report, May 7, 2020. Available at https://www.usnews.com/news/world/articles/2020-05-07/explainer-south-korean-findings-suggest-reinfected-coronavirus-cases-are-false-positives
[iv] Emma Brown et al, “U.S. deaths soared in early week of pandemic, far exceeding number attributed to covid-19,” The Washington Post, April 27, 2020. Available at https://www.washingtonpost.com/investigations/2020/04/27/covid-19-death-toll-undercounted/?arc404=true
[v] James Griffiths, “Wuhan to test all residents for coronavirus in 10 days after new cases emerge,” CNN, May 12, 2920. Available at https://www.cnn.com/2020/05/12/asia/wuhan-coronavirus-testing-china-intl-hnk/index.html
*** This is a Security Bloggers Network syndicated blog from BlogInfoSec.com authored by C. Warren Axelrod. Read the original post at: https://www.bloginfosec.com/2020/05/18/value-and-uncertainty-in-pandemic-metrics/?utm_source=rss&utm_medium=rss&utm_campaign=value-and-uncertainty-in-pandemic-metrics
