INFORMS Open Forum

  • 1.  The lament of COVID-19 modelers

    Posted 06-02-2020 06:46
    Edited by Rahul Saxena 06-02-2020 06:48

    Three medical professional associations in India have submitted a joint statement to the Prime Minister, criticizing the government's handling of the novel coronavirus epidemic and making recommendations for the future.

    1. Indian Public Health Association
    2. Indian Association of Preventive and Social Medicine, and
    3. Indian Association of Epidemiologists

    You can read it at: A few snippets:

    • India is paying a heavy price ... incoherent and often rapidly shifting strategies and policies
    • Open and transparent data sharing ... conspicuous by its absence till date
    • draconian lockdown is presumably in response to a modeling exercise from an influential institution which was ... way off the mark.

    Here is a view of the situation in India:

    1. The players
      1. The testing community of biotech experts sequenced the genome of the virus, started building RT-PCR test kits and antibody tests. Moved amazingly fast (e.g., compared to SARS & MERS). Test availability and costs remain impediments, but we note that previous epidemics were handled without such tests. It also seems that the medical community didn't know how to deal with test accuracy and asymptomatic cases. Maybe because previously they were used to owning the responsibility of identifying cases (a doctor's decision for each case), and cases were sick patients.
      2. The modeling community of epidemiologists jumped into the game of messy data with old and new models. Models came out worldwide within days of the problem hitting the news, and model results drove the public perceptions of the threat. An influential model contained coding flaws, and many models didn't include the dose effect (low-dose infection with low virulence and fast recovery), didn't model successive mutated waves, & ignored economic devastation and all-cause deaths. There was no "gold standard" model.
      3. The media (incl. social media) amplified extreme and novel views, created a WYSIATI-trap / filter-bubble (only see the disease news) forcing public opinion to coalesce towards "do something huge, now!", along with avoidable oscillations and more filter-bubbles due to a lack of gold-standard analyses.
      4. The medical community had epidemic and pandemic management playbooks in place for years, but apparently got stampeded (and in some cases sidelined) by the onslaught from the testers, modelers, and media. Their oscillations looked unedifying.
      5. The policy makers ping-ponged around, and their oscillations, too, have been unedifying.
    2. The data
      1. Revealed broken data supply chains. There was no timely, reliable, and public data for tests, cases, deaths, etc. Except a few insiders, all are starved of fresh & reliable data. Most worked on older data from more-reliable suppliers and tried to meld data from different sources. Startup sites like Worldometer and OurWorldInData became important sources of pandemic data.
      2. Exposed confusion about data semantics. Tests need to be understood in the context of test strategy (e.g., who gets tested). Tests can be RT-PCR or serological, and different kits and procedures have different sensitivities. Cases can be asymptomatic, mild, require hospitalization, or the ICU. The "cause of death" depends on local regulations, making death data from different jurisdictions hard to compare.
      3. Showed that data models couldn't handle data updates that included re-statements of earlier data, e.g., delays and changes in semantics.
    3. The models
      1. Reliance on bad models with design flaws (overlooked key issues: dose effect, successive mutated waves, economic impact), coding flaws, etc. In an amazing oversight, the presence of slums and >100 million population of poor people in cities who had traveled far from their villages and faced starvation from the lockdown in the cities came as a "surprise" for the policy makers, making us think that a non-Indian model was being used.
      2. No institutional gate-keeping. Bad models couldn't be held to a "scientific standard" because the expected standard-bearers (WHO, etc.) were unable to discharge that role. Cities and states came up with their own versions of models and dashboards.
      3. Rushed into draconian lockdowns, taking actions with impacts that haven't been modeled (ref. Prof Krass). It'll be a huge effort to decipher how the lockdowns alleviated the spread of the pandemic and how they inflicted economic, social, and public health harm. Even a single-objective problem (a minimize mortality model) requires a lot of analysis to layer an epidemic onto all-cause mortality for any country. And needed also to provide the foundations for multi-player multi-objective models.

    Similar laments seem to be rising in other countries. A paper to provide guidance on building public health analytics infrastructure (reliable & timely data supply chains and models) could be quite useful. Does such a paper exist? Requesting pointers, please.

    Rahul Saxena

  • 2.  RE: The lament of COVID-19 modelers

    Posted 06-03-2020 09:56
    Edited by Matt Walls 06-03-2020 09:59
    Hello Rahul,

    RE: A paper to provide guidance on building public health analytics infrastructure - A quick search of INFORMS PubsOnline ( finds several promising results among the INFORMS Journal of Applied Analytics, our TutORials in Operations Research series, Information Systems Research, and more:

    Please don't hesitate to contact me directly ( for additional assistance with our online journal archive.

    Best regards,

    Matthew Walls
    Director of Publications
    Catonsville MD