INFORMS Open Forum

  • 1.  Management Science: Data and Code Disclosure Policy

    Posted 11-23-2018 14:58
      |   view attached

    Dear Colleagues,

     

    In my last blog, I informed the community that the Management Science editorial board is considering a new Data and Code Disclosure policy. Here is some background.

     

    In 2013, Management Science established a data disclosure policy, which encourages, but does not require, the disclosure of data associated with published manuscripts. A few months ago, the board formed a committee whose focus was to strengthen Management Science's policy in this area. Attached is the committee's recommendation.

     

    The members of the committee are Yan Chen (DE for Decision Analysis), Kay Giesecke (Finance), Stephen Graves (Chair and former EiC MSOM), Hamid Nazerzadeh (Big Data Analytics), Sridhar Tayur (Entrepreneurship & Innovation), and Juanjuan Zhang (Marketing).

     

    Before finalizing the policy, the editorial board would like to solicit comments from members of our community. Please send comments to mseic@mit.edu

     

    David Simchi-Levi

    Editor-in-Chief

    Management Science

    E-mail: mseic@mit.edu



    ------------------------------
    David Simchi-Levi
    Professor of Engineering Systems
    Massachusetts Institute of Technology
    Cambridge MA
    ------------------------------


  • 2.  RE: Management Science: Data and Code Disclosure Policy

    Posted 12-11-2018 17:22

    Dear David –

    Per your invitation, I wanted to provide quick feedback on the proposed Data Disclosure policy for Management Science.  In sum, I think the policy, as currently stated, will severely limit the ability of researchers in Operations Management to publish important empirical work using secondary data and field experiments (I see no problems for lab and numerical experiments/simulations).  This Policy would potentially slow down development of our field, which is quite dangerous. Here are a few specific thoughts: 

    1. What are we trying to achieve? Empirical work using secondary data and field experiments in OM is rare as this survey demonstrates - we are far behind other fields.  By establishing additional hurdles we will reduce already very small number of empirical publications.  Is there evidence that we have a problem in OM with massive data fraud, involving secondary data or filed experiments? I am not aware of any. Do we have numerous researchers wishing to replicate studies of others?  I have not met any. I would suggest that we need fewer, not more hurdles at this point.  All other fields only relatively recently adopted policies for data disclosure: we are nowhere near their level of maturity and availability of data.  I would maybe offer incentives for disclosing data (e.g., faster publication process) but not create hurdles.
    2. Our field is fundamentally different from economics, finance and marketing in that we have not yet developed standardized archival datasets.  The de-facto standard in data dissemination is WRDS to which most institutions in the world subscribe.  It contains dozens of econ, marketing and finance databases but no operational databases, none.  This is why just about any empirical paper in OM that relies on archival data or field experiments is covered by NDAs.  Looking at my own publications, out of 15 or some empirical papers I published, only 1 or 2 rely on data that can be disclosed.  Certainly, we should require disclosing experimental data, simulation data and any code, but it seems unreasonable to me to go further than that at this point.
    3. The process by which exceptions will be managed according to the current Policy is unclear.  My work with a company starts with signing an NDA before I see any data.  3-5 years later, I might actually see data and write paper based on it.  It is difficult for me to affect wording of an NDA, and any company would immediately stop the conversation if I brought up data disclosure.  It seems unreasonable to me to spend 3-5 years procuring data and working on it just to find out later that the paper cannot be published because the exception cannot be granted.  Instead, there should be a specific set of rules in the Policy stating that, if an NDA is produced which prohibits data disclosure, then the data does not have to be disclosed.  Of course, the code should be disclosed in any case.
    I hope these suggestions are considered by the committee.
    Best,
    Serguei

    ------------------------------
    Serguei Netessine
    Professor
    The Wharton School
    Philadelphia PA
    ------------------------------