INFORMS Open Forum

  • 1.  M&SOM new "Review Less, Progress More" process: 2-2-2 + 3-3-4

    Posted 12-09-2019 17:53

    "Review Less, Progress More" Revisited: 2-2-2 + 3-3-4

    Christopher Tang

    When I began my term in January 2015, I collected suggestions from all of you and developed a plan as stated in https://pubsonline.informs.org/doi/abs/10.1287/msom.2014.0510.  Besides those three criteria ("novelty", "relevance" and "rigor"), I have developed the "2-2-2" principle:

    1. An original submission should be reviewed by 2 reviewers;
    2. A final decision should be reached within 2 review cycles (in general); and
    3. A rejected paper may be invited to be resubmitted as a new (and final) submission.

    Based on your direct/indirect comments, the 2-2-2 review process has worked quite well. 

    At the same time, I have learned from you regarding the concern over the consistency of reviewer reports.  To address this issue, I worked with the entire editorial board to develop specific review guidelines for the reviewers to consider as well as a form for the reviewer to use.  Since 2017, this reviewer form was attached in our email to the reviewer (instead of embedding the form in our Scholar One system).  I learned from you that these guidelines and the reviewer form were useful for the reviewers to provide more informative and constructive suggestions.

    Over the last two years, I have learned from you that all reviewers should use the reviewer form.  At the same time, you would like the reviewers and the associate editors to provide fewer major suggestions so that authors can receive clearer guidelines when they are invited to revise their papers.  To address your request, I have worked with the entire editorial board to develop an additional "3-3-4 principle" for reviewers and AEs to consider.

    Beginning from January 15, 2020, new guidelines for reviewers (and for AEs) will be sent to them via email when a paper is assigned to them.  Also, a simplified reviewer form (and a separate AE form) will be provided in our Scholar One system.  (The guidelines and the new form for the reviewer are provided below.  To avoid repetition, the guidelines and the new form for the AE are omitted.)  Specifically, in additional to minor comments, each reviewer will be invited to provide up to 3 major suggestions.   As the corresponding AE receives 3 major suggestions from one reviewer and 3 major suggestions from the other reviewer, he/she can synthesize these major suggestions along with his/her review to create 4 major suggestions for the authors to consider (if they were invited to revise their papers).  This way, we can improve the consistency of reviewer reports, and it can help the AE to develop a clear path for the authors to consider.   Ultimately, this additional "3-3-4" principle is developed in the spirit of "review less, progress more" as articulated in one of my blogs:  https://www.informs.org/Blogs/M-SOM-Blogs/From-M-SOM-Journal-Editor/Less-is-More-Fewer-Major-Improvement-Suggestions-Can-Enable-Us-to-Review-Even-Less-and-Progress-Even-More

    Thank you for your feedback: we hear you, and we work with you.  Together, our community will continue to thrive.

    Sincerely yours,

    Christopher Tang.

     =====================

    New guidelines for reviewers (provided via email from January 15, 2020 onwards.)

    The M&SOM editorial board believes that a referee report serves two purposes: (i) to help the editors to identify papers with high potential; and (ii) to provide the authors with a roadmap to bring out that potential.  This roadmap should be precise and concise, and is should provide constructive comments on the paper's strengths and areas of potential improvement.  There are no perfect papers, and perfection is not a requirement for publication. 

    Our goal is to publish papers that ask and rigorously answer novel and relevant questions.   Therefore, in general, a paper should ONLY be rejected due to: (1) a lack of relevance, i.e., the authors cannot reasonably state/articulate the question / topic about its relevance to operations management and originality; (2) a lack of novelty, i.e., the result / analysis is too incremental relative to the extant literature; or (3) a lack of rigor i.e., the analysis is erroneous or incomplete analysis which, if corrected, would dramatically change the conclusions.  The latter dimension can be judged relative to the relevance and novelty of the paper.  If you recommend rejecting a paper, we need you to state your reasons with explicit explanations / concrete examples / references to previous studies.  We also would need your help to explain your reasons why a potential revision is either impossible or will not be satisfactory. 

    To standardize our review process and effectively serve authors and the review team, the M&SOM editorial board would like all reviewers to clearly answer the following questions.  The form will be provided by Scholar One.

    1. What is the paper about? (Please provide a brief summary of the paper.)

    2. What are the paper's strengths and weaknesses? Please comment on the strengths in relation to: (a) Innovation---research question, modeling, methodology, etc.; (b) Relevance---research question, findings, etc.; and (c) Rigor---appropriate methodology, correctness of analysis, etc.
    3a. If the paper is a "revision" of a previously submitted paper, please focus on how the author(s) addressed those major suggestions provided by the AE earlier (which synthesized all reviewers' suggestions).   Specifically, did the author(s) address AE's major suggestions satisfactorily?   (In the spirit of "review less, progress more," you may provide a couple of "new" suggestions if you can state your strong reasons for doing so and please keep in mind that no paper is perfect.)
    3b. If the paper is a "new submission", then please consider the following question.  What, if any, are the major areas of potential improvement? The editors would like to set as a target for all reviewer reports to have up to three specific suggestions (with explanation) for papers that receive minor/major revision.  Please state your suggestions in the order of importance. For each suggestion, please provide clear description about what would represent a satisfactory revision (that is doable within 6 months in your opinion).   (As a courtesy to the author(s), please prepare your suggestions in the form of a roadmap, which if followed diligently, should result in a publishable paper.)  Also, in the spirit of "review less, progress more", fewer than three suggestions are desirable.  More than three are permitted, within reason.  For a paper that you would like to recommend rejection, please state your reasons as stated above clearly along with explicit explanations / concrete examples / references to previous studies.
     
    1. What, if any, are the minor areas of potential improvement? (Again, please be specific.)
    2. What is your recommendation - Accept, Minor Revision, Major Revision, or Reject? (Note: The reviewer's opinion is advisory only.)

     



    ------------------------------
    Christopher Tang
    Professor
    University of California-Los Angeles
    Los Angeles CA
    ------------------------------


  • 2.  RE: M&SOM new "Review Less, Progress More" process: 2-2-2 + 3-3-4

    Posted 12-10-2019 04:43
    Dear colleagues!

    It all seems very reasonable.

    I recently several times ran into more than strange rejections on my articles from the chief editors under surprising pretexts like "not that style" and ... no more arguments (although the requirements for the style of presentation were not formulated in principle and I did not have any exotic constractions. I did not write articles in the form of ballads or prophecies!), "this topic is prohibited for our journal" - it is not non-core, but prohibited in the style of "taboo" (although the article was devoted to the most important methodological problems in line with the main theme of the journal). I distinctly remember that I did not write about perpetual motion, transmutation of elements and biological contacts with aliens!

    Unfortunately, this practice tends to develop. Some journals are starting publicly boast about increasing the percentage of rejected articles. I believe that this indicator is absolutely not representative. It turns out that the journal that rejected all the articles - the best?!

    Many reviewers fill in the non-documented plate according to the scheme "like - dislike", "good - not good". Such a plate can be charged to fill not only people - its evidence will have the same force.

    The number of recommendations I would not limit, but would distribute them into substantive and technical.

    When I wrote and write a review of the article, the book or the dissertation project, it turns at least 5 pages. In distant times, when I was present in the some editorial boards first of all in the USSR and in Russia too, reviews with ingenious brevity such as "I recommend to publish" or "I do not recommend to publish" were simply thrown into the trash basket, and their authors were tacitly removed from further reviewing. The review is not a sentence, but an incentive to development.

    All the best with the upcoming new year holidays for all!

    ------------------------------
    Oleg Dmitriev
    Prof. of Dep. of Management and Marketing in h-t industries
    Moscow Aviation Institute (National Research University)
    Moscow
    ------------------------------



  • 3.  RE: M&SOM new "Review Less, Progress More" process: 2-2-2 + 3-3-4

    Posted 12-11-2019 05:03
    I have matured else one quite revolutionary and important consideration.

    In my opinion, anonymity of reviewers does more harm than good.

    I have written a lot of reviews and expert opinions in my life (the idea of counting their number at least approximately seems to me insane and at the same time meaningless). At the same time, I always indicated my position and full name and, of course, a little flourish signed. Many of them dealt with very important issues and affected the fundamental interests of extremely influential people and groups.

    Of course, it was dangerous in every sense.

    But I have always believed that peer review is quite naturally associated with specific professional threats such as small or large revenge. I have an extensive and not very pleasant experience of communicating with people who sincerely craved my "blood", whose interests I infringed. Secretly and openly ... Long and short ... Strong and weak ... Alone and in a group ...

    But...

    Absolute anonymity breeds absolute irresponsibility. If the reviewer is confident that his identity will remain unknown to the scientific community, then he/she may well be incompetent, unscrupulous or malicious (objectively and / or subjectively). And the materialization of these negative qualities of his will never become public, will not generate for him a disciplining reputational or other damages.

    Moreover, the widely spread, in my opinion, absolutely vicious "tabular" form of the review with questions similar to primitive sociological questioning, such as "the experience of predecessors is taken into account enough - the experience of predecessors is not taken into account enough" is emotional-subjective. It is a form of empirical questioning of unscientifically selected experts, semi-experts or non-experts: "Well, I thought so."

    Therefore, I am a supporter of public reviewing, including scientific articles.

    Adequate openness for all from all !


    ------------------------------
    Oleg Dmitriev
    Prof. of Dep. of Management and Marketing in h-t industries
    Moscow Aviation Institute (National Research University)
    Moscow
    ------------------------------