INFORMS Open Forum

"Review Less, Progress More" Revisited: 2-2-2 + 3-3-4

Christopher Tang

When I began my term in January 2015, I collected suggestions from all of you and developed a plan as stated in https://pubsonline.informs.org/doi/abs/10.1287/msom.2014.0510.  Besides those three criteria ("novelty", "relevance" and "rigor"), I have developed the "2-2-2" principle:

1. An original submission should be reviewed by 2 reviewers;
2. A final decision should be reached within 2 review cycles (in general); and
3. A rejected paper may be invited to be resubmitted as a new (and final) submission.

Based on your direct/indirect comments, the 2-2-2 review process has worked quite well. 

At the same time, I have learned from you regarding the concern over the consistency of reviewer reports.  To address this issue, I worked with the entire editorial board to develop specific review guidelines for the reviewers to consider as well as a form for the reviewer to use.  Since 2017, this reviewer form was attached in our email to the reviewer (instead of embedding the form in our Scholar One system).  I learned from you that these guidelines and the reviewer form were useful for the reviewers to provide more informative and constructive suggestions.

Over the last two years, I have learned from you that all reviewers should use the reviewer form.  At the same time, you would like the reviewers and the associate editors to provide fewer major suggestions so that authors can receive clearer guidelines when they are invited to revise their papers.  To address your request, I have worked with the entire editorial board to develop an additional "3-3-4 principle" for reviewers and AEs to consider.

Beginning from January 15, 2020, new guidelines for reviewers (and for AEs) will be sent to them via email when a paper is assigned to them.  Also, a simplified reviewer form (and a separate AE form) will be provided in our Scholar One system.  (The guidelines and the new form for the reviewer are provided below.  To avoid repetition, the guidelines and the new form for the AE are omitted.)  Specifically, in additional to minor comments, each reviewer will be invited to provide up to 3 major suggestions.   As the corresponding AE receives 3 major suggestions from one reviewer and 3 major suggestions from the other reviewer, he/she can synthesize these major suggestions along with his/her review to create 4 major suggestions for the authors to consider (if they were invited to revise their papers).  This way, we can improve the consistency of reviewer reports, and it can help the AE to develop a clear path for the authors to consider.   Ultimately, this additional "3-3-4" principle is developed in the spirit of "review less, progress more" as articulated in one of my blogs:  https://www.informs.org/Blogs/M-SOM-Blogs/From-M-SOM-Journal-Editor/Less-is-More-Fewer-Major-Improvement-Suggestions-Can-Enable-Us-to-Review-Even-Less-and-Progress-Even-More

Thank you for your feedback: we hear you, and we work with you.  Together, our community will continue to thrive.

Sincerely yours,

Christopher Tang.

 =====================

New guidelines for reviewers (provided via email from January 15, 2020 onwards.)

The M&SOM editorial board believes that a referee report serves two purposes: (i) to help the editors to identify papers with high potential; and (ii) to provide the authors with a roadmap to bring out that potential.  This roadmap should be precise and concise, and is should provide constructive comments on the paper's strengths and areas of potential improvement.  There are no perfect papers, and perfection is not a requirement for publication. 

Our goal is to publish papers that ask and rigorously answer novel and relevant questions.   Therefore, in general, a paper should ONLY be rejected due to: (1) a lack of relevance, i.e., the authors cannot reasonably state/articulate the question / topic about its relevance to operations management and originality; (2) a lack of novelty, i.e., the result / analysis is too incremental relative to the extant literature; or (3) a lack of rigor i.e., the analysis is erroneous or incomplete analysis which, if corrected, would dramatically change the conclusions.  The latter dimension can be judged relative to the relevance and novelty of the paper.  If you recommend rejecting a paper, we need you to state your reasons with explicit explanations / concrete examples / references to previous studies.  We also would need your help to explain your reasons why a potential revision is either impossible or will not be satisfactory. 

To standardize our review process and effectively serve authors and the review team, the M&SOM editorial board would like all reviewers to clearly answer the following questions.  The form will be provided by Scholar One.

1. What is the paper about? (Please provide a brief summary of the paper.)
   
   
2. What are the paper's strengths and weaknesses? Please comment on the strengths in relation to: (a) Innovation---research question, modeling, methodology, etc.; (b) Relevance---research question, findings, etc.; and (c) Rigor---appropriate methodology, correctness of analysis, etc.

4. What, if any, are the minor areas of potential improvement? (Again, please be specific.)
5. What is your recommendation - Accept, Minor Revision, Major Revision, or Reject? (Note: The reviewer's opinion is advisory only.)

"Review Less, Progress More" Revisited: 2-2-2 + 3-3-4

Christopher Tang

When I began my term in January 2015, I collected suggestions from all of you and developed a plan as stated in https://pubsonline.informs.org/doi/abs/10.1287/msom.2014.0510.  Besides those three criteria ("novelty", "relevance" and "rigor"), I have developed the "2-2-2" principle:

1. An original submission should be reviewed by 2 reviewers;
2. A final decision should be reached within 2 review cycles (in general); and
3. A rejected paper may be invited to be resubmitted as a new (and final) submission.

Based on your direct/indirect comments, the 2-2-2 review process has worked quite well. 

At the same time, I have learned from you regarding the concern over the consistency of reviewer reports.  To address this issue, I worked with the entire editorial board to develop specific review guidelines for the reviewers to consider as well as a form for the reviewer to use.  Since 2017, this reviewer form was attached in our email to the reviewer (instead of embedding the form in our Scholar One system).  I learned from you that these guidelines and the reviewer form were useful for the reviewers to provide more informative and constructive suggestions.

Over the last two years, I have learned from you that all reviewers should use the reviewer form.  At the same time, you would like the reviewers and the associate editors to provide fewer major suggestions so that authors can receive clearer guidelines when they are invited to revise their papers.  To address your request, I have worked with the entire editorial board to develop an additional "3-3-4 principle" for reviewers and AEs to consider.

Beginning from January 15, 2020, new guidelines for reviewers (and for AEs) will be sent to them via email when a paper is assigned to them.  Also, a simplified reviewer form (and a separate AE form) will be provided in our Scholar One system.  (The guidelines and the new form for the reviewer are provided below.  To avoid repetition, the guidelines and the new form for the AE are omitted.)  Specifically, in additional to minor comments, each reviewer will be invited to provide up to 3 major suggestions.   As the corresponding AE receives 3 major suggestions from one reviewer and 3 major suggestions from the other reviewer, he/she can synthesize these major suggestions along with his/her review to create 4 major suggestions for the authors to consider (if they were invited to revise their papers).  This way, we can improve the consistency of reviewer reports, and it can help the AE to develop a clear path for the authors to consider.   Ultimately, this additional "3-3-4" principle is developed in the spirit of "review less, progress more" as articulated in one of my blogs:  https://www.informs.org/Blogs/M-SOM-Blogs/From-M-SOM-Journal-Editor/Less-is-More-Fewer-Major-Improvement-Suggestions-Can-Enable-Us-to-Review-Even-Less-and-Progress-Even-More

Thank you for your feedback: we hear you, and we work with you.  Together, our community will continue to thrive.

Sincerely yours,

Christopher Tang.

 =====================

New guidelines for reviewers (provided via email from January 15, 2020 onwards.)

The M&SOM editorial board believes that a referee report serves two purposes: (i) to help the editors to identify papers with high potential; and (ii) to provide the authors with a roadmap to bring out that potential.  This roadmap should be precise and concise, and is should provide constructive comments on the paper's strengths and areas of potential improvement.  There are no perfect papers, and perfection is not a requirement for publication. 

Our goal is to publish papers that ask and rigorously answer novel and relevant questions.   Therefore, in general, a paper should ONLY be rejected due to: (1) a lack of relevance, i.e., the authors cannot reasonably state/articulate the question / topic about its relevance to operations management and originality; (2) a lack of novelty, i.e., the result / analysis is too incremental relative to the extant literature; or (3) a lack of rigor i.e., the analysis is erroneous or incomplete analysis which, if corrected, would dramatically change the conclusions.  The latter dimension can be judged relative to the relevance and novelty of the paper.  If you recommend rejecting a paper, we need you to state your reasons with explicit explanations / concrete examples / references to previous studies.  We also would need your help to explain your reasons why a potential revision is either impossible or will not be satisfactory. 

To standardize our review process and effectively serve authors and the review team, the M&SOM editorial board would like all reviewers to clearly answer the following questions.  The form will be provided by Scholar One.

1. What is the paper about? (Please provide a brief summary of the paper.)
   
   
2. What are the paper's strengths and weaknesses? Please comment on the strengths in relation to: (a) Innovation---research question, modeling, methodology, etc.; (b) Relevance---research question, findings, etc.; and (c) Rigor---appropriate methodology, correctness of analysis, etc.

4. What, if any, are the minor areas of potential improvement? (Again, please be specific.)
5. What is your recommendation - Accept, Minor Revision, Major Revision, or Reject? (Note: The reviewer's opinion is advisory only.)