After serving on the editorial board for various OM journals over the last 24 years, I have the privilege to review over 1,000 referee reports. Overall, I have an inkling feeling that the perceived notion of “constructive comments” has changed over the years.
In my opinion, constructive comments are meant to help the authors to improve the quality of their papers regardless whether a paper should be published. However, based on the referee reports I have observed over the last 15 years or so, I find reviewers often use the following criticisms (often without providing specifics) to reject an article.
1. The assumptions of the model are not realistic. In many instances, this criticism is valid. However, we should think about George Box’s famous quote “All models are wrong, but some are useful.” To a great extent, we should remind ourselves that all models are “approximations” of the real world. While some assumptions may appear to be strong, they may enable the authors to develop useful results. For example, one can argue that it is unrealistic to use Normal demand because it can be negative. Also, one can argue Poisson arrivals and Exponential processing times are unrealistic. However, these assumptions have advanced our understanding of many OM issues over the years. Therefore, as reviewers, we should give the authors a chance to justify/clarify these assumptions or evaluate the robustness of their results'
2. The results are expected: nothing surprising. As explained in one of my blogs, I do not think surprising results are “required” for M&SOM publication, see: https://www.informs.org/Blogs/M-SOM-Blogs/From-M-SOM-Journal-Editor/Unexpected-Results-are-Required-or-Not). Although surprising results are not required, we should be mindful that all top journals are looking for papers that contain “innovative, relevant and important” ideas, rigorous analysis, and "meaningful” results! (For a general discussion about interesting OM, please see Gérard Cachon’s OM Forum article: http://pubsonline.informs.org/doi/pdf/10.1287/msom.1110.0375).
3. The paper offers no or limited managerial insights. To a certain extent, meaningful insight is a subjective measure. I have written a blog (https://www.informs.org/Blogs/M-SOM-Blogs/From-M-SOM-Journal-Editor/What-is-Insight) that is intended to create an open dialogue to define what insight is. Although beauty may be in the eye of the beholder, authors should state clearly about what readers can learn from their articles, and reviewers should think whether these lessons are meaningful to readers.
At the end of the day, I think journal publication is a “co-production process”: authors should try their best to justify/clarify certain modeling assumptions, and explain the implications and managerial insights of their results. At the same time, reviewers should provide specifics that can help the authors to improve the quality of their articles regardless of the outcome.
To do so, I have the privilege to work with our AEs to develop more specific review guidelines and a template to facilitate the “co-production process” as follows:
New Review Process (beginning December 1, 2015)
The M&SOM editorial board believes that a referee report serves two purposes: (a) evaluation for the benefit of the editors; and (b) feedback for the benefit of the authors. A good report helps the editors to: (i) identify papers with high potential; and (ii) provide the authors with a roadmap to bring out that potential. This roadmap might be very short in the case of a high-quality submission/revision.
Regardless whether a submitted paper is suitable for publication in M&SOM, a good report provides constructive comments on the paper’s strengths and areas of potential improvement. There are no perfect papers, and perfection is not the bar for publication. Our goal is to publish papers that ask and rigorously answer interesting and relevant questions.
To standardize our review process and effectively serve authors and the review team, the M&SOM editorial board would like all reviewers to clearly answer the following questions. (To facilitate this process, reviewers are encouraged to use the attached form (embedded in the email generated by MC) or, if they prefer, to copy and paste the questions into their report.)
1. What is the paper about? [Please provide a brief summary of the paper.]
2. What are the paper’s strengths and weaknesses? [Please comment on strengths in relation to (a) Innovation(research question, modeling, methodology, etc.); (b) Relevance (research question, findings, etc.); and (c) Rigor(appropriate methodology, correctness of analysis, etc.).]
3. What, if any, are the major areas of potential improvement? [Please focus on key requirements/suggestions that, if appropriately addressed, would make the paper publishable. If you see insurmountable obstacles, please clearly describe your concerns. You can best serve the editors and the authors by being specific and by providing constructive comments accompanied, where possible, by actionable suggestions. A laundry list of all possible improvements is not helpful. Likewise, ambiguous and/or vague criticisms should be avoided.]
4. What, if any, are the minor areas of potential improvement? [Again, please be specific.]
5. What is your recommendation - Accept, Minor Revision, Major Revision, or Reject? [Note: The reviewer’s opinion is advisory only.]
This new process will begin on December 1, 2015. By doing so, M&SOM can continue to serve as a platform to facilitate “mutual learning” for the entire community.
Thank you for your continued support of M&SOM!
Sincerely yours,
Chris Tang.