Journal of Pediatric Psychology Advance Access published online on January 12, 2009
Journal of Pediatric Psychology, doi:10.1093/jpepsy/jsn142
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Editorial: How to Write Effective Reviews for the Journal of Pediatric Psychology
Cincinnati Children's Hospital Medical Center
All correspondence concerning this article should be addressed to Dennis Drotar, PhD, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, MLC 7039, 3333 Burnet Avenue, Cincinnati, OH 45229-3039, USA. E-mail: dennis.drotar{at}cchmc.org
The quality of editorial reviews is critical to the success of any scholarly journal, and the Journal of Pediatric Psychology (JPP) is no exception. Reviewers generously volunteer their time and intellectual energy to provide critiques of manuscripts that serve the critical function of enhancing the quality of science that is published in JPP (Drotar, 2008a
). Under tight timelines and in the context of multiple responsibilities, reviewers are asked to assume the difficult task of reviewing manuscripts that are often complex and detailed. Based on my experience over the past 2 years as editor of JPP, previous experience as managing editor, and feedback from JPP's associate editors, here are some suggestions to enhance the utility and quality of your reviews. Reviewers should also consult the JPP website (www.jpepsy.oxfordjournals.org) for specific guidelines concerning reviews as well as other relevant published sources (Drotar, 2000; Goldbeck-Wood, 1998; Hyman, 1995).
 |
Overview of The Manuscript Review Process |
|---|
Top
Overview of The Manuscript...
Importance of the Timeliness...The managing editor of a manuscript relies heavily on reviews in order to make an editorial decision and give clear feedback to authors concerning the rationale for this decision and/or suggestions to improve the manuscript through revision (in the event a revision is requested). Generally, at least three reviews are invited for each manuscript. Managing editors identify potential reviewers based on the specific key words that are listed in the JPP's Manuscript Central that correspond to the content of the manuscript as well as their experience in working with individual reviewers. Given the extraordinary breadth in the content of manuscripts that are submitted to JPP, it is not uncommon for members of the editorial board and reviewer panel to be asked to review manuscripts that contain content that is outside of their specific area of expertise. Reviewers’ response to such requests is greatly appreciated.
The managing editor reviews each manuscript independently but he or she does not make a final decision and write the decision letter to the author until all of the reviews are in. This means that the prompt disposition of a manuscript depends upon reviewers’ responsiveness at all phases of the review process.
|
Importance of the Timeliness of Reviews |
|---|
Given the importance of peer-reviewed publications to authors’ career development, the timeliness of reviewers’ feedback is very much valued and appreciated by authors as well as managing editors. Reviewer timeliness is critical for the following tasks: (1) a prompt (within a day or two) acceptance or rejection of an assignment to review; and, (2) once having accepted a review assignment, providing a review within the specified time limit (21 days). Thanks to the responsiveness of our reviewers and managing editors, the mean turn around time for an editorial decision and feedback to the author for initial manuscript submissions to JPP over the past year (2008) has been 32 days. This is an enviable track record for the first full year of the current team of managing editors, editorial board, and reviewer panel that we hope to sustain.
On the other hand, significant variation, including less than optimal individual reviewers’ response times over the past year, has also been noted: About 12% of reviews were more than 10 days late, which delayed editorial dispositions in each case. So there is clearly room for improvement. I recognize that there will be occasions when a review assignment comes at a very difficult time and cannot be accepted. However, if you need to decline a review (or decide to accept a review), we ask that you let us know as soon as possible so we can invite other reviewers. On average, managing editors of JPP invite at least four reviewers in order to obtain three reviews. Moreover, it is not uncommon to ask five to six reviewers in order to obtain three reviews. Delays in acceptance or decline of a review prolong the editorial process and create uncertainty among managing editors about whether to invite another reviewer.
On occasion, multiple competing responsibilities can also limit the timeliness of well-intentioned reviewers who have accepted an invitation to contribute a review. For this reason, if you see that you are going to be late with a review, or cannot complete it after having accepted the assignment, please let the managing editor know as soon as possible. Such prompt notification is very much appreciated as it facilitates the efficient management of reviews and lessens the managing editor's stress.
|
How to Enhance the Quality of the Content of Reviews |
|---|
What are the critical elements of the content of an effective review? The most important characteristics of an effective review are clarity, specificity, constructiveness, and thoroughness (Goldbeck-Wood, 1998
; Hyman, 1995). A review needs to inform the managing editor and author of the major strengths and weaknesses of a manuscript. In the event that a revised manuscript is requested, which is often the case, a review should provide clear, detailed suggestions for specific changes to improve the clarity of writing and the quality of the scientific contribution to the field of pediatric psychology. This is by no means an easy task: reviewers need to identify the most salient points of critique and communicate them clearly and in detail, sometimes by including examples and references to specific text so that authors can best understand and benefit from editorial feedback.
I strongly encourage reviewers to be honest in their appraisal of a manuscript and not pull their editorial punches in providing critique to authors. At the same time, the tone and content of all communications to authors should be both constructive and tactful, very much as one would converse with a respected colleague about ways to improve his/her research (Drotar, 2000). It is always difficult to hear let alone accept critique of one's work, especially from one's peers. For this reason, the constructiveness of how a critique is delivered will facilitate authors’ abilities to understand the information to improve the quality of their manuscripts. The constructiveness of a review is operationalized by concrete suggestions to improve the quality of the writing, the science, and significance of the work.
An effective review also needs to, carefully attend to the details of the author's presentation and provide a balanced consideration of strengths and problems contained in the manuscript. The identification of strengths should speak to the manuscript's scientific and/or clinical contributions. The identification of problems should identify both major and minor problems.
Key Content Areas of a Review
Several content areas of a review are particularly helpful in facilitating managing editors’ decisions and in providing the most effective feedback to authors. One of these is an explicit judgment about whether the research addresses a relevant and significant question in the field of pediatric psychology and has the potential of advancing the science in this field and/or having an impact on clinical care or public health (Drotar, 2008b). A second key content area is the identification of major methodological problems in design, measurement, statistical analysis that would limit the manuscripts’ scientific contribution. A third relevant content area relates to gaps, inconsistencies, and ambiguities in the presentation of information (e.g. literature review and framework, methods, analysis, interpretation and discussion of implications of findings).
|
How To Write A Review To Facilitate Feedback To Authors and An Editorial Decision |
|---|
Organize the Review
Very much like a manuscript, an effective review should be well organized. Here are several suggestions to help reviewers organize their reviews: First, in order to orient the author and managing editor to the most salient points of the critique, it is helpful to provide a brief overview about the major strengths and weaknesses of the manuscript. Second, I recommend that reviewers organize their critique by sections of the manuscript (e.g. abstract, introduction, methods, results, discussion). However, reviewers may prefer to state their key concerns about the manuscript prior to a sequential listing of key points in their review.
Effective reviews clearly distinguish between primary and secondary points of critique. Primary points are the key determinants of the decision to reject the manuscript or the critical issues that must be addressed in a revision in order to recommend publication (see previous section on the content of reviews). Secondary points are those that are less critical from the standpoint of "making or breaking" the scientific significance and validity of the research contained in the manuscript but are relevant to enhancing the clarity and quality of the manuscript and its utility to readers (e.g. points of omission in methods, quality of writing, etc.). Reviewers should keep in mind that minor matters of editorial style will be managed by the copy editor and need not be a focus of the review.
Represent the Content of Reviews Succinctly and Clearly
Authors face a formidable challenge in responding to each of the editorial points that are made by the reviewers and the managing editor. The combination of detailed reviews and a decision letter can yield more than six single-spaced pages of critique. For this reason, we ask reviewers to be as concise as possible in their reviews while being thorough (which is exactly what authors are asked to do). A comprehensive review can usually be conducted in two single-spaced pages.
Managing editors generally ask authors to respond in some way to each of the points of critique that are made by reviewers unless they are contradictory (in which case the managing editor will provide guidance to resolve contradictory points of critique). Consequently, numbering each specific points of critique facilitates authors’ organization of their response to the critique as well as the managing editor's task in determining how well authors have responded to the critiques.
Indicate Ratings and Recommendations for Editorial Decisions
In addition to providing detailed comments about the manuscript that are given to authors, reviewers rate the quality of the manuscript on a 1–5 scale in relevant content domains such as design and methods, innovativeness, analysis of data, etc. Finally, reviewers provide ratings of the manuscript's potential impact on the field, the quality of the manuscript compared with those typically published in JPP, and the overall quality of the manuscript. Managing editors appreciate the reviewers’ ratings and will use them to help guide their decision. Reviewers are also asked to communicate their recommendations to managing editors for a specific editorial decision, (e.g. accept, accept with revision, revise and resubmit, revise as a brief report, or reject). The "revise as a brief report" editorial decision is used for manuscripts that clearly advance science but are best presented in a briefer form. The standards for acceptance of brief reports are exactly the same as for full length manuscripts. Reviewers should consult recent issues of JPP for examples of brief reports.
Communicate Consistently with Authors and Managing Editors
Managing editors have the responsibility of integrating information from all the reviews (including their own) to render an editorial decision and generate suggestions to the author to enhance the quality of the manuscript's revision (in the event that a revision is recommended). In order to provide the most useful information to the managing editor, reviewers should communicate all of their most important concerns in their critique to the authors.
However, we do request that reviewers refrain from making any direct or implicit recommendation for acceptance or rejection of the manuscript in the narrative critique that is provided to authors. The reason for this is that reviewers may disagree concerning the disposition of a manuscript. Moreover, the managing editor's decision may sometimes reflect a minority opinion. Authors are understandably sensitive to such discrepancies in reviews and may be confused or frustrated by the differences in recommendations.
Reviewers also have the option of providing comments to the editor that are not seen by authors. However, we ask reviewers to give such comments to the editor sparingly. The reason for this is that authors should have the benefit of reviewers’ honest appraisals, which should always be consistent with the feedback given to the editor, given the potential confusion that is generated by discrepant editorial feedback as noted above. It is clearly not helpful to give any feedback to editors that is discrepant from what is communicated to authors.
|
How to Help Editors Manage Difficult Editorial Decisions |
|---|
Experienced authors and editors know that it is not uncommon to encounter minor and sometimes major disagreements among reviewers about the quality of a manuscript and the nature of its contribution to the field. Differences in training and experience, theoretical orientation, preference, and standards among reviewers can result in discrepant feedback in narrative critiques and recommendations for a manuscript's disposition (Fiske & Fogg, 1990
). Even when reviewers identify similar concerns about a manuscript, they may weigh the importance of the concerns very differently (Fiske & Fogg, 1990). The final editorial decision concerning the disposition of a manuscript reflects the managing editor's overall judgment about the quality of the manuscript and is not necessarily a consensus editorial statement among the reviewers. As noted above, an editor's opinion will sometimes reflect a minority opinion among reviewers.
In my experience, the most difficult editorial decision is whether to recommend a revision versus an outright rejection. It is tempting to bend over backwards to give authors every benefit of the doubt and recommend a revision if there is any chance that the manuscript can be accepted. Such empathy for authors is well intentioned. However, my experience in working with resubmissions, some of which were eventually rejected, have underscored the inefficiency and frustration raised by revise and resubmit editorial verdicts that turned out to be too lenient. For this reason, I would ask reviewers to consider not whether a manuscript could be acceptable someday with the best of possible revisions, but whether such as a revision will be likely to result in an important new contribution to research in pediatric psychology that transcends current research in method, results, and/or clinical significance (Drotar, 2008b). Authors should not be invited to revise manuscripts that will not contribute significantly to the science in the field of pediatric psychology. In making this judgment, it is very important that reviewers identify major problems with method, design, or significance that are not potentially correctable or feasible to correct and distinguish them from correctable problems.
|
How to Conduct an Ethical Review |
|---|
Several important ethical issues need to be considered by reviewers and managing editors (Drotar, 2000
). Manuscripts are to be treated as privileged and confidential communications that are seen by reviewers and editors but are not to be circulated to or seen by anyone else, with the exception of mentees who participate in a mentored review. Conflicts of interest between reviewers and authors can also raise important ethical issues: Given the relatively small group of researchers in the fields of pediatric and clinical child psychology, reviewers may be asked to review the work of authors one knows in some capacity. Knowing an author or having worked with him or her in some way does not necessarily pose a conflict. Reviewers need to consider whether they are able to review a manuscript of a former student, professional associate, or academic rival as impartially as they would if they did not know them. If the answer is no, they should inform managing editors and asked to be excused from the review. Managing editors cannot readily identify such conflicts and need to rely on reviewers to do so (Routh, 1995). Consequently, reviewers have the primary ethical responsibility to recognize conflicts of interest and to excuse themselves from a review if they feel that bias could interfere with an impartial review. Moreover, authors should declare known conflicts with managing editors and reviewers who are close colleagues and who have a vested interest in the science contained in the manuscript. We encourage authors and editors to review the JPP's conflict of interest statement (see JPP website) to recognize potential conflict of interests such as close, active collaboration in the content area of the manuscript, faculty members in the same institution, and current mentors and mentees. When in doubt about whether they are in potential conflict, reviewers should feel free to consult with the managing editors.
|
How To Become a Reviewer for JPP and/or Obtain Mentored Experience in Reviews |
|---|
I hope that I have not discouraged current or potential reviewers for JPP with this litany of "how tos," because this was far from my intent. Given the increasing number of manuscript submissions in the increasingly diverse field of pediatric psychology (which is a clear sign of the scientific health of our field), the editorial leadership of JPP is very invested in recruiting the most talented, conscientious reviewers to join our ranks. So, if you are interested in becoming an ad hoc reviewer for JPP, let me know. We have a particular need for reviewers with expertise in statistical methods.
Moreover, students, fellows, or junior faculty who are interested in receiving mentoring in the process of reviews should consider joining JPP's mentoring program, which now includes more than 100 mentors and mentees. We also are very interested in identifying reviewers who would like to serve as mentors.
Finally, if you are a reviewer who has been reviewing for JPP, enjoy it, and want to take on increased responsibility for reviews and receive professional recognition for this work you might consider applying to join either the reviewer panel (if you are a junior, less-experienced reviewer) or the editorial board (if you are a more experienced reviewer). The current terms of the editorial board and reviewer panel are for 2 more years. If you want to be considered for either of these positions in the future, please let me know.
|
Final Note |
|---|
As noted in my acknowledgement of the extraordinary work of our editorial board (Drotar, 2008), the quality and consistency of editorial review are the essential life's blood of JPP's success to date as well as the future accomplishments of JPP. Speaking for myself and my associate editor colleagues, it is a privilege to work with and learn from you. We hope that the feeling is mutual. Moreover, if you have suggestions to improve the review process and our collaboration with you, please do not suffer in silence. Let us know. Improving the quality of the editorial process and published science in the field of pediatric psychology is a relentless mission.
|
Acknowledgments |
|---|
The excellent work of Meggie Bonner in typing this manuscript and the valuable feedback contributed by the associate editors of JPP are gratefully acknowledged.
Conflicts of interest: None declared.
Received December 11, 2008; revision received December 11, 2008; accepted December 11, 2008
|
References |
|---|
Drotar D. Reviewing and editing manuscripts for scientific journals. In: Handbook of research methods in clinical child and pediatric psychology—Drotar D, ed. (2000) New York: Kluwer Academic/Plenum Publishers. 409–425.
Drotar D. Acknowledging the extraordinary service and contributions of the editorial board for the journal. Journal of Pediatric Psychology (2008a) 33:109.
Drotar D. Thoughts on establishing research significance and preserving scientific integrity. Journal of Pediatric Psychology (2008b) 33:1–5.
Fiske DW, Fogg L. But the reviewers are making different criticisms of my paper: Diversity and uniqueness in reviewer comments. American Psychologist (1990) 40:591–598.
Goldbeck-Wood S. What makes a good reviewer of manuscripts. British Medical Journal (1998) 316:86.
Hyman R. How to critique a published article. Psychological Bulletin (1995) 118:178–182.[CrossRef][Web of Science]
Routh DK. Confessions of an editor, including mistakes I have made. Journal of Clinical Child Psychology (1995) 24:236–241.[CrossRef][Web of Science]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||