How to Write an Effective Journal Article Review

11

Dennis Drotar, Yelena P. Wu, and Jennifer M. Rohan

The experience of reviewing manuscripts for scientific journals is an important one in professional development. Reviewing articles gives trainees familiarity with the peer review process in ways that facilitate their writing. For example, reviewing manuscripts can help students and early career psychologists understand what reviewers and editors look for in a peer-reviewed article and ways to critique and enhance a manuscript based on peer review. Experiences in review can facilitate early career faculty with early entry into and experience being a reviewer for a professional journal. The experience of journal reviews also gives students a broader connection to the field of science in areas of their primary professional interest. At the same time reviewing articles for scientific journals poses a number of difficult challenges (see Hyman, 1995; Drotar, 2000a, 2009a, 2009b, 2009c, 2009d, D. Drotar, PhD () Division of Behavioral Medicine and Clinical Psychology, Cincinnati Children’s Hospital Medical Center, MLC 7039, 3333 Burnet Avenue, Cincinnati, OH 45229-3039, USA e-mail: [email protected] Y.P. Wu, PhD Division of Behavioral Medicine and Clinical Psychology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati, Cincinnati, OH 45229-3039, USA J.M. Rohan, MA Division of Behavioral Medicine and Clinical Psychology, Department of Psychology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati, Cincinnati, OH 45229-3039, USA

2010, 2011; Lovejoy, Revenson, & France, 2011). The purpose of this chapter is to provide an introduction to the review process and give step by step guidance in conducting reviews for scientific journals. Interested readers might wish to read Lovejoy et al.’s (2011) primer for manuscript review, which contains annotated examples of reviews and an editor’s decision letter.

How to Get Experience and Mentoring in Reviewing Manuscripts for Scientific Journals The best way to get mentored experience in journal review is to identify a mentor who reviews regularly for a journal and has time and interest in providing such experience. Journal review mentorship has a number of advantages: Students and early career faculty can be very helpful in providing additional insight and critique concerning manuscript review. Moreover, mentors often enjoy the prospect of collaborating on a review and teaching students about the review process. Benefits of completing mentored manuscript reviews can also extend beyond the manuscript reviews (e.g., pursuing additional collaborations with mentors that could lead to publications, receiving career advice on professional development issues, widening one’s professional network, etc.; Wu, Nassau, & Drotar, 2011). We encourage students to take the initiative in letting their mentors know about their interest in reviewing and mentors to reach out to students to

M.J. Prinstein (ed.), The Portable Mentor: Expert Guide to a Successful Career in Psychology, Second Edition, DOI 10.1007/978-1-4614-3994-3_11, © Springer Science+Business Media New York 2013

163

164

help them obtain such experience. Although a graduate mentor is perhaps the most obvious choice, students may also seek mentored review experiences from other mentors or supervisors (e.g., within professional organizations, supervisors outside of a student’s graduate program), keeping in mind that mentored reviews can be conducted across geographic distances. In addition, if mentored experience is not available in specific graduate training programs, which is not uncommon, journals such as the Journal of Pediatric Psychology (JPP) have a formal program for students and junior faculty to obtain training in the review process from mentors in settings and programs around the country (see Wu et al., 2011 for more information). Although students and junior faculty must balance time spent completing manuscript reviews with their other responsibilities, it may be helpful to complete mentored reviews with more than one mentor in order to gain exposure to different perspectives on manuscript reviews as well as exposure to different review styles.

Overview of the Manuscript Review Process Role of the Editor Each manuscript that is submitted to a peer-reviewed journal is assigned to a managing editor who is responsible for the following tasks: (1) selection of reviewers, (2) independent review of manuscript content, (3) integration of both reviewers’ comments and feedback from his or her own independent review in order to prepare an editorial decision, and (4) preparation of detailed decision letter to the authors. The decision letter describes the editorial decision; documents the rationale for the decision; and in the case of revise and resubmit editorial verdicts, provides a detailed description of suggested changes, a rationale for why they are necessary and how the changes will enhance the quality of the writing and science of the manuscript. The managing editor of a manuscript relies heavily on reviewers in order to make an effective editorial decision and give clear feedback to authors concerning the rationale for the editorial decision and/or suggestions to improve the man-

D. Drotar et al.

uscript through revision (in the event a revision is requested). Generally, at least 2–3 reviewers are invited for each manuscript. Managing editors identify potential reviewers based on their knowledge of reviewers and key words noted in reviewers’ areas of expertise that correspond to the content of the manuscript.

Role of Reviewers in Manuscript Reviews The primary role of reviewers in the editorial process is to provide an honest, fair, clear, and concise critique concerning the quality of science that is contained in a manuscript submission and the extent to which the science is clearly expressed. Managing editors and authors benefit greatly from reviewers’ comments. Arguably, reviewers have a critical role to play in the manuscript process in helping to determine which manuscripts should be published vs. those that should not and in providing feedback to authors to enhance the quality of their published work. To facilitate this task, reviewers should familiarize themselves with the content areas of the journal they are reviewing for, and policies and procedures for reviewers. There are important individual differences in policies, rating forms, and deadlines from journal to journal that reviewers should understand. It should be noted that given the extraordinary breadth in the content of manuscripts that are submitted to journals, reviewers may be asked to review manuscripts that focus on content areas that are outside of their specific area of expertise, and reviewers should be prepared for such invitations. Reviewers should keep in mind that expertise in evaluating the quality of research design, measurement, and statistical analysis can be as valuable as specific content expertise.

Guidance for Mentored Reviews Obtain an Orientation to the Review Process Mentors who agree to work with students or junior faculty mentees to complete one or more mentored

11

How to Write an Effective Journal Article Review

reviews together may benefit from discussing several issues prior to reviewing a manuscript. Students might ask mentors to provide a brief overview of the manuscript review process from their perspective (e.g., the role of reviewers) and any special considerations relevant to the particular journal for which they will be reviewing (JPP Mentoring Policy & Suggestions for Conducting Mentored Reviews, 2009). In preparation for completing a manuscript review, mentees may also want to discuss with mentors the extent to which certain areas of critique should be a focus for the particular journals for which they are reviewing. Mentors may also provide materials for mentees to read (e.g., authors’ checklists, guidelines for manuscript reviewing; American Psychological Association Science Student Council, 2007; Sternberg, 2006; Weller, 2001) and discuss common issues that they address in their peer reviews (e.g., description of impact and clinical significance, design and statistical analyses). To help alleviate mentees’ concerns that they may not “catch all the important points” when reviewing a manuscript, it is important to remember that reviewers naturally focus on different issues, and that this is a key reason that manuscripts are reviewed by a mentor–mentee pair as well as several other reviewers. With this in mind, mentees should also be prepared for their mentor to identify areas of concern in manuscripts that had not previously occurred to them. It is these types of learning experiences which can make mentored reviews especially valuable. Ideally, mentors should also be open to discussing these contrasting points in the peer review so that their mentee can benefit from determining when and when not to include minor and major editorial points in the review.

Anticipate Logistical and Communication Issues In addition, we suggest that mentees and mentors discuss their preferences for logistical issues, such as how they will communicate during the mentored review process (e.g., e-mail, phone, inperson meeting) and how they will create the

165

mentored review (e.g., mentor and mentee write separate reviews and then merge them after discussion, mentor and mentee discuss key issues and write the review together, etc.). The mentor– mentee might also discuss and anticipate how mentored reviews may change over time for any particular student–mentor pair (Wu et al., 2011). Finally, it is recommended that the mentor– mentee discuss the review by phone or in person to flesh out the process and answer any remaining questions or concerns. This will be most beneficial for the student because there are some issues that cannot be thoroughly discussed in an e-mail. Common challenges to the mentored review process include the time commitment that is required for reviewing the manuscript, drafting the review, and communicating with their mentor; mentees having less experience with the subject areas of manuscripts; mentor/mentee timeliness; scheduling times to discuss reviews; and working across geographical distances, in some instances. As a result, mentors and mentees are encouraged to discuss these issues before and during the mentored review process, so that each party’s expectations are clearly stated and reasonable and that any challenges that arise are promptly addressed (Wu et al., 2011). Another question that ideally should be addressed early in the mentor–mentee relationship is whether or not the student will complete the mentored review for revise and resubmit papers. We believe that it is beneficial to mentees to be a part of the entire process of manuscript review, including reviewing revised manuscripts during a revise and resubmit process. However, given the time commitment involved in the review and the commitment to the process, it is important for mentees to be aware of when a revise and resubmit manuscript is due so that they can ensure that they are prepared to review and allot the necessary time.

Obtaining Proper Credit for a Mentored Review We believe that students and junior faculty should obtain proper credit for their collaborative reviews with mentors in the form of acknowledgment by name in the journal’s acknowledgement section.

166

D. Drotar et al.

Table 11.1 General strategies for reviewers Timeliness Provide a timely review that fits with the deadline for the review (if your review will be late for any reason, let the editor, and in the case of mentored reviews, your mentor, know) If you know that you cannot review the manuscript within a specified time, please decline ASAP so that the editor can invite someone else Structure and content of review Focus your review on key points of significance and contribution to science, methods and design, and clarity and consistency of reporting Organize your review on major points that are critical to address to enhance the contribution of the manuscript vs. minor points (e.g., editorial issues, consistency of presentation) Provide a succinct and clear review Give rationale for your editorial points Where possible, provide explicit suggestions for improving the manuscript and a rationale for these suggestions Give an honest, yet balanced, appraisal of strengths and weaknesses Use a constructive tone Proof your review for clarity, specificity, and tone Observe ethical guidelines for review (Palermo, 2010) If you have a conflict of interest (e.g., specific relationship with the author that could introduce bias or affect the review), let the editor know Share any ethical concerns you may have about the conduct of the study with the managing editor The content of the manuscript and the review are privileged communication and should not be shared with anyone Destroy and/or delete information from the manuscript and manuscript review after you have submitted the decision

Mentees can also list the review in their curriculum vitae as manuscript review experience. However, the way that many journals online review systems are designed, mentored reviewers can only receive credit if the managing editor is notified that an additional reviewer was involved (Lovejoy et al., 2011) and this information is retained for purposes of acknowledgement.

Specific Guidance for Reviewers Preparing to Review Before completing a review, it is very important that reviewers familiarize themselves with policies for reviewers as well as specific editorial policies for the journal for which they will be reviewing. For example, some journals have policies for the reporting of randomized clinical trials (Stinson, McGrath, & Yamada, 2003), for studies of measurement development (Holmbeck & Devine, 2009), and for reporting of results such as confidence intervals (Cumming & Finch, 2008) and effect sizes (Durlak, 2009). The American

Psychological Association Publications and Communications Working Group on Journal Article Reporting Standards (2008) has prepared information that is very helpful to reviewers. Lovejoy et al. (2011) published a very helpful, detailed overview for novice and senior reviewers offering guidance for conducting manuscript peer reviews for the Annals of Behavioral Medicine. However, many of the points discussed in this article could be generalized to manuscript reviews for other journals. Finally, the APA Publications Manual (American Psychological Association, 2010) is another useful reference for reviewers. In order to facilitate students’ and early faculty members’ knowledge and skills concerning manuscript reviews, we have developed two tables that summarize our guidance for preparing reviews. Table 11.1 summarizes general features of an effective review and Table 11.2 summaries specific areas of critique that focus on key content areas of manuscript review. Various key features of reviews are highlighted in the following sections (for additional guidance, see Drotar, 2000a, 2009a, 2009b, 2009c, 2009d, 2010, 2011; Lovejoy et al., 2011).

11

How to Write an Effective Journal Article Review

167

Table 11.2 Checklist for specific points of critique in manuscript review Introduction Is the purpose of the research specified? Is the relevance of research to the field clearly articulated? Is the scientific and/or clinical significance of the research clearly evident compared with previous research? Does the author present a clear research question? Does the author present specific hypotheses? Is a guiding theoretical/conceptual framework for the research and hypotheses clearly evident? Method Does the author provide a clear overview of the study design? If the study is part of a larger study or data set, is the relationship to the larger study and published work derived from it clearly specified? Does the author follow published standards (APA Publications and Communication Working Group on Journal Article Standards; CONSORT Guidelines for Reporting of Randomized Clinical Trials)? Are relevant human participant information and ethical considerations detailed? Is a description of the method of sampling provided? Does the author provide a clear and comprehensive description of study participants (e.g., demographic characteristics, characteristics of chronic illnesses as relevant)? Are the number and characteristics of nonparticipants (e.g., those who fit study criteria but refused participation and/ or could not be located) and reasons for refusal (if known) described? Is a complete description of sample attrition provided (e.g., number, characteristics, and reasons for attrition (if known))? Is the description of study procedures complete (e.g., data collection setting, timing of measures, methods of data collection, and quality control including training)? Are the measures well described (e.g., relationship to study aims and hypotheses, scoring and coding, reliability and validity including relevant data for current study)? If this is a study of measurement development, does the author follow journal guidelines for measurement development? Does the author provide an overview of the data analytic plan, sample size, and power considerations? Results Does the author follow American Psychological Association guidelines for the presentation of data (American Psychological Association, 2010; APA Publications and Communications Board Working Group on Journal Reporting Standards, 2008)? Are the primary sections of the results, described in an overview? Are the results presented in accord with primary study questions and hypotheses? Are data for statistical relevant assumptions presented? Are effect sizes and confidence intervals reported? Is the text describing the tables and figures integrated with tables or figures? Are missing data described? Are the results succinctly presented? Does the author make use of the journal website to present supplementary data to save journal space (if available)? Does the author use statistical analyses that document clinical significance of results (most relevant for studies of interventions)? Discussion Does the author organize results around key findings and conclusions rather than repeat key findings already presented? Does the author synthesize findings in reference to previous research as described in the previous introduction? Are the novel contributions of the present findings relative to previous research on the topic described? Is the scientific and/or clinical significance of the present results relative to previous research on the topic articulated? Are limitations of the study, relevant threats to validity (e.g., internal and external), and implications for conclusions that are drawn clearly described? Are implications for future research including recommendations for next steps in research clearly described? Are the relevant implications of the research for clinical management clearly described?

168

Importance of Timeliness of Reviews The managing editor reviews each manuscript independently, but he or she does not make a final decision and write the decision letter to the author until all of the reviews are in. For this reason, prompt disposition of a manuscript depends upon reviewers’ responsiveness at all phases of the review process. Apart from the importance to editors, timeliness of reviewers’ feedback is very much valued and appreciated by authors because of the importance of peerreviewed publications for career development. Reviewer timeliness is critical in two phases of the review process: (1) a prompt (within a day or 2) acceptance or rejection of an assignment to review and (2) once having accepted a review assignment, providing a review within the specified time limit for the journal. Reviewers, including mentored reviewers, have multiple responsibilities and time commitments and are volunteering their time to conduct reviews, which can be time consuming. Given multiple constraints, there will be times when it is impossible for you to review a manuscript within the time frame that is required. For this reason, if you see that you are going to be late with a review, or cannot complete it after having accepted the assignment, please let the managing editor know as soon as possible so that they can decide how to proceed. Such prompt notification is very much appreciated because it facilitates the efficient management of reviews. Delays in acceptance or declines of an invitation to review prolong the editorial process and create uncertainty among managing editors about whether to invite another reviewer.

Enhancing Quality of Journal Article Reviews: Process Considerations What are the critical elements of the content of an effective review? The most critical characteristics of an effective review are clarity, specificity, constructiveness, and thoroughness (Hyman, 1995). A journal article review should inform the managing editor and author of the primary

D. Drotar et al.

strengths and weaknesses of a manuscript in a focused way (see Table 11.1). In the event that a revised manuscript is requested, which is often the case, a review should provide clear, detailed suggestions for specific changes to improve the clarity of writing and the quality of the scientific contribution to the field that is most relevant for the journal that the manuscript is being reviewed. Reviewers should identify the most salient points of critique and communicate them clearly and in detail. In this regard, it is especially helpful to include examples where relevant and/or specific references so that authors can understand the key points of critique, the rationale for them, and some potential ways to address them. This is particularly important for design and statistical analysis issues. An effective review needs to carefully attend to the details of the author’s presentation and provide a balanced consideration of strengths and weaknesses contained in the manuscript. Reviewers have a difficult task balancing honest critique and a constructive, fair review. On the one hand, reviewers should be honest in their appraisal of manuscripts in providing critique to authors. At the same time, the tone and content of reviews should be both constructive and tactful (Drotar, 2000a). Although it is always difficult to hear critique of one’s work, the way in which critique is delivered will facilitate authors’ abilities to improve the quality of their manuscripts (Drotar, 2009a). The constructiveness of a review can be operationalized by concrete suggestions to improve the quality of the writing, the science, and significance of the work and an absence of critical description of how the authors should have conducted the work. It is difficult to write a fair, clear, and concise review. For this reason, we would suggest that reviewers take time to critique their reviews in order to enhance their fairness, clarity, and tone.

Organizing the Content of a Review Table 11.2 provides guidance concerning various content areas of a manuscript that might be considered when providing critique. However, in

11

How to Write an Effective Journal Article Review

light of the issues of the length discussed above, it is very important to note that it is neither necessary nor desirable to comment on each and every one of the areas that are listed. Such an approach adds significant burden to authors, especially for manuscripts that are revised. No manuscript is perfect. Consequently, it is very important that reviews focus on major or key points that limit the significance and impact. For this reason, we suggest that reviewers focus their editorial energies on the following key content areas: One of these is an explicit judgment about whether the research addresses a relevant and significant question in the field of psychology and advances the science in this field and/or has an impact on clinical care or public health (Drotar, 2009b). A second key content area is the identification of major methodological problems in design, measurement, statistical analysis that would limit the manuscript’s scientific contribution. A third relevant content area relates to gaps, inconsistencies, and ambiguities in the presentation of information (e.g., literature review and framework, methods, analysis, interpretation, and discussion of findings). A final key content area concerns the implications of the findings for future research including recommendations for research and implications for clinical care. Very much like a manuscript, an effective review should be well organized. We recommend that reviewers organize their critique by sections of the manuscript (e.g., abstract, introduction, methods, results, discussion), and focus on key points.

Focus Your Review on Key Points The most important take home message for reviewers is to focus their critiques primarily on the major scientific issues that are raised by the manuscript (see above). We also strongly encourage reviewers to clearly distinguish and identify their points of critique as major points that limit the significance and methodological quality of the manuscript vs. minor points that impact the clarity of the presentation. It should be noted that highly specific editorial points involving matters

169

of wording and writing style are much less important and generally do not warrant extensive description. Reviewers can simply include a sentence if there were many grammatical or typographical errors or lapses in APA style (or other formatting styles) contained in the manuscript. Most journals have copy editors who manage such issues as (1) correct use of English grammar and punctuation; (2) consistency of display of text, tables, and figures; (3) consistency of language; (4) consistency of style in accord with APA, American Medical Association (AMA), or another format. For this reason, reviewers should focus their critique on primary areas of specific method. On the other hand, we recognize in some instances problematic writing can obscure science in significant ways. If this is the case, it is most helpful for reviewers to identify the primary problems with the writing and include selected examples as opposed to a detailed list of each and every specific example with specific corrections. Mentored reviewers will likely benefit from discussing with mentors how to conceptualize the major points of critique and how to focus their written reviews on them, particularly if they feel more comfortable giving feedback on minor issues such as formatting or writing style.

How Long Should a Review Be? In our experience, an effective review can be provided in 1–2 single-space pages. In practice, the length of individual reviews can vary from a paragraph to as many as five single-space pages; authors receive from 2 to 3 individual reviews (sometimes more) and an editorial decision letter. Consequently, the burden on authors in responding to reviews for manuscripts that are revised is a significant one that reviewers should consider. It is not uncommon that the combination of detailed reviews and a decision letter can yield more than seven single-spaced pages of critique. For this reason, we suggest that reviewers try to achieve the difficult balance between being as concise as possible in their reviews while also being thorough. As a reviewer or mentored

170

reviewer, if you find that your list of points far exceeds this length, it may be useful to ask another colleague or your mentor for feedback on your review and specifically for suggestions on prioritizing and summarizing key points. Moreover, reviewers who collaborate with mentees on reviews face the significant challenge of integrating reviews without producing an excessively lengthy critique. For this reason, mentored reviews usually require extra attention to focusing and summarizing key points.

Describe the Content of Critique Succinctly and Clearly: Number All Specific Points Managing editors of scholarly journals generally ask authors to respond in some way to the points of critique that are made by reviewers unless they are contradictory (in which case the managing editor should provide guidance to authors to resolve the discrepant feedback). Numbering all specific points of reviewer critique will facilitate the author’s organization of their response to critique and the managing editor’s task in determining how well authors have responded to critique.

Provide Explicit Suggestions for Key Revisions and a Rationale for Considering Them Clarity of reviewer feedback and rationale for specific suggestions are very important. Such feedback informs authors and managing editors about the need for specific revisions that are recommended, the rationale for the recommendations, and how the manuscript will be improved by making the changes. On the other hand, authors and editors can be confused and burdened by ambiguous reviews. Authors may have difficulty understanding precisely what reviewers want and, as importantly, the rationale for their suggestions. Clear communication facilitates authors’ understanding and acceptance of critique. For example, consider the difference in how you might respond to a critique that indicates that

D. Drotar et al.

your statistical analysis is “problematic” and “needs to be changed” without a specific rationale vs. one that explains why the reviewer thinks the change should be made (e.g., listing the specific threat to validity that needs to be addressed and how it would improve the science described in the manuscript; and/or directing the author to additional resources that can assist with the statistical analyses, etc.).

Indicate Specific Ratings and Recommendations for Editorial Decisions that Reflect Journal Policies In addition to providing detailed comments about the manuscript that are given to authors, many journals require reviewers to rate the quality of the manuscript in relevant content domains such as design and methods, innovativeness, analysis of data, etc. Reviewers may be asked about ratings of the manuscript’s potential impact on the field, the quality of the manuscript compared with those typically published in the journal, and the overall quality of the manuscript. Managing editors appreciate the reviewers’ ratings and will use them to help guide their decision. Depending on the policy of a specific journal, reviewers may be asked to communicate their recommendations to managing editors (but not directly to authors) for a specific editorial decision, (e.g., accept, accept with revision, revise and resubmit, revise as a brief report, or reject).

Communicate Consistently with Authors and Managing Editors Managing editors have the responsibility of integrating information from all the reviews (including their own) to render an editorial decision and articulate suggestions to the author to enhance the quality of the manuscript’s revision (in the event that a revision is recommended). In order to provide the most useful information to the managing editor, reviewers should communicate all of their most important concerns in their critique to the authors.

11

How to Write an Effective Journal Article Review

In some journals, reviewers have the option of providing comments to the editor that are not seen by authors. In our opinion, such comments should be given sparingly and carefully. The reason for this is that authors should have the benefit of reviewers’ honest appraisals, which should always be consistent with the feedback given to the editor. Because confusion and frustration can be generated by discrepant editorial feedback, it is usually not helpful to give any feedback to editors’ that is highly discrepant from what is communicated to authors. Reviewers should note that most journals ask them to refrain from making any direct or implicit recommendation for acceptance or rejection of the manuscript in the narrative critique that is provided to authors. The reason for this is that reviewers may disagree concerning the disposition of a manuscript. Moreover, the managing editor’s decision may sometimes reflect a minority opinion. For this reason, it is important that reviewers familiarize themselves with the procedures and policies that are required by the journal.

How to Help Editors Manage Difficult Editorial Decisions The final editorial decision concerning the disposition of a manuscript reflects the editor’s overall judgment about the quality of the manuscript and does not necessarily reflect a consensus editorial statement among the reviewers. As noted above, it is not uncommon to encounter minor and sometimes major disagreements among reviewers about the quality of a manuscript and the nature of its contribution to the field. Discrepant training and experience, theoretical orientation, preference, and standards among reviewers can result in different feedback in critiques and recommendations for a manuscript’s disposition (Fiske & Fogg, 1990). Even when reviewers identify similar concerns about a manuscript, they may weigh their importance very differently (Fiske & Fogg). In order to facilitate managing editors’ decisions, our best advice for reviewers is to be as forthright and direct as

171

possible in their critiques. Out of kindness and empathy for authors, some reviewers are tempted to bend over backwards to give authors every benefit of the doubt and recommend a revision if they believe there is any chance that the manuscript can be accepted. However, editorial experience in working with authors with their resubmissions, some of which were eventually rejected, has underscored the inefficiency and frustrations raised by revise and resubmit editorial verdicts that turned out to be too lenient. For this reason, we suggest that reviewers focus on whether a revision will be likely to result in an important new contribution to research in psychology that transcends current research in method, results, and/or clinical significance as opposed to whether the manuscript would be improved by a revision. Almost all manuscripts would be improved by a revision, but they will not all make a contribution to science (Drotar, 2009b). A revise and resubmit decision should be considered only for those manuscripts that have the potential to advance the science and practice of psychology. It is very important that reviewers identify major problems with method, design, or significance that are not potentially correctable or feasible to correct and distinguish them from non-correctable problems.

Conducting an Ethical Review Important ethical issues need to be considered by reviewers and managing editors (Drotar, 2000a; Palermo, 2010). For example, manuscripts are to be treated as privileged and confidential communications that are seen by reviewers and editors but are not to be circulated to or seen by anyone else (with the obvious exception of mentees who participate in a mentored review). Conflicts of interest between reviewers and authors can also raise important ethical issues: Given the relatively small group of researchers in some areas of science, reviewers may be asked to review the work of authors one knows in some capacity. Knowing an author or having worked with him or her in some way does not necessarily pose a conflict. However, reviewers do need to consider whether

D. Drotar et al.

172

they are able to review a manuscript of a former student, professional associate, or academic rival as impartially as they would if they did not know them (Routh, 1995). If the answer is no, they should inform managing editors and asked to be excused from the review. Managing editors cannot readily identify such conflicts and need to rely on reviewers to do so. Consequently, reviewers have the primary ethical responsibility to recognize conflicts of interest and to excuse themselves from a review if they feel that bias could interfere with an impartial review. We encourage authors and editors to review the conflict of interest statement in the specific journals that they are reviewing for to recognize potential conflict of interests such as close, active collaboration in the content area of the manuscript, faculty members in the same institution, and current colleagues. When in doubt about whether they are in potential conflict, reviewers should feel free to consult with the managing editors. Manuscripts and reviews are privileged, confidential communication. Reviewers should not discuss the contents of manuscripts they are reviewing with anyone else or share the information about a manuscript they are reviewing with anyone else, no matter how interesting it is. Mentees should take the similar extra precautions to ensure that confidentiality is not broken (e.g., keeping manuscripts being reviewed in a location or on a private computer not accessible by others and refraining from discussing manuscript content in group settings (e.g., journal clubs)). Once the manuscript is reviewed, it is helpful to destroy or delete the manuscript and review.

Improving the Quality of Your Reviews Graduate students and early career faculty should appreciate that excellent reviewers do not come ready made. As is the case for any kind of writing, it takes a long time to develop your method and style. For this reason, practice in writing manuscript reviews under supervision is the best way to improve the quality of your reviews. Mentored manuscript reviews are one way to do

this. From the standpoint of learning, it is also very useful to compare your reviews with those of others who have reviewed the same manuscript and with the editor’s decision letter. This kind of feedback, which is provided by many editors after all reviews have been compiled, provides an opportunity to observe commentaries and differences in reviews. Another way to improve the quality of your reviews is to obtain practice in written critiques of published work. This can help to sharpen editorial critique within a class format, which can provide an opportunity to compare one’s reviews with peers (Drotar, 2000b).

Epilogue We hope our discussion will help students and early career faculty to obtain experience in reviewing manuscripts for scientific journals and to develop their skills in this important area of professional development. We invite mentors and students to describe innovative and effective training and “on the job” learning experiences in manuscript review.

References American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association. American Psychological Association Science Student Council. (2007). A graduate students’ guide to involvement in the peer review process. Retrieved July 15, 2011, from http://www.apa.org/research/publishing/. APA Publications and Communications Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology. Why do we need them? What do they need to be? American Psychologist, 63, 839–851. Cumming, G., & Finch, S. (2008). Putting research in context: Understanding confidence intervals from one or more studies. Journal of Pediatric Psychology, 34(9), 903–916. Drotar, D. (2000a). Reviewing and editing manuscripts for scientific journals. In D. Drotar (Ed.), Handbook of research methods in clinical child and pediatric psychology (pp. 409–425). New York: Kluwer Academic/ Plenum.

11

How to Write an Effective Journal Article Review

Drotar, D. (2000b). Training professional psychologists to write and publish. The utility of a writer’s workshop seminar. Professional Psychology: Research and Practice, 31, 453–457. Drotar, D. (2009a). Editorial: How to write effective reviews for the Journal of Pediatric Psychology. Journal of Pediatric Psychology, 34, 113–117. Drotar, D. (2009b). Editorial: Thoughts in improving the quality of manuscripts submitted to the Journal of Pediatric Psychology: How to write a convincing introduction. Journal of Pediatric Psychology, 34, 1–3. Drotar, D. (2009c). Editorial: How to report methods in the Journal of Pediatric Psychology. Journal of Pediatric Psychology, 34, 227–230. Drotar, D. (2009d). How to write an effective results and discussion section for the Journal of Pediatric Psychology. Journal of Pediatric Psychology, 34, 339–343. Drotar, D. (2010). Editorial: Guidance for submission and review of multiple publications derived from the same study. Journal of Pediatric Psychology, 35, 225–230. Drotar, D. (2011). Editorial: How to write more effective, user friendly reviews for the Journal of Pediatric Psychology. Journal of Pediatric Psychology, 36, 1–3. Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34, 917–928. Fiske, D. W., & Fogg, L. (1990). But the reviewers are making different criticisms of my paper: Diversity and uniqueness in reviewer comments. American Psychologist, 40, 591–598. Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and

173 validation manuscripts. Journal of Pediatric Psychology, 34(7), 691–696. Hyman, R. (1995). How to critique a published article. Psychological Bulletin, 118, 178–182. Journal of Pediatric Psychology mentoring policy & suggestions for conducting mentored reviews (2009). Retrieved July 15, 2011, from http://www.oxfordjournals.org/our_journals/jpepsy/for_authors/msprep_ submission.html. Lovejoy, T. I., Revenson, T. A., & France, C. R. (2011). Reviewing manuscripts for peer-review journals: A primer for novice and seasoned reviewers. Annals of Behavioral Medicine, 42, 1–13. Palermo, T. M. (2010). Editorial: Exploring ethical issues in peer review for the Journal of Pediatric Psychology. Journal of Pediatric Psychology, 35(3), 221–224. Routh, D. K. (1995). Confessions of an editor, including mistakes I have made. Journal of Clinical Child Psychology, 24, 236–241. Sternberg, R. J. (Ed.). (2006). Reviewing scientific works in psychology. Washington, DC: American Psychological Association. Stinson, J. N., McGrath, P. J., & Yamada, J. T. (2003). Clinical trials in the Journal of Pediatric Psychology: Applying the CONSORT statement. Journal of Pediatric Psychology, 28, 159–167. Weller, A. C. (2001). Editorial peer review: Its strengths and weaknesses. Medford, NY: Information Today, Inc. Wu, Y. P., Nassau, J. H., & Drotar, D. (2011). Mentoring reviewers: The Journal of Pediatric Psychology experience. Journal of Pediatric Psychology, 36, 258–264.