Journal of Pediatric Psychology Advance Access originally published online on September 8, 2005
Journal of Pediatric Psychology 2006 31(2):227-232; doi:10.1093/jpepsy/jsj071
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Editorial: Pediatric Psychology and the National Institutes of Health
Department of Public Health, Temple University
All correspondence concerning this article should be addressed to Ronald T. Brown, PhD, ABPP, Temple University, 3307 North Broad Street, Philadelphia, Pennsylvania 19140. E-mail: rtbrown{at}temple.edu.
External grant support has assumed increasing importance in the field of pediatric psychology. For example, during 2004, 41.5% of research articles published in the Journal of Pediatric Psychology contained author acknowledgments noting that the research was supported by the National Institutes of Health (NIH), and another 27.8% of the articles published acknowledged support from another federal agency or foundation. Thus, over two-thirds of research published in the Journal has endorsed support from external agencies. Although specific data are as of yet unavailable for 2005, it is suspected that external support for research articles published in 2005 will exceed that of 2004. Thus, investigator-initiated grants are driving research in pediatric psychology. In contrast, two decades ago little research in pediatric psychology was externally funded, least of all by federal agencies such as the NIH.
Although there are many pediatric psychologists who are well integrated into the NIH system, there are many of us who are well trained, excellent scientists and productive investigators yet are not necessarily integrated into the NIH system. Over the past decade, the NIH has attempted to target investigators at various career levels from the predoctoral student to the very experienced researcher. For the predoctoral student there is the F-31 award which typically supports dissertation research. A similar award level exists for the postdoctoral fellow (F-32) that supports training in postdoctoral level research. At the next level, there are the K-awards that target clinicians who wish to become researchers (K-23) as well as other type of K-awards for nonclinician scientists. The K-23 award is most relevant for pediatric psychologists who have attained training as practitioners yet wish to engage in science in a health care related area and setting. Indeed, the K-award is an excellent means for clinicians to obtain research training and support during the beginning years of their career as pediatric psychologists. Following the K-award, the investigator is next ready for the R01 award which is an independent application for a specific program of research. The NIH has within its portfolio a number of other types of awards that are designed to assist investigators in collecting pilot data for their programs of research (R-21). There also exists the program project type of award (PO) for the experienced investigator wishing to conduct an entire program of research across several areas. Other programs also exist including midlevel research career awards, and the interested reader is referred to www.nih.gov. Admittedly, the NIH is complex and many find the process quite intimidating. There frequently are presentations pertaining to the application process at several professional association meetings including the annual meetings of the American Psychological Association, the American Psychological Society, and the Society of Behavioral Medicine among other national and international associations. On one occasion, the Society of Pediatric Psychology sponsored an NIH presentation related to the grant application process at the annual meeting of the American Psychological Association in San Francisco.
Arguably, the grant review system for extramural research, as exemplified by the NIH, is the prototype against which research throughout the world is judged (Drazen & Ingelfinger, 2003
). In large part, this is because the system is based primarily on the merits of the research rather than politics. Although some exceptions have been noted (Drazen & Ingelfinger, 2003) and have tested the political process of the US Congress (Weiss, 2003), I for one still continue to be impressed with the majesty of the grant review process both at the NIH, where I have served as a member on a study section for the past several years, and at other federal and foundation agencies that model themselves on the NIH’s peer review system. This includes research related to health care in children and adolescents, which includes basic science research at the cellular level, epidemiological studies examining prevalence of disease across social classes and racial groups, and research related to behavior and physiological outcomes (the type of research that frequently makes its way into publication in the Journal). Regardless of the type of research, the underlying theme is that the research must have a national public health concern.
The NIH’s scrutiny of scientific research is impressive. First, there are specific requirements for scientists that review others’ work as part of the study sections at the NIH. These requirements include an ongoing history of federal funding and a history of research accomplishments, including articles published in peer-reviewed research that have been demonstrated to have had a significant impact in the literature. Thus, once an investigator conceptualizes an idea for a research program and the idea is discussed with colleagues (research in pediatric psychology is frequently multisite because of the small number of participants at various institutions), the idea is put into an NIH grant application wherein the investigator(s) details the need for the project and explains how it will advance knowledge in our field. Also, the application must provide significant detail on how the investigation would be conducted, participants for the proposed study, and the type of data analyses that will be employed.
Once the application is prepared—which is frequently no small feat: most applications are typically 25 single-spaced pages and include an array of forms—the applications are triaged by the Center for Scientific Review at the NIH. The Center for Scientific Review is independent of the various institutes and various initial review groups (IRGs or "study sections") has under its direction. Many of the IRGs are composed of as many as 40 standing members who are esteemed scientists in a designated area of expertise. For example, many of the grant applications in the fields of pediatric psychology, health psychology, and behavioral medicine make their way to either the Behavioral Medicine, Intervention, and Outcome IRG or the Risk Health Prevention and Behavior IRG. It should be noted that an applicant may request a specific study section assignment in a cover letter that accompanies the application. Regardless of the IRG, each application receives rigorous review by three or four members or guest reviewers who have particular expertise in the area of the application reviewed. Anyone on the IRG who has a conflict of interest with the investigators or the investigators’ institution is asked to recuse themselves from reviewing the application or providing any evaluation regarding the application.
Reviewers are requested to provide an evaluation of the application in several domains, including the importance or significance of the proposed research, the planned approach for addressing the questions it proposes, the innovativeness (the potential to add to the extant research literature), the qualifications of the investigators—as indicated by their published work—to pursue the proposed program of research, the environment in which the program of research will take place (including the quality of institutional resources), the appropriateness of the proposed budget, and finally the ethics of the research as related to issues of human participants. Each of the reviewers is asked to assign a score to the application of 100 ("best") to 500 ("worst"). It is estimated that reviewing an individual grant application takes approximately 5–8 hr, including reading the application, discussing the application at the meeting and providing a written critique, a copy of which is given to the applicant.
Study sections or IRGs typically meet three times a year and applications ranked in the top 50% are discussed. Typically, the primary reviewer discusses and critiques the application in detail, and the two reviewers provide discussion and critique of the application’s scientific merits. In all cases, reviewers are requested to address the seven aforementioned areas and to address only scientific issues. Political issues with regard to the investigator or the scientific merits of the work are always discouraged (Drazen & Ingelfinger, 2003), and in fact there is extraordinary peer pressure among the members of the IRG to focus solely on the scientific merits of the application.
Following the meeting of the IRG, the scientific review administrator for each IRG compiles the list of grant applications in order of priority scores, the scores are converted to a percentile score, and those applications with the lowest percentile score are considered for funding by the various Institutes. Depending upon the funding available, grant applications are considered for payment. Funding of grant applications has varied throughout the years; over the past 5–7 years, many of the institutes have been able to make payments down to the 20th percentile including the National Cancer Institute, whereas other Institutes with smaller resources have held payment lines to lower levels. More recently, funding for the NIH has not been increased at levels beyond inflation and thus payment for grant awards has been tighter. Despite the payment lines of the various institutes, pediatric psychologists have frequently managed to get their research projects funded because of the important questions that they have asked over the years and their strong training in research design, methodology, and statistics, albeit after several submissions.
The NIH grant review process is not without difficulties. Clearly, the strengths and weaknesses of the system are displayed during the grant review process (Lavigne J., personal communication). Owing to the keen competition of the NIH system, only the very best grant applications are funded. Herein lays the problem because very good grants may not be funded. Because the system is so competitive, the process forces the very good grant applications to even get better before making funding possible. Indeed this makes for increasingly good science, whereas at the same time it also means that reviewers become more fastidious in what they expect of an application in order for the applicant to receive a fundable score. The good news is that this clearly makes for good science; the bad news is that such a process may lead reviewers to make comments that may not necessarily always be substantive. Such a process seems to be human and my experience suggests that it is not out of maliciousness or with the intent of obstructing an investigator. Rather, there is increasing pressure from the Center for Scientific Review at the NIH to "spread out" the scores and separate the very best applications from those that are very good, albeit not good enough for funding. In this process, approximately one-half of the applications submitted are not scored which clearly can be devastating to an applicant, particularly those among the readership who are more junior. One coping mechanism with this dilemma that I believe may prove useful is to attend more to the critique rather than the score provided and to recognize the context whereby the NIH system generally suggests that 50% of applications not be scored. I do believe that it is important to remember this process as an applicant because it generally assists the applicant in making attributions as to why a score was not necessarily sufficient for funding. The critical issue here is that persistence is imperative. There are many applications to the NIH that do not get funded initially yet are frequently funded on the second or third review cycle; three is the maximum number of time that an investigator may submit the same application.
As an investigator, at times I have been disappointed with the outcome of my evaluation, although I am always impressed with the grant review process, including the comprehensive reviews and constructive feedback. I believe these reviews have taught me to be a better scientist. More recently, as a member of an IRG, I am amazed how scientists come together to review applications, prepare detailed reviews before the meeting, and at times agonize over a grant application to provide a fair and constructive review. Applications that are deemed to be potentially significant, innovative, and methodologically rigorous are typically funded, whereas applications with questions in one or more of these areas are not approved for funding. As an investigator, the important thing to remember is that at first if you do not succeed, respond to the criticisms of the reviewers carefully, and resubmit your application.
Finally, the process is not over when the investigator receives an award. Progress toward meeting research goals is monitored annually; the research team is expected to complete the program of research and have viable data to report once the research is complete. Eventually the investigator is expected to publish results of the investigation and this becomes an important criterion if further research funding is awarded. Although there is some variation with regard to investigators’ productivity and, ultimately, to the contributions derived from extramural research, most research grants lead to published research. In fact, there has been a marked increase in the submission of manuscripts to the Journal over the past 3 years, which in part may be attributed to the increase in funding at the NIH, as well as the fact that pediatric psychologists employed at health sciences centers are increasingly responsible for generating their own compensation, whether through extramural funding or clinical activities. As noted earlier, the NIH, through its more recent mechanisms of funding, also has allowed more junior investigators to advance in their research careers through the mentored K-series awards and the R-21 grant award that funds pilot projects. The traditional pre- and postdoctoral fellowship award mechanisms also continue to be popular.
Although not without some difficulties, the NIH extramural grant review process—as well as the grant processes of other federal agencies and foundations modeled after the NIH—has been rigorous, with the result being research that has enhanced the public health of individuals in this nation. Funding has been provided to productive investigators who have developed innovative and excellent research and have been successful in disseminating this research. These are the individuals who publish their work in the Journal and have made a contribution to the field of pediatric psychology. Indeed some have been concerned pertaining to the political threats of the peer review system (Drazen & Ingelfinger, 2003). Some organizations have engaged in more policy related activities in this domain including the American Psychological Association. I anticipate that pressures on Congress by universities, professional associations and organizations to preserve the peer review system at NIH will serve to productive. My hope also is that over the next several years that those who have authored articles in the Journal or members of the Society of Pediatric Psychology who have had successes with the funding mechanisms at the NIH will mentor our less-experienced colleagues in this process by navigating them through the NIH process, assisting them in the writing of applications, reading of drafts and the revision of applications after peer review.
 |
NIH Standards of Research and Peer Review and the Impact on Pediatric Psychology |
|---|
 Top NIH Standards of Research... AcknowledgmentsReferences |
|---|
In recent years, there has been significant pressure—and in many cases federal mandates—placed upon both investigators who receive extramural funds from the NIH (and other federal agencies) as well as those who have not received extramural funding to conform to a number of standards. These standards include regulatory issues, informed consent, conflict of interest, translational research, registration of clinical trials, and advance access to data from large investigations as well as access to information that will eventually be available for publication. Please allow me to review each of these issues and how the Journal proposes to be responsive to them. In an attempt to elicit the readers’ thoughts on these issues, I also pose some questions and dilemmas.
Regulatory Issues
It is fairly apparent that regulatory issues have become a standard in academic research. In recent years, the requirement is that investigators be trained in the conduct of ethical research and be competent in conducting ethically responsible research. Although potential authors are required to certify that they have conformed to the ethical standards of the American Psychological Association in the conduct of their research, this raises the question as to whether authors should certify that they have been trained in the conduct of ethically responsible research. Although all authors are required to attest within the text of their articles that their research had been approved by the Institutional Review Board of their institution, currently no documentation of research and regulatory training is required. Perhaps we may take comfort in the fact that the NIH requires some ethical training with regard to the conduct of research. Thus, over 40% of our authors as well as their coauthors have completed this training. Ethical and regulatory issues are serious and are not a subject for debate. Investigators must follow institutional and professional guidelines. Although my preference is to keep the publication bureaucracy to a minimum with regard to documenting these requirements for publication in the Journal, I invite your comments with regard to this issue.
Informed Consent
As noted, all authors must assure both the editor and the readership that informed consent was obtained from all participants involved in a specific investigation, that risks were identified, potential participants were informed of such risks, and that the risks were kept to a minimum. The Journal is dependent on the author’s institution that the appropriate review took place. If not already present in the manuscript, the author is requested to insert this information in the text. The dilemma here is that the editor is dependent on the author’s institution to assure compliance with regulatory issues pertaining to the governance of Institutional Review Boards. Despite the federal regulation of such review boards, institutions enforce regulatory requirements with varying degrees of rigor. We know of institutions that have been cited for not following regulatory procedures and have even lost federal funding for a short period of time. Nonetheless, research approved under such Institutional Review Boards has made its way to the Journal. Our hope is that all Institutional Review Boards are regulated to the same degree thereby allowing confidence in the publication of pediatric research with regard to protection of human participants.
Conflict of Interest
All prospective authors (or the senior author on behalf of the other authors) are requested to sign a "Conflict of Interest Form" certifying that they had no conflict of interest in employment or consultation for a pharmaceutical firm or medical device company. In fact, such conflicts have been rare. The issue of "Conflict of Interest" perhaps is a concern when an author has received funding from a federal agency and is compelled to support the hypotheses of a funded investigation when the data are tenuous regarding support of those hypotheses. Universities and hospitals also may pose conflicts of interest when they discourage the reporting of findings that are not necessarily socially desirable (e.g., the minimal effects of an illegal drug substance on the developing fetus). These are issues that we must keep in mind when disseminating our research findings.
Translational Research
Lenfant (2003) has suggested that our capacity to bring laboratory-based discoveries related to health care to those individuals who are in need of such discoveries may be "lost in translation." In pediatric psychology, most of us do a fine job in conducting clinically relevant and ecologically valid research. There is little debate concerning the clinical relevance of most of the articles published in the Journal. However, one concern is whether as investigators pediatric psychologists attend sufficiently to findings of laboratory-based research in generating specific hypotheses for their own research as well as explaining data from their investigations. It is hoped that, over time, there will be greater integration of laboratory discoveries and clinical research in the field of pediatric psychology. I would certainly welcome such articles for potential publication in the Journal.
Registration of Clinical Trials
The International Committee of Medical Journal Editors (ICMJE) has now required that a clinical trial may only be considered for publication in one of the major medical journals (e.g., Journal of the American Medical Association, New England Journal of Medicine, The Lancet, Annals of Internal Medicine) if it has been registered before the enrollment of the first participant (DeAngelis et al., 2004; DeAngelis et al., 2005). The ICMJE defines a clinical trial as "Any research project that prospectively assigned human subjects to intervention and comparison groups to study the cause-and-effect relationship between a medical intervention and a health outcome" (DeAngelis et al., 2004). The committee defines "clinical" as "any intervention used to modify a health outcome" and one that includes a concurrent control or comparison group (DeAngelis et al., 2004). Behavioral treatments are included as interventions; any investigator wishing to publish a controlled trial in a medical journal must demonstrate that the trial has first been registered. Thus, if a pediatric psychologist wished to publish a controlled clinical trial in any of the journals covered under this agreement, the investigation would need to be registered.
Registration signifies that the study is publicly accessible at www.clinicaltrials.gov and that the following are included in the registration: unique trial number, the trial registration date, secondary identification numbers, funding sources for the investigation, the primary sponsor of the research, the secondary sponsor of the investigation, the responsible contact person, the research contact person, the title of the study, the official scientific title of the investigation, whether the study received appropriate ethics committee approval, the medical condition under investigation, a description of the intervention and the comparison/control intervention, key inclusion and exclusion criteria, specific drop downs for selection including whether the investigation is randomized versus nonrandomized, the type of controls, etc., the anticipated trial start date, the targeted sample size, the recruitment status of the investigation, the primary outcome that the study was designed to evaluate, and the key secondary outcomes specified in the protocol.
The underlying rationale for such a registry is to acknowledge the participants who "placed themselves at risk by their volunteering for clinical trials." More specifically, this record serves to attest that decisions made about patient care were based on all of the available evidence that was examined through research and not simply those clinical trials that an author may have chosen to report or that was eventually accepted for publication in a journal.
Admittedly, registration of clinical trials has not been part of our practice in pediatric psychology. However, if we are to sustain the same standard as medical journals, it would seem reasonable that we follow the same practice. Many of our contributors including recipients of NIH awards to fund clinical trials in the area of pediatric psychology are not necessarily aware of this registration issue. Making investigators aware of this issue is apt to be an issue. The readers are also reminded of the CONSORT requirements with regard to clinical trials that have previously been reviewed in the Journal (Brown, 2003; McGrath et al., 2003). My only reluctance with regard to registration of clinical trials is placing one more requirement upon an author or investigator in addition to the list of regulatory and Institutional Review Board requirements. As both readers and contributors of the Journal, I would look forward to learning your thoughts about this.
Advance Access
Similar to issues of registration of studies and making knowledge public with regard to research (and particularly research studies that are supported by federal agencies) is the notion of advance access to studies that will be published in journals. Thus, in attempting to adhere to public sharing of research (including that research that is supported by extramural research), the Journal, through Oxford University Press, has launched Advance Access that will enable the Journal to publish manuscripts online in their final form approximately 1–2 months after acceptance of the manuscript yet before publication. As with many psychology journals, the time from acceptance of a manuscript for publication and actual publication in the Journal was about 6 months to 1 year. All manuscripts accepted for publication in the Journal will be copyedited, typeset, and proofed immediately after acceptance. At the time that authors’ corrections have been incorporated into proofs, the manuscript will be put into Advance Access. (It will be taken off Advance Access when the manuscript has been incorporated into an issue of the Journal.) It should be noted that the version of the article that appears in Advance Access and in the issue of the Journal will be identical and no changes will be made between the time it appears in Advance Access and the time it appears in the Journal. In this way, the accepted manuscript is available almost immediately upon acceptance and interested readers will have almost immediate access to the article.
Beginning with Volume 31, in 2006 the Journal of Pediatric Psychology will expand its pages and will add two additional issues per year. Thus, beginning in 2006, the Journal will publish 10 issues per year. The advantages of adding additional pages and two issues per year are that, first, it will reduce the publication lag time significantly and, second, it will also allow us to publish more articles than had been possible in the past.
I look forward to these exciting new changes to the Journal of Pediatric Psychology and to your feedback pertaining to the issues that I have raised in this editorial.
|
Acknowledgments |
|---|
|
TopNIH Standards of Research...AcknowledgmentsReferences |
|---|
The author extends his appreciation to Maureen Black, Barbara Fiese, Grayson Holmbeck, John Lavigne, and Lonnie Zeltzer for providing critical comments and information pertaining to this article.
Received August 1, 2005; accepted August 1, 2005
|
References |
|---|
|
TopNIH Standards of Research...AcknowledgmentsReferences |
|---|
Brown, R. T. (2003). Editorial: The Journal of Pediatric Psychology will support the publication of clinical trials. Journal of Pediatric psychology, 3, 173.
DeAngelis, C. D., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., et al. (2004). Is this clinical trial fully registered?—A statement from the International Committee of Medical editor. New England Journal of Medicine, 352, 2436–2438.
DeAngelis, C. D., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., et al. (2005). Is this clinical trial fully registered?—A statement from the International Committee of Medical editor. New England Journal of Medicine, 352, 2436–2438.
Drazen, J. M., & Ingelfinger, J. R. (2003). Grants, politics, and the NIH. New England Journal of Medicine, 349, 2259–2261.
Lenfant, C. (2003). Shattuck lecture: Clinical research to clinical practice—lost in translation? New England Journal of Medicine, 349, 868–874.
Stinson, McGrath, P., & Yamada, J. T. (2003). Clinical trials in the Journal of Pediatric Psychology: Applying the CONSORT statement. Journal of Pediatric Psychology, 9, 159–167.
Weiss, R. (2003, October 30). NIH faces criticism on grants: Coalition assails "smarmy projects." The Washington Post, p. A21.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  T. Wysocki Editorial: Journal of Pediatric Psychology Statement of Purpose Section on Randomized Trials J. Pediatr. Psychol., January 1, 2008; 33(1): 12 - 15. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||