Journal of Pediatric Psychology Advance Access originally published online on March 30, 2006
Journal of Pediatric Psychology 2007 32(2):150-153; doi:10.1093/jpepsy/jsj111
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Examining the Influence of the Journal of Pediatric Psychology: An Empirical Approach
University of Kansas
All correspondence concerning this article should be addressed to Michael M. Steele, MA, 2601 Kensington Road, Lawrence, KS 66046. E-mail: myksteel{at}aol.com.
 |
Abstract |
|---|
 Top Abstract MethodResultsDiscussionReferences |
|---|
Objective To empirically examine whether research published in the Journal of Pediatric Psychology (JPP) guides research both within and outside the field of pediatric psychology. Methods Articles published in JPP from 2000 to 2004 were evaluated to determine the frequency with which articles in JPP from 2000 influenced subsequent research. Citation rates by other journals were also examined. Results Articles in JPP from 2000 were frequently cited in subsequent research, but authors rarely cited previous research as being instrumental in the development of their study design. Articles in JPP from 2000 were also frequently cited by outside journals. Specifically, journals classified as "medical and medical subspecialties" had the highest rates of citing JPP articles. Conclusions The finding that JPP is cited both within JPP and other journals provides further evidence that pediatric psychology is a thriving interdisciplinary field.
Key words: citation rates; journal content analyses; Journal of Pediatric Psychology; methodology; research impact.
In the context of the history and development of the field of pediatric psychology, the Journal of Pediatric Psychology (JPP) has been credited with being an essential element of the founding of the Society for Pediatric Psychology (SPP; Division 54 of the American Psychological Association) and has been called the flagship publication of SPP and of the field in general (Roberts, 1992
; Roberts, Mitchell, & McNeal, 2003). The link between JPP and the research life of pediatric psychology has been claimed many times. For example, in 1992, Roberts stated that the "journal is a reflection of progress in the field" and suggested that the growth and success of JPP was correlated with a similar expansion of pediatric psychology (p. 802). This link has not been empirically validated, but it is possible that this link occurs through two sources: through editors and authors. In the outgoing editorial comments of Roberts (1992), La Greca (1997), and Kazak (2002), these former editors of JPP charged the Journal to include research that reflects the ever-changing and expanding field. Additionally, Kazak (2002) described editorial decisions to conduct special issues of the journal as a means to shape the field and highlight important advancements.
Authors publishing in JPP may influence pediatric psychology research through the recommendations made at the end of research articles, which may then encourage investigation along a specific line of research. Encouraging research in particular areas is not only important to individual lines of research but also may be vital to the future of the field. Specifically, promoting and publishing research that demonstrates pediatric psychology’s "reasons for being" has been identified as an important issue for the future of pediatric psychology (Brown & Roberts, 2000). Thus, the vitality of pediatric psychology can be measured by whether JPP guides pediatric psychology research and also by determining whether JPP is being recognized by related fields, such as medicine and clinical child psychology.
Most frequently, the citation impact factor (i.e., Social Science Citation Index; Thomson.com, 2005) has been used as the accepted measure of the degree to which a journal has scientific influence (Roberts, 1992). But despite the popularity of the citation impact factor as an assessment tool, McGarty (2000) stated that popularity and reliability are no substitute for validity. Even in Roberts’s (1992) enthusiastic statements about JPP’s citation index score, he stated that this score is just one means of measuring the effect of the research published in JPP. McGarty concluded that, to take into account the amount of time that it takes for published research to impact research design and then, in turn, for that new research to be published, a 4-year span is necessary rather than the 2-year citation impact factor. Because the citation index factor may not be the most accurate measure of a journal’s influence on the scientific community, a more thorough analysis should provide greater insight into the interdisciplinary nature of pediatric psychology.
The goal of the current study is to examine the scientific impact of JPP using an alternative empirical method to the citation impact factor. First, the types of research suggestions found in JPP for the year 2000 will be examined for descriptive purposes. Next, within a 4-year span, we will determine the frequency of journal self-citations from JPP and also how often an author’s suggestions for future research are cited as influential in study design for articles published in JPP (representing a relationship between researches within the field). Finally, the citation frequency in journals other than JPP will be examined to determine the citation rates for JPP in other fields (representing interdisciplinary dissemination of pediatric psychology research, a stated goal of the journal).
|
Method |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
Procedures
All articles published in JPP during 2000 were compiled and evaluated to determine the relationship between these research articles and subsequent research articles both within and beyond JPP (as determined by articles published from 2000 to 2004). A total of 61 articles were published in JPP in 2000. However, to determine eligibility for the present study, we made an initial review of each article to evaluate whether the authors had made recommendations for future research. Editorials (n = 1), opinion pieces/addresses (n = 3), and memorials (n = 1) were not included in the review. An additional nine articles were excluded after the review, because they either made no research recommendations or only clinical recommendations were made. This resulted in a total of 47 articles being coded for the current study.
A graduate research assistant used a 4-step process to code the recommendations from the 47 articles. First, each article was read to develop a preliminary list of recommendation categories. Second, the categories were independently reviewed by two faculty members who actively participate in pediatric psychology research, and the categories were revised based on their recommendations. The final coding list consisted of 11 coding categories (Fig. 1). Third, each article was re-read by the same graduate research assistant and coded into one of the coding categories. For articles that made more than one recommendation, each recommendation was coded separately. This resulted in a total of 178 recommendations being coded for the current study. To ensure reliability of the coding procedures, a second graduate research assistant coded a random selection of the articles (25%). To determine reliability between raters, we calculated a kappa coefficient with the assumption that no single category would be more prominent than the others. Excellent agreement between raters (
= .84) was found. Finally, descriptive statistics were computed for the coding categories, and Chi-square analysis was conducted to determine whether differences between categories were statistically significant.
|
To determine whether author recommendations were indeed related to subsequent research in JPP, we used the Web of Science® citation index to locate articles for the years 2000–2004 that had cited any of the year 2000 articles. (Note: The year 2000 articles will subsequently be referred to as the "recommendation articles"). This database allows users to identify any published articles that have cited a specific article. For this study, the Web of Science® was used to identify any JPP articles that cited each recommendation article. Each publication that cited a recommendation article was examined to determine whether the citation was related to the original author’s recommendations for future research. For a citation to be considered directly related to suggestions made in the recommendation article, the author(s) had to explicitly state that the research question or design was based on those suggestions.
To determine whether or not changes in citation rates would be a significant factor for the years 2000–2004, two separate Chi-square analyses were conducted. The first Chi-square analysis examined whether there were significant differences in the number of general citations, whereas the second examined the number of citations that specifically mention author recommendations. Next, a linear regression analysis was conducted to determine whether the number of citations made by authors from the recommendation articles predicted whether their recommendations would be cited as instrumental in the formulation of later research.
To assess the degree to which articles published in JPP were related to research in other scientific journals from 2000 to 2004, we conducted a second search using the Web of Science®. The journals containing articles citing JPP recommendation articles were identified. By examining each journal’s mission, journals were coded into one of four categories: (a) medical and medical subspecialties (e.g., JAMA, Pediatrics), (b) psychological, (e.g., Journal of Consulting and Clinical Psychology), (c) multidisciplinary (e.g., Obesity Research), or (d) journals from other disciplines (e.g., Social Work in Health Care). Similarly, Chi-square analyses were conducted to determine whether observed differences between journal types were statistically significant. As with the recommendation categories, it was assumed that the journal categories would be equally represented and that no single category would be more prominent that the others.
|
Results |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
Results indicated that, on average, authors made 3.79 recommendations per article with a range of 1–13 recommendations. The majority of recommendations centered on expanding the variables examined for a specific area of study, whereas the fewest focused on determining the clinical significance of research findings (Fig. 1). Chi-square analyses indicated a significant difference in the number of recommendations across coding categories:
2(10, n = 179) = 134.84, p < .01.
A total of 33 (70.2%) articles from the year 2000 articles were cited in JPP from 2000 to 2004. These 33 articles were cited an average of 3.24 times with a range of 1–16 citations and a total of 107 citations. Chi-square analysis showed that, for the years 2000–2004, there were no significant differences in the number of general citations by year: 2(4, n = 49) = 9.27, p > .05. Similarly, Chi-square analysis showed that the number of citations that specifically mentioned the original author’s recommendations as a guide for the current study were not significantly different across the 4 years sampled: 2(3, n = 11) = 6.09, p > .05.
Analyses indicated that, of the 107 citations, only nine (8.4%) specifically referred to the original author’s suggestions for future research. Additionally, self-citation of an author’s own recommendations was relatively low (3% of all authors). To determine whether the number of suggestions made by the authors was related to subsequent research, we conducted a linear regression analysis. The number of recommendations made by authors was entered as the independent variable and the number of recommendations acted on was entered as the dependent variable. Results indicated that the number of recommendations made did predict the chance that a recommendation was cited as influential to research design F(1, 32) = 4.40, p < .05.
Analysis revealed that, during 2000 and 2004, a total of 146 journals cited research from the recommendation articles (i.e., papers citing recommendation articles; n = 284). Further analyses indicated that the majority of citations appeared in journals coded as "medical and medical subspecialties" (n = 58, 39.7%), the next highest number occurred in "psychological" journals (n = 45, 30.82%), the next highest in "multidisciplinary" journals (n = 32, 21.92%), and the fewest number of citations were in journals coded as "other disciplines" (n = 11, 7.5%). Chi-square analysis revealed statistically significant differences in frequency across journal categories: 2(3, n = 146) = 33.01, p < .05. Follow up (i.e., posthoc) Chi-square analyses indicated that citations in "other" journals were significantly lower than journals categorized as medical and medical subspecialties: 2(1, n = 69) = 32.01, p < .001, multidisciplinary: 2(1, n = 56) = 20.64, p < .001, or psychological: 2(1, n = 43) = 10.26, p < .01. Furthermore, the recommendation articles were cited in psychological journals significantly less frequently than in medical and medical subspecialties journals: 2(1, n = 90) = 7.51, p < .01.
|
Discussion |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
The results from the current investigation offer a number of interesting findings. First, the majority of articles written in 2000 were cited in subsequent articles. This suggests that authors are reading the research of colleagues within the field and using it to inform their own research. Thus, research in the journal does appear to be influencing subsequent research within the field. The limited evidence of direct influence of author recommendations on study design, although unexpected, makes intuitive sense. Researchers often have ongoing, well-defined, and long-term lines of research, such that changing a study design based on the recommendations of a peer’s published articles might be difficult to do. Given this reality of research, the existence of any evidence of direct impact is remarkable, thus providing more evidence for the vitality and impact of JPP. The finding that the number of suggestions made by an author significantly predicted the likelihood of that author’s recommendations being used for study design is an interesting one. It could be that numerous study recommendations are more likely to grab the attention of a researcher, who will in turn use a recommendation in future study design.
Finally, the finding that JPP is being cited in other journals may indicate that researchers in other fields are reading JPP and incorporating pediatric psychology into their own research. An alternative explanation is that pediatric psychologists are also publishing their research in these medical and medical subspecialties journals with citations from JPP articles. However, the large number of medical and medical subspecialties journals that cite research from JPP indicates that, at the very least, medical researchers are being exposed to research published in JPP. In that sense, there does appear to be a "spreading" of pediatric psychology to physicians and other medical researchers who might not necessarily be otherwise exposed to primary psychology journals.
This analysis of JPP is not meant to be comprehensive. For example, the data do not allow for a determination of who is writing the articles in the "non-JPP" journals or what these articles are about. Thus, the research citing JPP articles published in other journals may be conducted by pediatric psychologists who are publishing in diverse journals, which is a different phenomenon than other mental health professionals or physicians citing JPP articles. This still indicates that professionals outside pediatric psychology are exposed to research influenced by JPP articles. Future research is recommended to examine who is publishing in JPP, clinical child psychology, and medical and medical subspecialties journals. Such research should provide some quantification of the interdisciplinary relationships between pediatric psychology and related fields.
Additionally, the time frame for assessing citation rates does not allow for a comprehensive assessment of each and every study that will be developed based on author recommendations. By way of recommendation, we suggest additional research related to the citation of JPP use a longer window of assessment. We offer this initial study as a benchmark for subsequent evaluations.
Received February 14, 2005; revision received June 10, 2005; revision received December 10, 2005; revision received February 20, 2006; accepted March 5, 2006
|
References |
|---|
|
TopAbstractMethodResultsDiscussionReferences |
|---|
Brown K. J. and Roberts M. C. (2000) Future issues in pediatric psychology: Delphic survey. Journal of Clinical Psychology in Medical Settings 7:5–15.[CrossRef]
Kazak A. E. (2002) Journal of Pediatric Psychology (JPP), 1998–2002: Editor’s Vale Dictum. Journal of Pediatric Psychology 27:653–663.
La Greca A. M. (1997) Reflections and perspectives on pediatric psychology: Editor’s Vale Dictum. Journal of Pediatric Psychology 22:759–770.
McGarty C. (2000) The citation impact factor in social psychology: A bad statistic that encourages bad science? Current Research in Social Psychology 5:1–16.
Roberts M. C. (1992) Vale dictum: An editor’s view of the field of pediatric psychology and its journal. Journal of Pediatric Psychology 17:785–805.
Roberts M. C., Mitchell M. C., McNeal R. (2003) The evolving field of pediatric psychology: Critical issues and future challenges. In Roberts M. C. (Ed.). Handbook of Pediatric Psychology 3rd (Guilford Press, New York) pp. 3–18.
Thomson.com. (2005) Social Sciences Citation Index®. Retrieved November 27, 2005, from http://scientific.thomsom.com/products/ssci.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  B. S. Aylward, M. C. Roberts, J. Colombo, and R. G. Steele Identifying the Classics: An Examination of Articles Published in the Journal of Pediatric Psychology from 1976-2006 J. Pediatr. Psychol., July 1, 2008; 33(6): 576 - 589. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||