Journal of Pediatric Psychology, Vol. 26, No. 7, 2001, pp. 407-415
© 2001 Society of Pediatric Psychology
Using Process Evaluation to Describe a Hospital-Based Clinic for Children Coping With Medical Stressors
1 Harvard Children's Initiative/Harvard University, 2 Children's Hospital/Harvard Medical School
All correspondence should be sent to Susan Douglas Kelley, Children's Hospital, Department of Psychiatry, Fegan 8, 300 Longwood Ave., Boston, Massachusetts 02115. E-mail: susan_d_kelley{at}harvard.edu .
 |
Abstract |
|---|
 Top Abstract IntroductionMethodResultsDiscussionReferences |
|---|
Objective: To use process evaluation methods to describe the development of a hospital-based mental health clinic for children facing medical stressors.
Methods: Over a 21-month time period, we collected data regarding presenting concern, service use, and referral source using hospital administrative, clinic intake, and clinical records for 356 children.
Results: Nearly 90% of the children were referred to the clinic from sources within the hospital. With the exception of single session interventions, there were no differences in average length of services according to presenting concern. Hospital pediatric specialists and psychology consultants were the primary referrers to the program. Pediatric specialists referred more often for procedural concerns and chronic illness than other hospital referrers.
Conclusions: These findings support the feasibility and usefulness of a process evaluation approach in shaping clinical program directions, creating opportunities for collaboration with medical providers, and planning effectiveness research.
Key words: psychological intervention; medical stress; coping; process evaluation; tertiary care.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Children and adolescents facing medical stressors represent an important clinical population in need of mental health intervention. The impact of a medical stressor may range from the sudden onset of physical illness or injury that resolves after a short time period to a chronic health problem that impinges on a child's daily life. Estimates are that 10 to 20 million American youths have a chronic physical illness (American Academy of Pediatrics, 1993
). Although most of these conditions are relatively mild, at
least 10% of children with chronic conditions have a physical illness severe
enough to affect their daily lives
(American Academy of Pediatrics,
1993). In addition, each year nearly 5 million American children
and adolescents experience a wide range of medical procedures for diagnostic
or treatment purposes (Bush, Melamed,
Sheras, & Greenbaum, 1986).
There is an increasing emphasis on developing and implementing empirically
supported mental health interventions for pediatric populations, including
efficacy (LaGreca & Varni,
1993; Wallander &
Thompson, 1995) and effectiveness studies
(Drotar, 1997;
Kelleher, 1999). This growing
awareness of the need to expand intervention research underscores the
importance of using efficacious treatment strategies and research agendas in
"real world" clinical settings. Kelleher
(1999) suggested that
"pediatric psychology investigators... must expand their models of
intervention research to consider the myriad factors that influence the
implementation, dissemination, and conduct of interventions in primary care
settings" (p. 460). He noted that system-level factors, such as
organization of services and provider characteristics, might affect the
effectiveness of such interventions. Indeed, such implementation issues should
be under-stood before attempting to study the impact of an intervention on
children's outcomes and can be viewed as intermediate outcomes themselves
(Kelley, Nixon, & Bickman,
2000).
Mental health services research offers a needed approach to examining
moderating factors that affect use and access, as well as individual outcomes
for children and families (Barnes, Stein,
& Rosenberg, 1999; Drotar,
1997; Kelley et al.,
2000). One component of mental health services research, process
evaluation, provides an empirically based and comprehensive approach to
understanding how a program functions. Process evaluation, also known as
formative evaluation or program monitoring, uses a variety of methodology
tools, including record review, interviewing, focus groups, on-site
observation, and case studies (Brownson et
al., 1997; Devaney & Rossi,
1997; Owen & Rogers,
1999; Weiss,
1998). Process evaluation findings are typically used both to
identify potential revisions to improve a program and to plan an impact
evaluation.
Process evaluation focuses on system and organizational factors in the
implementation of clinical services, which corresponds to the investigation of
outcomes in pediatric settings as identified by Kelleher
(1999). Furthermore, process
evaluation presents a responsive methodological approach to introducing a
research agenda into clinical settings. It has been suggested that conducting
a process evaluation can lead to changes in attitudes about and increased
commitment to evaluation research (Owen
& Rogers, 1999). For interventions in pediatric settings,
evaluation of referral patterns for mental health services is critical, given
that medical providers are often the gatekeepers to psychosocial services
(Costello et al., 1988;
Dulcan et al., 1990). In
addition, understanding the nature of the presenting concerns for referred
children and the initial use of services would allow a program to better
target clinic needs.
This study presents initial findings from an ongoing process evaluation of a hospital-based mental health clinic designed to help children facing medical stressors. A primary goal of the clinic was to identify existing clinical needs of patients and providers within the hospital and to develop specialized interventions and research initiatives responsive to those needs and feasible given resource constraints. The process evaluation was conducted during the phase of clinic development and was used to inform its subsequent evolution and continuing evaluation.
We examine the feasibility of establishing this clinic by describing the system and organizational components that affect the use of clinic services, including child's presenting concern characteristics, service use factors, and referral source information. This project is unique in that it represents the first description of the use and integration into hospital services of a medical coping clinic for children affected by medical stressors. The discussion will present the findings related to development of the clinic, as well as focus on the benefits of an ongoing program evaluation approach in identifying future questions.
|
Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Program Description
The hospital records approximately 18,000 inpatient admissions each year, and its more than 100 outpatient clinics and emergency services care for over 275,000 patients annually. The outpatient program in the hospital's Department of Psychiatry has about 19,000 patient visits annually, of which nearly 30% are for youngsters with physical illnesses. These outpatient services are divided into a general clinic as well as several specialized clinics (i.e., urgent care, Latino, neuropsychology, and mood disorders). The department also operates a pediatric psychiatry consultation program that evaluates nearly 800 medical and surgical inpatients each year. All program services are funded through third-party payors or self-pay by parents. Because the hospital is a member of nearly all state insurance networks, almost all families can be seen in the department.
Informal communications with medical providers and a review of referrals from the department's pediatric psychiatry consultation program indicated a need for an outpatient program that would centralize and standardize the mental health care of children facing medical stressors. In response to this need, the Medical Coping Clinic was established in the Department of Psychiatry in 1998. Several avenues were used to disseminate information about the clinic within the hospital, including staff web-sites, staff and patient newsletters, liaison and development of materials for a hospital-based family resource center as well as direct contact with individual pediatricians, nurses, psychology/psychiatry consultants, child life specialists, and social workers throughout the hospital.
The name, Medical Coping Clinic, was selected to emphasize positive adaptation to physical illness and to facilitate parental acceptance of a mental health referral. The mission of this program was to provide and study a range of mental health interventions for children facing acute and chronic medical stressors. Using a biopsychosocial approach, the clinic aims to assess the interplay of medical and psychosocial issues; build on existing strengths and protective resources of both the child and the family; provide cognitive, emotional, and behavioral coping strategies to manage stressors; and provide psychoeducation about the psychological aspects of medical disease. The clinic model emphasizes ongoing contact and collaboration with medical and support staff around these issues as a valuable component of the child's overall health care. The staff of this clinic consists of 1.5 full-time equivalent (FTE) staff psychologists, 0.2 FTE staff psychiatrists, and 1.0 FTE trainees in both psychology and psychiatry. For training purposes and to facilitate awareness of current empirical intervention research, teaching conferences were held weekly.
All referrals for outpatient care are made through a central intake office. For the majority of cases, parents are given the intake office's telephone number by a referring party and are asked to call the intake office directly for an appointment in the Medical Coping Clinic. A smaller number of children are referred to the clinic by intake office staff because a parent calls with a primary medical concern.
Prior to scheduling the first evaluation appointment, cases referred to the clinic are reviewed by the clinic's co-director (MVH). Cases that fit the Medical Coping Clinic criteria are assigned to individual clinicians based on availability, area of expertise, and expected match with the family. Cases that do not fit the clinic criteria (i.e., those that do not involve medical issues) are reassigned to one of the other outpatient psychiatry services or are referred to an appropriate external agency.
Sample Selection
Three sources of record review provided information for the process
evaluation. Hospital administration records provided information on child
gender, age, and service use characteristics, including date of initial
evaluation appointment and length of services. Intake office records provided
information about a child's presenting clinical concern and referral source.
Records from these two sources were then matched with clinic medical
records.
Using the above process, we reviewed clinic records between September 1998 and May 2000, resulting in an initial sample of 406 children. For 35 cases, the date of clinician assignment could not be verified. In the majority of these cases, the patient did not have a medical problem and was transferred to another psychiatric outpatient clinic. An additional 15 cases were assigned, but contained conflicting or missing information from the three sources for the record review. These 50 cases were treated as missing, resulting in a final sample of 356 children.
Presenting Concerns Categories
Parents who contacted the intake office to request an appointment are asked
to provide a brief description of their concerns about their child, a summary
of their child's medical situation, and any additional relevant background.
Using this initial information provided by parents, the clinic co-director
categorized cases into three broad groups. Case categorization was based on
the initial referral concern, rather than concerns that may have emerged
during evaluation or treatment.
Parents who identified adjustment to chronic illness as the primary concern constituted a chronic illness group. Common presenting issues in this group included individual and family adjustment to diagnosis, adaptation to illness-related functional impairments, adherence to a medical regimen, and emotional disturbances related to chronic illness. Parents who identified pre- or postprocedural adjustment problems or general medical anxiety were categorized in a procedural group. This group included children facing routine well-care procedures (e.g., shots) and diagnostic procedures (e.g., a catheterization procedure to test for urological abnormalities), as well as children with chronic illness dealing with treatment procedures (e.g., a surgical procedure for a child with congenital heart disease). The last group, other concerns, included all other children referred for a myriad of concerns, including acute injuries, grief issues, and adjustment to sibling or parent physical illnesses.
The primary presenting concern was categorized by the first author (SDK)
for approximately 25% of the cases from the original sample (n = 107)
and compared to the categorization by the clinic codirector.
Interrater-reliability was calculated using the kappa coefficient and percent
agreement. Agreement between raters was excellent, according to Cicchetti and
Sparrow's (1981) standards,
with a kappa of.81 and percent agreement of.89.
Service Use and Referral Source
Service use information included date of intake, including whether or not a
child was actually seen for an initial evaluation appointment after it had
been scheduled. Length of services referred to the number of completed
sessions according to hospital administration records.
Referral source was first categorized as within the hospital or from an outside agency. Referrals from within the hospital were further categorized into four groups: (1) specialty pediatrics, consisting of referrals from pediatric medical or surgical specialists; (2) primary pediatrics, consisting of referrals from primary care pediatricians; (3) psychiatry, consisting of referrals from the inpatient pediatric psychiatry consultation program or other outpatient psychiatry services, and (4) support services, consisting of referrals from nonphysicians within the hospital (e.g., social workers, child life specialists, or family resource staff).
|
Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Baseline Characteristics
Table 1 presents child and service use characteristics, as well as referral source information for the 356 children who were referred to the Medical Coping Clinic. There were nine adults, ages 20 years and above. Slightly more than half the children were female, with ages for all the children ranging from 1 month to 19 years of age. The primary concern for approximately half of the children was categorized as chronic illness, followed by the procedural and other concerns groups.
|
About a quarter of the children referred for services were not seen for an initial evaluation interview. Information pertaining to reasons for not following through with a referral was not consistently documented or available. Because the hospital is a member of nearly all state insurance networks, there would be few patients who could not be seen due to lack of insurance coverage. Thus, the majority of cases not seen involved a parental decision not to follow through with a physician's recommendation. For those cases with documentation, among the reasons given were no interest, no perceived benefit, insufficient time or resources, and excessive distance from home.
Of those children who did receive services, 14% were in continuing treatment at the end of the review period. For those who had terminated treatment during the review period (n = 213), 50 cases had a single session only. The remaining cases (n = 163) ranged from 2 to 35 sessions, with 95% having 13 visits or less.
There was a relatively steady rate of referrals, as indicated by the date of the initial evaluation. Nearly 90% of the children were referred from sources within the hospital. Pediatric specialists referred 57% of the children from the hospital, representing 16 separate departments or divisions. Within the specialty pediatric group (n = 165), urology (26%) and cardiology (18%) made the highest number of referrals. The psychiatry group referred 25% of the children from the hospital. The primary pediatrics (13%) and support services (5%) groups made the least number of clinic referrals.
Factors Affecting Service Use and Referral Patterns
Child Seen for Services. Because there were so few cases referred
by the primary pediatric and support services groups, referring comparisons
were conducted using the specialty pediatric and psychiatry groups only.
Referrals from specialty pediatrics were less likely (69%) to be seen than
those from psychiatry (82%; 
2 [1, 237] = 4.20, p
<.05). There was no significant difference in likelihood of keeping an
appointment based on presenting concern category or time of referral.
Number of Sessions. Length of services was examined only for those
cases that had completed treatment during the study period (n = 213).
When divided into two groups, those with a single session and those with more
than one session, children referred for procedural concerns (34%) were more
likely to have a single session than children referred for chronic illness
(17%; 2 [1, 213] = 5.26, p <.05). There was no
significant difference between the other concerns group (24%) and the
procedural or chronic illness groups. A complete review of these cases
indicated that information regarding the reason for a single session was not
consistently documented. However, most of the children referred for procedural
concerns were seen for a single session prior to the procedure date, intended
as a one-time treatment consult. Documented reasons for a single session only
in the chronic illness and other concerns groups included not appropriate for
clinic services, transfer to another psychiatry service in the hospital or
community, parental decision not to follow through with services, or need for
treatment not indicated at the time. For this latter case, an example would be
a family in crisis during a child's acute hospitalization who had been
referred to the Medical Coping Clinic for continuing services on an outpatient
basis, which were not indicated after the acute stressors of hospitalization
subsided. There were no differences by referral source for like-lihood of a
single session only.
The average number of sessions was positively skewed and kurtotic due to the influence of several outliers, so only those cases below the 90th percentile (13.6 sessions) were used in analyses. For those 147 cases with 2 to 13 sessions, the mean was 5.1 sessions (SD = 2.8). There were no differences in the average number of sessions by presenting concern or by referral source.
Rate of Growth. The number of referrals over time did not
significantly differ by presenting concern and referral source. Although there
was no indication of a steady increase over time, the percentage of cases
referred for procedural concerns nearly doubled from the first quarter (17%)
to the last four quarters combined (32%), with a corresponding decrease in all
other concerns (chronic illness and other concerns included; 2
[1, 356] = 5.65, p <.05).
Referral Source. Referral source factors differed significantly by
the type of presenting concern for origination inside or outside of the
hospital (2 [2, 332] = 10.79, p <.01) and for
group type (2 [2, 237] = 24.27, p <.001). Post-hoc
analyses indicated that referrals for procedural concerns were less likely to
originate outside the hospital (5%) than referrals for other concerns (22%;
2 [1, 168] = 10.89, p <.005). There were no
referral source differences among chronic illness referrals (13%) and the
other two groups. A stronger relationship was found for presenting concern by
group type comparing specialty pediatric and psychiatry groups. Post-hoc
analyses indicated differences between all three types of presenting concern.
Compared to psychiatry, there were more specialty pediatrics referrals for
procedural concerns (87%) than for chronic illness (68%) or other concerns
(44%; 2 [1, 237] = 15.69, p <.001). In addition,
there were significantly more referrals from specialty pediatrics for chronic
illness than other concerns (2 [1, 161] = 7.66, p
<.005).
|
Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
This project is unique in its application of process evaluation, a component of mental health services research, to examine the initial implementation of a hospital-based clinic for children coping with medical illness. The implementation issues identified in this study have been used to shape the direction for the clinic by increasing the specialization of clinical services, building collaborative relationships with referring providers to improve service utilization, developing criteria for referral decision making, and integrating the Medical Coping Clinic services with hospital medical departments. Specific examples of current projects stemming from this evaluation are given below, followed by a discussion of the limitations of the evaluation and clinical and research implications.
The clinic was designed to offer preventive interventions and treatment to children and families facing medical stressors. Many children seen in the clinic did not meet DSM-IV criteria for psychiatric disorder but were instead experiencing milder emotional, behavioral, or developmental issues associated with their medical condition. A multi-disciplinary staff familiar with the issues affecting families facing physical illnesses provided interventions ranging from behavioral to psychoeducational to medication treatments. Due to constraints such as limited staff and budget, the central goal of clinic development was not growth in overall patient volume per se but the specialization of services according to identified clinical needs within the hospital.
Three issues were documented that could affect the viability of such a specialized clinic. First, the presenting concerns of within-hospital referrals, primarily for emotional and behavioral issues related to having a chronic physical illness or for help with medical procedures, suggest that an appropriate patient population was referred to and seen through the clinic. Second, the finding that the majority of referred cases were seen at least once also underscores the clinical need for such specialized services and the willingness of most parents to seek them. Third, although volume growth was not a goal for the Medical Coping Clinic, the issue of revenue generation is important to the ongoing viability of mental health services. Successful training and research in an academic institution often necessitate lower clinical volumes and, as a result, lower revenue. One potential solution to this dilemma is to contract directly with medical or surgical services to provide payments for behavioral health consultation. After experiencing the value of Medical Coping Clinic services, several hospital departments expressed interest in formalizing programmatic collaboration, and one department contracted for consultation services by funding a portion of a post-doctoral fellowship in psychology.
More than half of the hospital referrals came from pediatric specialists,
who were more likely to refer patients for procedural concerns than for
chronic illness or other concerns. Although patient growth remained fairly
steady over the review period, the rate of referrals for procedural concerns
nearly doubled after the first quarter. This information on referral patterns,
along with the recognition of the within-hospital need for procedural
preparations, was used to direct the allocation of clinic resources to the
development and implementation of responsive and accessible interventions. To
reach the broadest audience, a parent manual designed to help children cope
with medical procedures was developed (Van
Horn & DeMaso, 1998). The manual was used as a tool in
treatment, but was also made available to families not seen in the clinic. In
addition, a single session group and one-time individual consults were offered
to parents whose children were facing medical procedures. These interventions
were developed in collaboration with pediatric specialty staff, the primary
referral source within the hospital. A particular point was made to provide
the manuals and information about the parent group to the family resource
center staff, who were in the group that made the fewest referrals.
The results suggest that families were less likely to follow through with referrals made by pediatric specialists than for referrals made by mental health providers. Importantly, this lower rate of follow-through may be an underestimate, as parents who were referred to the clinic but did not contact the psychiatry intake office were not included in these analyses. All cases included in this evaluation had at a minimum contacted the intake office to schedule an appointment. Increased likelihood of follow-through with referrals from mental health providers may have been related to previous or current beneficial experiences with other mental health services. Mental health referrers may have also more accurately identified patients' current psychosocial needs, as compared to medical providers and support staff.
To gain further information on actual referrals and referral decision making, a departmental case study has been added, which uses a clinic staff member as a direct liaison to the urology department, the pediatric specialty department with the most referrals. Qualitative interviews have focused on referral decision making by medical providers and staff. A brief, readily administered measure was designed to screen for risk of medical distress and identify children who might benefit from preventive interventions prior to procedures. The clinical utility of the measure, including its usefulness in making decisions about referrals, is being examined as part of a collaborative quality improvement initiative with referring providers. In addition, the feasibility of implementing a streamlined referral management system, which would provide information on all referrals regardless of whether the parent contacts the Medical Coping Clinic, is being examined.
The lowest rate of referrals to the Medical Coping Clinic came from primary pediatric and support departments. This may indicate a lower need for services, given that primary pediatric departments, particularly those in a large tertiary care center, may be less likely to encounter children with chronic health conditions. However, the low rate of referrals may also represent a lack of broad integration of services throughout the hospital. Modified focus groups are being planned, consisting of structured in-service training sessions by clinic staff and serving two potential goals. First, the meetings could provide an opportunity to collect data pertaining to referral decision making and perceptions of need for clinic services. Second, they could increase knowledge of clinic services, which may be a factor in the low referral rates.
Limitations
Reliance on record review is a significant limitation, given that records
are often missing or incomplete. For this study, there was a 12% rate of
missing or incomplete records from the compilation of the three different data
sources. Important variables (e.g., medical diagnosis, reason for termination)
were not available in the records reviewed. However, given that most programs
keep records, their accessibility increases the feasibility of conducting
descriptive evaluations, given the limited resources typical of "real
world" clinical settings (National
Advisory Mental Health Council, 1999). Furthermore, program
records provide continuous information over long periods, particularly well
suited to the purposes of a process evaluation, as opposed to one-time studies
(Weiss, 1998). Conducting a
process evaluation using existing records can also assist in clarifying
documentation needs and improving record keeping in the future. For example,
in this evaluation, information regarding lack of follow-through with a
referral or reason for a single session was often not available. Current
efforts are under way to expand systematic documentation regarding referral
information and service utilization for use in future evaluations.
A second limitation is the lack of involvement of all relevant
stakeholders, broadly defined as individuals affected by the program. For this
clinic, the group of stakeholders would include clinic staff, hospital
administrators, hospital medical providers, support staff, and children and
their families. Involvement of stakeholders in all stages of an evaluation,
from development of research questions to dissemination of results, has been
suggested as a valuable way to increase the validity and utilization of an
evaluation (Kelley et al.,
2000). For example, in this study, child and family perspectives
were missing. It will be important to include them in further evaluations,
particularly regarding follow-through with referrals. Although this study also
did not include medical providers and department staff in the initial record
review, the ongoing process evaluation includes active collaboration with
various hospital staff.
Clinical and Research Implications
The ongoing process evaluation is being used to provide an empirical basis
(e.g., Jensen et al., 1996) to
identify and refine measurement needs for intermediate outcomes (e.g., more
specific information on referral sources) and potential mental health outcomes
(e.g., child's functioning status) for further research on clinic process and
effectiveness. The goal of this study was to provide an example of the
application of a process evaluation approach to investigating the development
and implementation of a mental health clinic in a pediatric setting. Although
such factors will be unique to each clinic, it is important to understand
their influence on access to, provision of, and effectiveness of mental health
interventions in pediatric settings
(Kelleher, 1999). Aside from
those presented here, other important process variables would include
information on interventions used, premature terminations versus treatment
completion, linking length of services to individual and system variables, and
others. Such implementation monitoring can serve as a strategy to strengthen
internal validity (e.g., Bickman,
1992) of future outcomes evaluation.
Incorporating a mental health services research agenda in pediatric settings can be challenging, for by definition a clinical program's primary goal is the provision of services. In an academic environment, this hospital-based clinic established the evaluation of their clinical interventions as an equally important second goal. In addition, the clinic had a mandated requirement to respond to the hospital's quality improvement directives. As is occurring nationally, these directives are increasingly requiring programs to go beyond the mere provision of clinical service to studies that essentially promote intervention and outcome research. Thus, both due to its own academic mission as well as the need to respond to an external mandate, the Department of Psychiatry has supported the development of a research agenda in the clinic. The clinic directors identified the need for a postdoctoral fellow with dedicated nonclinical time to assist them in the development and implementation of the clinic's field projects. In addition, volunteer graduate research assistants have been used to collate data and enter them into computerized data sets.
Our findings suggest that the use of process evaluation as an intervention
research tool in pediatric settings is both feasible and informative in
guiding the strategic development of a clinical program, while operating
within reality-driven resource constraints. Several of the barriers to
conducting studies of "real world" interventions in pediatric
settings (e.g., Drotar, 1997;
Kelleher, 1999;
Kelleher & Long, 1994) can
be addressed by using a process evaluation approach. Furthermore, the findings
are representative of the types of system-level factors important for any
program in developing responsive clinical and research agendas. The utility of
process evaluation as a research tool has been acknowledged in other fields,
such as health care (Brownson et al.,
1997), teenage pregnancy risk-reduction programs
(Devaney & Rossi, 1997),
and child and adolescent mental health services
(Kelley et al., 2000).
By no means should process evaluation be taken as a substitute for outcomes or effectiveness research. Instead, it should be viewed as an integral part of the comprehensive evaluation of a complex system. Clinical interventions in pediatric settings are dynamic entities, having to adapt continuously to meet changing needs and conditions. Mental health services research must be flexible and responsive to clinical needs yet also maintain scientific integrity.
|
Acknowledgments |
|---|
Dr. Kelley is supported by a W. T. Grant Fellowship. The data presented, the statements made, and the views expressed are solely the responsibility of the authors. We thank Elisabeth Morray for volunteering many hours of her time for data entry.
Received July 31, 2000; revision received December 1, 2000; accepted January 15, 2001
|
References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
American Academy of Pediatrics Committees on Children with Disabilities and on Psychosocial Aspects of Child and Family Health. (1993). Psychosocial risks of chronic health conditions in childhood and adolescents (RE9338). Pediatrics, 92, 876-878.
Barnes, J., Stein, A., & Rosenberg, W. (1999).
Evidence based medicine and evaluation of mental health services:
Methodological issues and future direction. Archives of Disease in
Childhood, 80,
280-285.
Bickman, L. (1992). Designing outcome evaluations for children's mental health services: Improving internal validity. New directions for program evaluation: Vol. 54 (pp. 57-68). San Francisco: Jossey-Bass.
Brownson, R. C., Mayer, J. P., Dusseault, P. M., Dabney, S., Wright, K. S., Jackson-Thompson, J., Malone, B. R., & Goodman, R. M. (1997). Developing and evaluating a cardiovascular risk reduction project. American Journal of Health Behavior, 21, 333-344.[Web of Science]
Bush, J. P., Melamed, B. G., Sheras, P. L., & Greenbaum, P. E. (1986). Mother-child patterns of coping with anticipatory medical stress. Health Psychology, 5, 137-157.[Web of Science][Medline]
Cicchetti, D. V., & Sparrow, S. S. (1981). Developing criteria for establishing the interrater reliability of specific items in a given inventory. American Journal of Mental Deficiency, 86, 127-137.[Web of Science][Medline]
Costello, E. J., Burns, B. J., Costello, A. J., Edelbrock, C.,
Dulcan, M., & Brent, D. (1988). Service utilization and
psychiatric diagnosis in pediatric primary care: The role of the gatekeeper.
Pediatrics, 82,
435-441.
Devaney, B., & Rossi, P. (1997). Thinking through evaluation design options. Children and Youth Services Review, 19, 587-606.
Drotar, D. (1997). Intervention research: Pushing back
the frontiers of pediatric psychology. Journal of Pediatric
Psychology, 22,
593-606.
Dulcan, M. K., Costello, E. J., Costello, A. J., Edelbrock, C., Brent, D., & Janiszewski, S. (1990). The pediatrician as gatekeeper to mental health care for children: Do parents' concerns open the gate? Journal of the American Academy of Child and Adolescent Psychiatry, 29, 453-458.[Web of Science][Medline]
Jensen, P. S., Irwin, R. A. C., Josephson, A. M., Davis, H., Xenakis, S. N., Bloedau, L., Ness, R., Mabe, A., Lee, B., Traylor, J., & Clawson, L. (1996). Data-gathering tools for "real world" clinical settings: A multisite feasibility study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 55-66.[Web of Science][Medline]
Kelleher, K. J. (1999). Commentary: Pediatric
psychologist as investigator in primary care. Journal of Pediatric
Psychology, 24,
459-462.
Kelleher, K., & Long, N. (1994). Barriers and new directions in mental health services research in primary care settings. Journal of Clinical Child Psychology, 23, 133-142.
Kelley, S. D., Nixon, C. T., & Bickman, L. (2000). Evaluating mental health services for children and adolescents. In D. Drotar (Ed.), Handbook of research in pediatrics and clinical child psychology: Practical strategies and methods. Issues in clinical child psychology (pp. 463-489). New York: Kluwer Academic/Plenum.
La Greca, A. M., & Varni, J. W. (1993).
Interventions in pediatric psychology: A look toward the future.
Journal of Pediatric Psychology,
18, 667-680.
National Advisory Mental Health Council Clinical Treatment and Services Research Workgroup. (1999). Bridging science and service (NIH 99-4353). Bethesda, MD: National Institute of Mental Health.
Owen, J. M., & Rogers, P. J. (1999). Monitoring evaluation. In, Program evaluation: Forms and approaches (pp. 239-262). Thousand Oaks, CA: Sage.
Van Horn, M., & DeMaso, D. R. (1998, Fall). Helping your child with medical experiences: A practical parent guide. (Available from Children's Hospital, Coping Clinic, Fegan 8, 300 Longwood Ave., Boston, MA, 02115)
Wallander, J., & Thompson, R. (1995). Psychosocial adjustment of children with chronic physical conditions. In M. Roberts (Ed.), Handbook of pediatric psychology (pp. 124-141). New York: Guilford Press.
Weiss, C. (1998). Collecting data. In, Evaluation: Methods for studying programs and policies (2nd ed., pp. 152-179). Upper Saddle River, NJ: Prentice Hall.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||