Journal of Pediatric Psychology, Vol. 26, No. 1, 2001, pp. 55-60
© 2001 Society of Pediatric Psychology
Brief Report: Psychometric Evaluation of the Severity of Illness Scale in a Pediatric Oncology Sample
1 Columbus Children's Hospital, 2 Florida State University
All correspondence should be sent to Frances Prevatt, Department of Human Services, Stone 215, Florida State University, Tallahassee, Florida 32306-4458. E-mail: Fprevatt{at}coe.fsu.edu .
 |
Abstract |
|---|
 Top Abstract IntroductionMethodResultsDiscussionAppendixReferences |
|---|
Objective: To examine the psychometric properties of the Severity of Illness Scale (SOIS), a measure that focuses on the medical severity of illness of children with cancer, from the point of view of medical personnel.
Methods: Following pretesting, the SOIS was administered to nurses and physicians of 55 pediatric cancer patients at three time periods: entry into study, 2-week follow-up, and 3-month follow-up. Validity determination included analyses of relapse status and bone marrow transplant. Test utility was determined via a respondent questionnaire.
Results: Test-retest reliability coefficients were.96 and.92 for 2-week and 3-month time periods. Interrater reliability, assessed by comparing physician ratings to nurse ratings, was.89. Evidence for criterion-related validity revealed that the SOIS discriminates both bone marrow transplant and relapse status. Physicians and nurses rated the SOIS positively for brevity, ease of completion, and usefulness in depicting medical severity of disease.
Conclusions: There is preliminary evidence for the psychometric utility of the Severity of Illness Scale for a pediatric cancer population. The inclusion of illness parameters in current models of risk and resiliency dictate the need for such a measure.
Key words: severity of illness; childhood cancer; assessment.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
Many studies of children's adaptation to cancer have utilized a risk and resiliency model. For example, Wallander, Varni, Babani, Banis, and Wilcox (1989
) hypothesize that risk
and resistance factors mediate the relationship between disease and
adaptation. One of their risk factors, severity of illness, allegedly has both
indirect and direct effects on a child's adjustment. The literature specific
to childhood cancer has not clearly defined the relationship between
psychosocial adjustment and severity of disease
(Dolgin, Phipps, Harrow, & Zeltzer,
1990; Kupst et al.,
1995; Lemanek,
1994; Worchel, Copeland, &
Barker, 1987; Zeltzer,
1993). Diverse findings regarding the relationship between disease
severity and adjustment partially reflects the lack of a consistent definition
and measure of illness severity.
Illness severity is a multidimensional concept, including such domains as
physiological factors, disruption to the physical development of the child,
visibility of the condition, and illness course and prognosis
(Bradford, 1994;
Lemanek, 1994). Illness
severity may well mean different things in different conditions
(Bradford, 1994). For example,
some children (e.g., those in remission) may have few daily symptoms to
contend with but still have a poor prognosis due to the stage of their cancer.
A child who is receiving intensive treatment may suffer from daily disease
symptoms (nausea, losing hair, weight loss or gain) yet still have a good
long-term prognosis.
Researchers have used a variety of methods to measure cancer severity,
including months since last treatment, total time in treatment, medical status
(e.g., remission, relapse, death), a one-question global assessment of
severity, illness course (stable, labile, deteriorating), prognosis (good,
guarded, poor), degree of physical impairment (none, mild, moderate, severe),
and visibility of physical residua (none, mild, moderate, severe)
(Butler, Rizzi, & Handwerger,
1996; Dolgin et al.,
1990; Kellerman, Zeltzer,
Ellenberg, Dash, & Rigler, 1980;
Kupst et al., 1995;
Sanger, Copeland, & Davidson,
1991; Susmam et al.,
1982; Worchel et al.,
1987). A small number of standardized measures, such as the
Therapeutic Intervention Scoring System or the Physiologic Stability Index,
focus on physiological symptoms and are designed only for acutely ill
inpatients (Sivan, Schwartz, Schonfeld,
Cohen, & Newth, 1990).
Other investigators have measured health-related quality of life (HRQOL),
rather than severity of illness. HRQOL is conceptualized as a patient's
perception of the impact of disease and treatment functioning in the physical,
mental, and social domains (Varni, Seid,
& Rode, 1999). A review of HRQOL measures by Speith and Harris
(1996) evaluated six
instruments, all completed by the parents or child patient. HRQOL differs from
illness severity in three important ways. First, severity of illness focuses
primarily on disease status and functional impairment, while HRQOL measures
include psychological functioning. Second, severity of illness is more
appropriately rated by medical personnel, whereas HRQOL involves the
perceptions and observations of patients and their caretakers. Third, HRQOL
measures typically do not predict prognosis
(Kaplan & Anderson,
1990).
There have been numerous well-documented measures of HRQOL. Lacking in the
pediatric oncology literature is a standard measure for determining illness
severity related to medical functioning. Because illness severity is included
in theoretical models of illness adaptation, a measure is needed that can be
used in empirical evaluations of these models. This article presents
information on the development of a measure evaluating medical severity of
illness in children diagnosed with cancer. The Severity of Illness Scale
(SOIS; Worchel & Rae,
1990) is a six-item Likert format scale yielding an overall score
for severity of illness. The SOIS is completed by medical personnel. The
article addresses the following areas regarding the psychometric properties of
the SOIS: (1) test development, (2) reliability (internal, test-retest,
inter-rater), (3) concurrent validity, and (4) test utility.
|
Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
Test Development
Initial item selection was based on theoretical areas suggested by research, combined with clinical input from a panel of three pediatric psychologists, two pediatric oncologists, and two pediatric nurse clinicians. The following areas were identified: degree of impairment, future outlook, quantity of medical procedures required, number of hospitalizations, ability to participate in activities, and prognosis. It was anticipated that these items would yield a unidimensional measure of illness severity; therefore, items were summed to create a single total score, with a higher score indicating greater impairment. The resulting 7-item scale was piloted on a sample of 19 children with cancer who were enrolled in a research project on depressive symptomatology. (A description of the sample can be found in Worchel, Rae, Olson, & Crowley, 1992.) Based on feedback from physicians who completed the scale, one item dealing with prognosis was dropped and several were reworded. The resulting 6-item scale (see appendix) was readministered to physicians and nurses in this study.
Participants
The sample consisted of 55 pediatric oncology patients from a large
regional children's hospital. Children were inpatients or outpatients
undergoing treatment for their disease. Mean age of the sample was 8
years (SD = 4.9; range = 4-19); 87% of the sample were Caucasian and
38% were female. Primary diagnosis was ALL (51%), followed by brain tumor (9%)
and lymphomas (7%). Twenty (36%) of the subjects had suffered a relapse and 14
(25%) had undergone bone marrow transplants. The average length of time since
diagnosis was 20 months (SD = 18.4; range = 3-104 months). The
average number of hospitalizations was 5.09 (SD = 4.53; range =
0-20). During the course of the study three participants died and one moved
away; bringing the final sample size to 51.
Procedure
All participants over a 3-month period who met the following selection
criteria were selected for the study: between the ages of 4 and 19 years,
followed by both a primary physician and a nurse clinician, and in active
treatment or in remission and still being followed in the Hematology/Oncology
Clinic. No subjects were excluded who met this criteria. Because this was a
teaching/research hospital, consent was obtained for specified procedures
(e.g., chart review, questionnaire administration, noninvasive research
projects) upon admission. Because patients were not personally involved in any
way, additional consent was not required of patients or their parents. The
project received IRB approval from both the hospital and the university of the
authors affiliated with the project. Raters (e.g., physicians and nurses) who
participated in the project completed informed consent documents. Raters were
told that the purpose of the ratings was to evaluate the psychometric
properties of a scale measuring medical severity of illness. The SOIS was
completed independently by the participant's primary physician and primary
nurse clinician on three separate occasions: at entry into the study, 2 weeks
later, and 3 months later. Raters included five physicians and three nurse
clinicians, and each patient was treated as an independent observation. The
participants' medical charts were reviewed by a research assistant on the same
three occasions for ethnicity and age of child, date of diagnosis, date of
remission and relapse if appropriate, number of and reason for
hospitalizations in the last year, phase and course of treatment, and
treatment protocol. The items on the SOIS at the 2-week retest were reordered
to rule out the possibility of practice effects. Following the 3-month SOIS
administration, raters completed a 9-item questionnaire regarding length of
time required to complete the SOIS, ease of completing the SOIS, and
usefulness in measuring severity of illness.
|
Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
Reliability
For internal consistency reliability, data were analyzed separately for physicians and nurse clinicians. Within each group, data were collapsed across times 1, 2, and 3. Cronbach's alpha was used to calculate internal consistency. Total score alpha was.80 for the nurses and.79 for the physicians. For test-retest reliability, data were collapsed across raters, and Pearson product-moment correlations were used. The correlations were as follows: Time 1-Time 2, r =.96; Time 2-Time 3, r =.89; Time 1-Time 3, r =.92. Paired differences were also computed as a means of assessing whether a genuine change in illness severity may have occurred between the time intervals. Changes over time were clinically insignificant, with mean scores as follows: Time 1 (M = 22.13, SD = 7.26), Time 2 (M = 22.84, SD = 7.49), and Time 3 (M = 22.79, SD = 7.71).
For interrater reliability, data were collapsed across time, and Pearson product-moment correlations were used to analyze the relationship between ratings by physicians and nurses. The correlation between nurse and physician ratings was.89. The difference between mean ratings of the nurses (M = 24.96, SD = 7.85) and the physicians (M = 22.12, SD = 7.13) was significant, t(50) = -5.56, p <.001. Nurses' mean ratings were higher, indicating more severity.
Validity
Two criterion measures were utilized: bone marrow transplant (generally
considered as a treatment in more severe cases) and whether or not the child
had relapsed. Bone marrow transplant and relapse status are clearly
overlapping and highly correlated variables (all children with BMT had also
relapsed); however, due to the limited number of variables available that were
not already being measured on the SOIS, both were utilized. Data were
collapsed across time and computed separately for nurses and physicians, using
analysis of variance (ANOVA). Children who underwent bone marrow transplant
received significantly higher SOIS total scores than children who did not.
This held for both the nurse (M = 22.98 vs. 28.59) and physician
ratings (M = 20.71 vs. 24.73). In addition, children who had relapsed
received significantly higher SOIS total scores than children without relapse.
This also held for both nurse (M = 24.16 vs. 28.23) and physician
ratings (M = 21.39 vs. 25.10).
Test Utility
There was 100% agreement by the physicians and nurses that the SOIS could
be completed in less than 5 minutes. Mean rating for ease of completion (1 =
not at all difficult to 5 = very difficult) across raters was M =
1.0, SD = 0. Mean rating of usefulness in measuring severity of
illness (1 = not at all useful to 5 = very useful) across raters was
M = 4.87.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
This study evaluated the validation of a severity of illness measure for children with cancer. Although severity of illness is important in current theoretical models of children's adjustment to cancer, no validated tool has been consistently used to measure this construct. Internal consistency, test-retest, and interrater reliabilities were all acceptable. Test-retest reliabilities were high over 2-week and 3-month intervals. Although test-retest analyses over a longer period would be desirable, this is not practical given the nature of childhood cancer, as test properties and changes in disease status are confounded.
Although high interrater reliability was found (r =. 89), the mean ratings of the nurses on the SOIS total score were higher than those of the physicians, indicating that nurses rated the illness severity for the children as more severe. This may in part be due to the differences in knowledge that each has about the particular cases. The nurse practitioners often see the children more frequently in the outpatient clinics than do the primary physicians and may have more information about the child's day-to-day functioning. Alternatively, the lack of clear criteria on some of the anchors might make scoring too subjective. Although the mean ratings were statistically different, (M = 24.96 vs. 22.12), the practical significance of these differences may be quite small. However, when used for research purposes, it may be advisable to ensure that ratings be obtained from a consistent source.
Analyses lent support for the concurrent validity of the SOIS. Since there are no other validated measures of severity of illness useful across the range of medical status, constructs hypothesized to measure severity, yet which were not directly measured by the scale, were chosen. We determined that SOIS total scores could discriminate children who had relapsed, as well as those who had undergone bone marrow transplant.
Overall, the SOIS appears to demonstrate many of the characteristics
suggested by Mulhern et al.
(1989) for use with pediatric
cancer patients. It is brief, simple, easy to administer, and repeatable. It
assesses two important medical constructs, prognosis and impairment in
functioning. Suggestions for future research include the need to evaluate
predictive validity of the scale. For example, a useful predictive criterion
could be 3- and 5-year remission status. Additionally, comparison to HRQOL
measures would be useful in determining whether these two scales reflect
similar underlying constructs. Additional samples should include larger groups
of patients with diagnoses more proportional to the incidence of larger groups
of tumors. Finally, more descriptive anchors might increase the objectivity of
the ratings. In particular, the anchors for the item evaluating
age-appropriate social activities might be reworded to better clarify
developmental differences. Although additional psychometric work is needed, we
hope that the SOIS might be utilized as a research tool in evaluating risk and
resiliency models of children's adaptation to childhood cancer.
|
Appendix |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
|
Received April 1, 1999; revision received June 28, 1999; accepted November 30, 1999
|
References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionAppendixReferences |
|---|
Bradford, R. (1994). Children with liver disease: Maternal reports of their adjustment and the influence of disease severity on outcomes. Child Care, Health and Development, 20, 393-407.[ISI][Medline]
Butler, R. W., Rizzi, L. P., & Handwerger, B. A.
(1996). Brief report: The assessment of post traumatic stress
disorder in pediatric cancer patients and survivors. Journal of
Pediatric Psychology, 21,
499-504.
Dolgin, M. J., Phipps, S., Harrow, E., & Zeltzer, L. K.
(1990). Parental management of fear in chronically ill and
healthy children. Journal of Pediatric Psychology,
15, 733-744.
Kaplan, R. M., & Anderson, J. P. (1990). The general health policy model: An integrated approach. In B. Spiker (Ed.), Quality of life in clinical trials. New York: Raven.
Kellerman, J., Zeltzer, L., Ellenberg, L., Dash, D., & Rigler, D. (1980). Psychological effects of illness in adolescents: Anxiety, self-esteem, and perception of control. Journal of Pediatrics, 97, 126-131.[ISI][Medline]
Kupst, M. J., Natta, M. B., Richardson, C. C., Schulman, J. L.,
Lavigne, J. V., & Das, L. (1995). Family coping with
pediatric leukemia: Ten years after treatment. Journal of Pediatric
Psychology, 20,
601-617.
Lemanek, K. (1994). Editorial: Research on pediatric
chronic illness: New directions and recurrent confounds. Journal of
Pediatric Psychology, 19,
143-148.
Mulhern, R. K., Ochs, J., Armstrong, D., Horowitz, M. E., Friedman, A. G., Copeland, D., & Kun, L. (1989). Assessment of quality of life among pediatric patients with cancer. Psychological Assessment, 1, 130-138.
Sanger, M. S., Copeland, D. R., & Davidson, E. R.
(1991). Psychosocial adjustment among pediatric cancer patients:
A multidimensional assessment. Journal of Pediatric
Psychology, 16,
463-474.
Sivan, Y., Schwartz, P. H., Schonfeld, T., Cohen, I. J., & Newth, C. J. (1990). Outcome of oncology patients in the pediatric intensive care unit. Intensive Medical Care, 17, 11-15.
Speith, L. E., & Harris, C. V. (1996). Assessment
of health-related quality of life in children and adolescents: An integrative
review. Journal of Pediatric Psychology,
21, 175-193.
Susman, E. J., Hersh, S. P., Nannis, E. D., Strope, B. E.,
Woodruff, P.J., Pizzo, P. A., & Levine A. S. (1982).
Conceptions of cancer: The perspectives of child and adolescent patients and
their families. Journal of Pediatric Psychology,
7, 253-261.
Varni, J. W., Seid, M., & Rode, C. A. (1999). The PedsQL (TM): Measurement model for the pediatric quality of life inventory. Medical Care, 37, 126-139.[ISI][Medline]
Wallander, J. L., Varni, J. W., Babani, L., Banis, H. T., &
Wilcox, K. T. (1989). Family resources as resistance factors for
psychological maladjustment in chronically ill and handicapped children.
Journal of Pediatric Psychology,
14, 157-173.
Worchel, F. F., Copeland, D. R., & Barker, D.
(1987). Control related coping strategies in pediatric oncology
patients. Journal of Pediatric Psychology,
12, 25-38.
Worchel, F. F., & Rae, W. (1990). Severity of Illness Scale. Unpublished manuscript, Texas A&M University (Educational Psychology Department, College Station, TX).
Worchel, F., Rae, F., Olsen, T., & Crowley, S. (1992). Selective responsiveness of chronically ill children to assessments of depression. Journal of Personality Assessment, 59, 605-615.[ISI][Medline]
Zeltzer, L. K. (1993). Cancer in adolescents and young adults: Psychosocial aspects. Cancer, 71, 3463-3468.[ISI][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||