| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
To purchase reprints, contact The InnoVision Group, 101 Columbia, Aliso Viejo, CA 92656. Phone, (800) 809-2273 or (949) 362-2050 (ext 532); fax, (949) 362-2049; e-mail, reprints{at}aacn.org
EVALUATING CRITICAL CARE OUTCOMES
Critical care teams recognize that a patient’s status at discharge is only a preliminary measure of success in therapeutic outcomes. Short-term measures include changes in clinical status and amelioration of signs and symptoms. Long-term measures take into account the effects of hospitalization in critical care units on patients’ future quality of life.5,6 Assessment on the day of discharge measures the short-term impact of the critical care experience, but a longer period of assessment is required to judge lasting and more meaningful clinical outcomes. To be confident that clinical protocols have led to desired outcomes of morbidity and function, researchers must assess patients after the patients return to the community.4,6 Follow-up is especially important when researchers study new approaches to clinical problems. For example, animal-assisted therapy in cardiac care units7 might achieve wider acceptance if use of a health-related quality-of-life measure, such as SF-36, indicated the long-term benefits of the therapy.
A person’s perception of his or her health is a unique and key factor in explaining and predicting outcomes.8,9 Sobel’s call for rethinking medicine8 and the secondary analysis of data from the National Health and Nutrition Examination Survey (NHANES),9 indicate that laboratory reports and medical and nursing assessments are not sufficient to be predictive of morbidity, mortality, or even ability to function (eg, activities of daily living and social role accomplishments). A standardized tool that measures a patient’s health perceptions and permits valid and reliable comparisons across intervention groups is essential to understand change across studies10 and to compare treatments in clinical trials. Use of well-known and widely respected tools provides credibility that is valuable in situations in which controlled, randomized clinical trials are not possible. Critical care nurses as research partners can advocate for including measures of health-related quality of life in critical care research.
The SF-36 measures patients’ perceived health and is used in a variety of healthcare settings.11 My purpose in this article is to provide sufficient background information and tips on the use of the SF-36 for nurses to become knowledgeable about its use in healthcare studies.
WHAT THE SF-36 COVERS
Patients’ perceptions of their health-related quality of life help nurses evaluate the effectiveness of nursing care and innovations in care. The SF-36 developed by Ware and colleagues11,12 (often referred to as the Short Form 36) is increasingly the gold standard for studies of effectiveness. The tool is popular for several reasons. It is available for free to researchers. The SF-36 has been translated and used in many countries. National norms by age and sex of respondent are available. The original work was done at the prestigious Rand Corporation in Santa Monica, Calif. Over the years, the research validating the quality of the tool and the wide range of its applications has multiplied. Pertinent scholarly articles are readily accessed through the Cumulative Index to Nursing and Allied Health (CINAHL), PubMed, and MEDLINE by using "SF-36" and "SF36" as key words. An interactive Web site provides current information. (The Web site at http://www.sf-36.com is sponsored by QualityMetric Inc, a company owned by John E. Ware, the developer and copyright owner of SF-36. Through the QualityMetric Inc Web site at http://www.qualitymetric.com, Ware provides industry-leading support for the use of the SF-36 [and related outcome assessment tools] and provides patented computer software technology that improves the functionality of the SF-36. I have no affiliation with Ware or QualityMetric Inc. Although other companies provide SF-36 products and services, as far as I know, QualityMetric is the industry leader and the only such company offering advantageous services over the Internet as discussed in this article.)
The SF-36 includes 1 transition question and 35 questions on quality of life. The transition question asks patients to rate the amount of general health change they have experienced during the past year. The remaining 35 questions are organized into 8 sub-scales. The survey addresses limitations in physical functions and role activities due to health problems, bodily pain, general health perceptions, vitality (energy and fatigue), social limitations as a consequence of physical or emotional concerns, limitations in role activity due to emotional problems, and mental health. The 8 scales included in the SF-36 fall into 2 categories: the Physical Component Summary (Table 1
) and the Mental Component Summary (Table 2). These designations allow simpler analysis in monitoring disease groups over time and in making comparisons with the normative general US population.12 Scales included in the Physical Component Summary are Physical Functioning, Role-Physical, Bodily Pain, and General Health; scales included in the Mental Component Summary are Vitality, Social Functioning, Role-Emotional, and Mental Health. The scales comprise health-related concerns that are rated by the patient and together provide the variables used to produce the final scores. (Detailed manuals with scoring protocols, norms, and software for transforming scores and a scoring service with a base price and nominal per-case fee are available at http://www.qualitymetric.com.)
|
|
SELECTING THE VERSION OF THE SF-36
The research team must agree on questions for research and determine that a measure of health-related quality of life provides a satisfactory assessment of outcomes from the patients’ perspective. Critical care nurses can contribute to study quality early on by ensuring that the study team selects SF-36, Version 2, known as the international version.12 Version 2 can be downloaded without charge from the QualityMetric Inc Web site (http://www.sf-36.com) or can be found in the scoring manual.14 (If the 2000 manual is used, there is an error on page 107 of Appendix A. Question 2 should have a time period of "1 year ago." Change the question and the response options that use "weeks.") The response choices in Version 2 are more sensitive to detection of changes in health status and use words that translate more easily than the choices in the initial version do. This version evolved as the International Quality of Life Assessment project team worked with 14 countries to establish translations14 that meet reliability and validity standards for quality instruments.15 The entire November 1998 issue of the Journal of Clinical Epidemiology (vol 51, No. 11) details the translation project. This work has been extended to 45 countries.12 The Health Assessment Laboratory has beta versions of translations for ethnic groups living in the United States who are most comfortable using the groups’ native languages. The psychometrics for these translations have not yet met criteria for full approval,15 but the literature is emerging, especially for the Mandarin and Spanish versions for the United States.12,16–19 Investigator-initiated translations have been reported, but are inappropriate for use because of the rigor and scope of work of the International Quality of Life Assessment team. Shorter versions of the tool, SF-12 and SF-8, are available, but these are suited only to population studies because they lack the precision required for making clinical decisions about individual patients (see the following Web sites: http://www.qualitymetric.com and http://www.sf-36.com).
WHAT THE SF-36 DOES NOT COVER
Critical care nurses will want to know what the SF-36 does not include and make recommendations accordingly. For example, the tool does not target signs and symptoms related to sleeping patterns, memory, concentration, substance abuse, hearing, vision, incontinence, and dizziness.20 A review of the questions makes clear that the authors assumed that compliance with clinical regimens, health promotion habits, housing and employment situations, access to healthcare resources, and social networks can be assessed generically in terms of the impact on functional status. Nurses may be aware that patients face challenges in following protocols to manage some conditions after discharge. Failure to adhere to the postdischarge protocol may compromise a patient’s clinical status and lead to underestimation of the impact of an intensive care intervention or treatment. Nurses can request that data be collected specific to these items of concern. By proposing additional hypotheses or research questions, nurse partners can enrich the study.
A different but related critique concerns the complexity of items that require respondents to combine perceptions of physical and emotional problems that relate to relationships with "family, friends, neighbors, and groups."14 Usually, physical and emotional responses are rated separately. The item is effective for assessing persons who complete the survey in English; however, it is less successful in other languages.17 One explanation might be that cultural groups differ from one another in willingness to lump relationships to family with their perceptions of their relationships to friends, neighbors, and groups.17
CLINICAL AND SCHOLARLY RESOURCES
As a study evolves, staff nurses may contribute to building a rigorous study by using their insights to ensure that key variables are included. For example, in a study of whether pet therapy will improve clinical outcomes, experienced staff nurses can report that quality and quantity of sleep, family tension, and other signs or symptoms particular to a diagnosis might affect results but would not be included in SF-36 measurements. Equally important is awareness of organizational situations such as staff shortages, turnover of residents and fellows, preparation for inspection by the Joint Committee on Accreditation of Healthcare Organizations, and other variables that could account for a failure to show desired results. Nurses can appreciate the potential effects of these variables in a particular setting. They can help investigators decide whether someone should simply monitor and keep track of data on these variables, or whether the variables are so powerful that their presence must be accounted for in a systematic way. For example, sleep-deprived patients could be excluded from a study so that their fatigue would not affect the accuracy of the study’s results. Nurses might want to add in the family members as subjects to study the impact of the critical care experience on the health of those family members.
Staff nurses can help to decide whether the study should take place on all shifts or be restricted to a single shift. Staff nurses can facilitate research by sharing their knowledge of critical care patients and by recommending the use of other data instruments or SF-36 adaptations to obtain a complete picture.
Reviewing studies that report SF-36 data as outcome measures can augment clinical insight and experience. Such a review could provide nurses with a context to predict findings, select related indicators, and alert the team to problems experienced by other investigators. Many articles relevant to the SF-36 in the critical care environment have been published. Examples include an Australian article on the theoretical basis of quality-of-life measures and a review of findings from recent studies,4 a study of patients awaiting coronary artery bypass,21 an assessment of the health status of patients in the intensive care unit (ICU),22 a study that used the SF-36 to look at mortality advantages of preserving left ventricular function and health-related quality of life,23 a study to establish feasibility of high-risk interventions in the elderly (simultaneous bilateral renal revascularization),24 a study of long-term quality of life in survivors of sepsis,25 and British studies comparing survival and morbidity with quality of life after stays in the ICU.6 The nursing literature on use of the SF-36 in critical care is limited. A search turned up only 2 articles with nurses as the first author. One article reviews other studies that assessed quality of life in critical care units,4 and the other focuses on revascularization.24 It is beyond the scope of this article to review the specifics of the literature. Overall, support for using quality-of-life measures—and the SF-36 in particular—in the critical care unit is provided by valid and reliable evidence produced by such instruments that indicates that quality of life is poorer for patients after their stay in the ICU than it was before their stay.4
DATA ANALYSIS PLANNING
Data analysis methods for the SF-36 are best determined before data are collected. Mapping the analysis often reveals a lack of clarity in the research questions. One approach is to develop samples of the tables that one would like to include in the final report. These "shells" lack data but indicate all of the key variables and the variables’ relationships to one another. Equally important is determining who will analyze the data. Will the study team use an on-site statistician or hire one on contract? In either instance, the statistician should be involved as early as possible. A key role of the statistician is to make recommendations about how to format questionnaires and data records in a way that will facilitate speedy and accurate data entry and analysis. Failure to have a statistician review the tools that the data collectors will use can severely restrict the options for analysis. In addition to on-site or contract statisticians, the critical care nurse partner can alert the study team to the possibility of having QualityMetric Inc staff analyze the SF-36 data for a nominal fee. If nursing questions have been added to the study, they should be included in the budget for analysis.
The analysis includes checking every administration of the survey to verify the internal consistency (reliability).15 The results of data analysis will indicate whether the team can use the data summarized as scale scores or must report findings item by item (36 in this instance). The statistical procedure to test internal consistency is used to determine whether the items collected into a scale all contribute significantly (reliably) to the scale. If the resulting 
values meet the criterion, the team can group responses in the promised categories for the tool. For example, the team checks whether the SF-36 results are valid as indicated by the 8 subscales that are categorized by the Mental and Physical Component scales or if items must be examined independently, one by one. Having reliable scale scores that summarize is simpler for research purposes that group results across patients and for clinical projects in which findings for individual patients are examined. Compared with the efficiency of looking at scores for the 8 scales or the 2 summary scores, juggling 36 separate items is both tedious and complex.
Preparation for data analysis should address validity along with reliability. The update article by Ware12 reports details of reliability with values that exceed .80 for all scales except Social Functioning ( =.68). Froman26 provides coaching on the meanings of validity and reliability and how to meet these requirements. She reminds nurses to consult current editions of standard research methods texts because criteria change over the years.26 For example, the standard for internal consistency has shifted from the previously acceptable .60 Cronbach estimate27,28 to .70, which many researchers now consider "only modest."29 Internal consistency coefficients tell investigators, and clinicians, whether results of individual items can be grouped under a single scale score. Traditionally, if the criterion of .70 is not met, data are discarded—not reported. However, Froman and Owen30 point out that especially for culturally diverse groups of patients, when internal consistency scores for scales do not meet the threshold, valuable data are available by examining responses to the individual questions in the scale. Especially in the clinical setting, nurses may find that the answers of patients and patients’ family members to questions about the patients’ roles and functional status are more helpful than the overall summary scores for physical and mental health.
DATA COLLECTION
Ethical Practice
Staff nurses play an important role as gatekeepers of integrity in the research process. Each nurse is responsible for knowing the circumstances under which the data collector has been approved to administer the survey to patients and patients’ families. Nurses could independently make decisions at the unit level that help ensure that good quality data are obtained, but those data would not be publishable. For most institutions, the study team must obtain formal approval if the data will be presented at an outside meeting of any kind or published in any format that will circulate outside the institution. An institutional review board would study the proposal and provide a written approval that would be kept in the unit. The days of casual decisions to undertake a study are over. Staff nurses can protect the privacy of patients and patients’ families, and the burden on them during a difficult time, by insisting that the study team follow institutional rules.
Methods of Data Collection
In deciding methods of data collection, it is important to establish in advance when the study team will make exceptions to the data collection protocol and how those who administer the survey will have the patients record and report responses. Some possible variations are interview versus self-completion, family versus patient as reporter, clinical staff versus external research assistant as administrator of the questionnaire, and the timing of administration of the questionnaire in relation to other data being collected.
Timing of Administration of the Questionnaire.
In the critical care context, the study team should decide at what intervals the patients’ health status will be assessed. The standard format asks patients to rate their health during the 4 weeks preceding assessment, and the acute version asks for a rating during the 1 week preceding assessment. It is important to determine if the baseline for comparison will be patients’ perceptions of their health status before admission or at the time of discharge. Pread-mission baselines would be possible in those situations in which admission to the ICU is part of a planned surgical process, for example, with patients awaiting transplants. Patients can be given the standard version, which calls for thinking about the "past 4 weeks" if that time period makes sense. The alternative, acute version calls for patients to think about the "past week" and might be used if recent changes in clinical status are important. More accurate baselines are possible if the medical teams know the patients well and administer health-related quality-of-life measures as part of routine care in the outpatient setting. Collection of SF-36 scores at multiple points in time may indicate how recovery is affected by social role, mental health, or other social variables independent of clinical prognosis.22
Style of Administration of the Questionnaire.
It is especially important not to mix the interview style of administration (the questions are read to the patient) with self-completion. Some evidence20,31,32 suggests that scores are higher when the interview format is used, so if the data collectors administer the survey as an interview for the baseline and then as a mailed questionnaire for follow-up, the final analysis could lead to an underestimate of a treatment’s effectiveness. If reading the questions is the only option, the study team should consult the scoring manual, which recently was updated,14 for instructions that address variations and limitations in this approach. Data collectors should code data obtained through the interview format separately so that the analysis can address significant variations compared with paper and pencil responses.
Similarly, in intensive care a temptation exists to use proxy reports (eg, a family member completes the questionnaire on behalf of a patient). Proxy reporters tend to report more health limitations for the patient than patients do for themselves. This overreporting creates validity problems if the baseline administration is by proxy and the follow-up is by the patient.29
Other Variables.
The SF-36 questionnaire takes approximately 10 minutes to complete, although older patients need more time and may have trouble with self-completion. Critical care nurses can play a key role by ensuring that fatigue, pain, and adverse effects of medications do not cloud a patient’s ability to respond and by assisting when literacy is a factor, even when translations are available.
INTERPRETATION
Data Quality
Revisiting the purpose of the study and the questions for research is essential. Findings from the SF-36 are just one dimension of evaluating new approaches to care or applying existing treatments to new groups of patients. Critical care nurses can provide leadership by examining the quality of the analyzed data. The nurses have knowledge of changes in staffing, patient/family variables, and other influences that should be considered as findings are reviewed. Nurses can also look for the proportion of missing responses, what the statistician did about missing data, whether patients were consistent in the way they answered questions, and whether differences were clinically as well as statistically significant. Sometimes, a small change in scores may meet standards for statisticians examining groups but may be too small to be of use to clinicians concerned about individual patients.
Normative Comparisons
It is also valuable to take advantage of the norm-based scoring developed by Ware and colleagues from studies in the United States in which the English-language SF-36, Version 2, was used.12 The algorithm for calculating norm-based scores is available with purchase of the manual.14 A very general summary is available in the update article.12 More precise norms for sex and age group for the scores on the subscales, the Mental Component Summary, and the Physical Component Summary are available at http: //www.sf-36.com (first click "Research," then "Norms"). These norms supersede those provided in the 1994 scoring manual.11 Comparing your data with the national norms may provide valuable insight into unexpected findings and may help validate data collected by using the preliminary or beta versions of translations. The lack of data specific to ethnically diverse groups should be weighed in evaluating findings. To date, language preference is the only indicator for selecting among the versions.
FUTURE DIRECTIONS
Internet Applications
Recently, QualityMetric Inc has made it possible to use Internet access to directly enter online the responses to health status screening tools based on the SF-36. The results are computed immediately. This format is valuable whenever patients are physically able to use a wireless keyboard. (Eventually, use of a personal digital assistant might be an option.) If a patient cannot use a keyboard, then a surrogate can enter the patient’s responses. Patients’ family members may also be asked to participate by entering the family members’ own data for studies of the impact of the critical care experience on caregivers and other family members. When this method is used, privacy issues are important and the use of code numbers becomes critical for confidentiality.
When data are entered and computed simultaneously by using Internet access, some problems with the traditional paper-and-pencil method are avoided. A major problem with the paper-and-pencil method of data collection is the delay that occurs during transcription as responses are later entered into computer files. This process also can lead to errors in computer input.
A third disadvantage of a paper-and-pencil format is that it locks the survey into a set of questions that may be more than is needed for healthy respondents and less than is needed to pinpoint problems of the chronically ill. Ware et al33 have pooled scales from around the world and incorporated the scales into the digital testing set that is administered over the Internet. This feature provides astounding flexibility. In brief, the computer software gives a patient a small set of questions and the patient’s answers trigger probes that are more detailed. This "computerized adaptive testing" is currently available for patients with asthma, congestive heart failure, and other conditions. With this offering, QualityMetric Inc sets up a free "notebook" for your patient group at their Web site. The study team can add questions specific to a particular study for an extra charge. Critical care teams can preview this option by clicking the "amIhealthy.com" link at http://www.qualitymetric.com. Details of the theory and statistics behind this digital administration of the SF-36 can be accessed by reading the Medical Care article by Ware et al,33 available in full text at http://www.sf-36.com (first click "Research," then "Classic Articles").
Patients With Chronic Pain
Often, patients in critical care units have chronic pain. This condition can cloud their responses to treatment and act as a variable that may lead to an underestimation of interventions that are effective for other patients. A new adaptation of the SF-36 called TOPS, or the Treatment Outcomes in Pain Survey, is available for use with this population of patients.34 Currently, researchers have used it only with outpatients. However, the tool is designed to assess changes in individual patients when relief of chronic pain is an intended outcome of interdisciplinary treatments. Critical care nurses may wish to take the lead in exploring the use of this survey with their patients.34
Adaptations for Practice
Some research teams are selecting scales from the SF-36 to use as a quick, easily scored check on health status.22 One example is the General Health Scale (Table 1
). Welsh et al22(p1470) recommend the 5-item General Health Scale, which correlates with functional outcome, as a "reasonable index to include in admission assessments, even for busy clinicians." This team, which included a nurse and examined critical care patients with mean scores of 12.5 on the Acute Physiology and Chronic Health Evaluation (APACHE), echoes the growing consensus4,8,9 that personal perception of health status "may be better than what our scientific bias toward objective data has led us to believe."22 Patients’ own beliefs about their well-being can confound the judgments of clinicians that are based solely on findings from laboratory tests, diagnostic procedures, and physical examinations. Welsh et al22(p1470) suggested that patients’ perception of function before the patients’ stay in the ICU may account for variability in outcomes despite common diagnoses and severity ratings. The SF-36, they observed, works best with patients in the intermediate illness range, in contrast to the APACHE II, which is better with patients at the extremes. The SF-36, they argue, will permit interhospital comparisons and may be useful to ascertain patients at risk for poor outcomes who should be referred for support after hospitalization.
Finally, creating a true research partnership calls for adding questions to a proposed study that are specific to nursing concerns and nursing therapeutics. Adding nursing questions to already funded research multiplies the impact of the original study. When paired with nurses’ offers of assistance with recruitment, data collection, and the tasks of manuscript preparation, other investigators are likely to welcome such requests if the requests are made early in the process. The background provided in this article can assist critical care nurses in demonstrating their ability to make a meaningful contribution to research designed to study clinical outcomes in critical care.
Acknowledgments
The encouragement of Randy Caine as a catalyst for this manuscript is gratefully acknowledged. The editorial assistance of Martha Bowes was valuable.
Work related to this manuscript was partially supported by a grant from the James H. Zumberge Faculty Research and Innovation Fund at the University of Southern California, Los Angeles, Calif.
References
This article has been cited by other articles:
|
|
 |
|
  M. Jordan-Marsh, M. Cody, M. Silverstein, S.-Y. Chin, and R. Garcia Assessing a Self-Report Health Measure for Non-English-Speaking Elders: Issues in Using the SF-36 Health Survey Research on Social Work Practice, January 1, 2008; 18(1): 55 - 65. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
