ORIGINAL ARTICLE

Reliability of the Balanced Inventory for Spinal Disorders, a Questionnaire for Evaluation of Outcomes in Patients With Various Spinal Disorders Elisabeth Svensson, PhD,* Birgitta Schillberg, RN,w Anna-Maria Kling, MSc,z and Bo Nystro¨m, MD, PhDw

Study Design: An intrapatient reliability study of the previously validated 18-item questionnaire, the Balanced Inventory for Spinal disorders (BIS), in patients referred for planned spinal surgery. Statistical evaluation of the test-retest assessments was performed by a rank-based method that allows for separate analyses of the systematic and individual components of an observed disagreement. Objective: To evaluate the intrapatient reliability and the art of disagreement, when present, in assessing the extent to which pain affects perceived physical health, social life, mental health, and quality of life according to the BIS. For comparative reasons corresponding items in Short-Form-36 (SF-36) and Oswestry Disability Index (ODI) were also evaluated. Summary of Background Data: The questionnaires were filled in by 101 patients the evening before going to the clinic for planned spinal surgery and the following evening at the clinic. Results: The percentage agreement in test-retest assessments of the items varied from 52% to 84%. The important items of pain, physical activities, social life, overall mental health, and quality of life showed high levels of reliability. An intrapatient disagreement of more than 1 category was seen in 4 items of mental health and in physical health, only. The observed individual variability and the significant systematic decrease on the second occasion could be explained by the fact that the patients were at the hospital on the retest occasion. The variation in percentage agreements found, and the different reasons for disagreement in items speak against that memory alone could have caused the retest assessments. Conclusion: The comprehensive evaluation of test-retest reliability showed that the test-retest assessments on the BIS could be regarded as reliable, and the measures of reliability of the BIS items were on the same levels as for corresponding items of the SF-36 and the ODI questionnaires.

Received for publication May 20, 2010; accepted February 8, 2011. From the *Department of Statistics, Swedish Business School at O¨rebro University; wClinic of Spinal Surgery, Stra¨ngna¨s; and zDepartment of Epidemiology, Swedish Institute for Infectious Disease Control, Solna, Sweden. The authors declare no conflict of interest. Reprints: Elisabeth Svensson, PhD, Sla¨ndva¨gen 6, SE-386 34 Fa¨rjestaden, Sweden (e-mail: [email protected]). Copyright r 2012 by Lippincott Williams & Wilkins

196 | www.jspinaldisorders.com

Key Words: agreement, back pain, ordinal data, reliability, spinal disorders (J Spinal Disord Tech 2012;25:196–204)

A

ssessment of outcome in patients with various spinal disorders is a multidimensional task involving pain and quality of functioning and of health.1–6 The large number of different generic and disease-specific questionnaires used for functional status, disability, and outcome reflects the multidimensional complexity of outcomes in patients with spinal disorders.1–5,7–11 A small number of patient-oriented questionnaires are recommended for clinical research, such as the Short-Form-36 (SF-36),12 the Oswestry Disability Index (ODI),5,13 and the European Quality of Life Scale (Euro-QoL).14 These questionnaires are normally used at our clinic together with a more disease-specific questionnaire, the Balanced Inventory for Spinal disorders (BIS) an 18-item questionnaire developed at the clinic (Table 1). The BIS is aimed for assessments of pain and pain-related dysfunction and also for evaluation of outcomes after treatment in patients with spinal disorders. A prospective study has been performed to evaluate the validity, reliability, and responsiveness of the BIS. The validity of the BIS is confirmed and reported.15 The conclusion was that BIS is a valid disease-specific questionnaire for use in clinical practice and research. The disease-specific well-defined items cover both painrelated limitations in specific activity variables and in overall physical, mental, and social functioning and of quality of life. The evaluation of responsiveness will be reported in a forthcoming paper. The study of intrapatient reliability is presented in this paper. Perception of pain and pain-related activities are extremely subjective and so are the assessments. The aim of a test-retest reliability study is to evaluate the extent to which scale assessments are reliable, which means will yield stable and reliable results when assessed repeatedly under unchanged or controlled conditions. The time span between the repeated assessments will depend upon the variables being measured, the target population with reference to the clinical use.16,17 In intrapatient reliability studies, paired data are obtained by test-retest assessments. Such assessments on J Spinal Disord Tech



Volume 25, Number 4, June 2012

J Spinal Disord Tech



Volume 25, Number 4, June 2012

scales produce ordinal data, the ordered categories representing only a rank order of the intensity of a particular variable and not a numerical value in a mathematical sense, even when the assessments are numerically labeled.18–22 Consequently, nonparametric statistical methods must be used. Svensson has developed a rank-based statistical method for paired ordinal data that makes it possible to identify and measure the systematic disagreement, when present, separately from the disagreement caused by individual variability. In reliability studies, these 2 sources of disagreement have different impacts on the quality of scales.23–25 A typical reason for systematic disagreement, or bias, is a systematic change in conditions between the 2 occasions of assessments. A high level of additional individual variations in paired assessments indicates that the items or the response categories do not fit well to all, which might cause uncertainty in some patients, or that the assessments are sensitive to disturbing factors between the assessments.25,26 The aim is to study the test-retest reliability of the BIS items, and evaluate the levels of systematic and occasional components of disagreement, when present. For comparative reasons the reliability of similar item variables from the SF-36 and ODI questionnaires will also be evaluated.

MATERIALS AND METHOD Questionnaire The BIS is a multidimensional paper questionnaire regarding the extent to which spinal disorders affect the physical and mental health, social life, and quality of life as perceived by the patient (Table 1). The assessments are made on verbal descriptive scales and the verbal categories are not numerically coded as such a coding does not transfer ordinal data to quantitative data.18–22 Referring to the level of perceived low back pain and leg pain, each dimension of the BIS is measured not only by items of specified activities, but also by an item regarding the overall perception of the dimension. The items 5, 9, 15, and 18 measure the dimensional variables physical health, social life, mental health, and quality of life, respectively and provide verbal descriptive data.

Patients and Design Patients referred for planned spinal surgery at the clinic were eligible for this study. Between September and December 2004, 108 patients having one of the diagnoses of disc herniation and stenosis in the lumbar region, segmental discogenic pain, and isthmic spondylolisthes were informed and consented to participate in this study. Seven patients were excluded during the study, because of another surgery (n = 4), additional disease, recovery, and 1 missing set of follow-up assessments. Therefore, the study group comprised 101 patients (52 females); the median age was 57.0 (Q1, 41.0; Q3, 69.0; range, 19-85) years. The SF-36, the Euro-QoL (EQ), the ODI, and the r

2012 Lippincott Williams & Wilkins

Reliability of the Balanced Inventory for Spinal Disorders

TABLE 1. The Dimensions, Items, and Scales of the Swedish Balanced Inventory for Spinal Disorders Questionnaire Effects on physical health 1. How severe low back pain have you had over the last four weeks? 2. How severe leg pain have you had over the last four weeks? The pain scale: none, negligible, moderate, rather severe, very severe. To what extent have the following activities been limited because of your back/leg complaints over the last 4 weeks? 3. Indoor activities, eg, cleaning, cooking, laundry 4. Outdoor activities, eg, shopping, visiting the bank or post office etc The limitation scale: not at all, negligible, moderate, quite a bit, very much 5. With respect to your back/leg complaints, how is your perceived physical health? Very good, rather good, neither good nor poor, rather poor, very poor 6. Walking ability: My back/leg complaints do not limit my walking at all My back/leg complaints limit my walking distance to about 1 km My back/leg complaints limit my walking distance to about 200 m My back/leg complaints limit my walking distance to 50 m My back/leg complaints are so severe that I must use crutches/ walker Effects on social life To what extent have the following activities been limited because of your back/leg complaints over the past 4 weeks? 7. Leisure activities e.g. traveling, sports, societies, dancing, going to cinema, restaurant etc. 8. Social activities with your family, friends or others. 9. To what extent is your social life limited by your back/leg complaints? The limitation scale: not at all, negligible, moderate, quite a bit, very much Effects on mental health How well do the following statements apply to how you are affected by your back/leg complaints? 10. I feel down 11. I feel impatient 12. I have lack of initiative 13. I have difficulty concentrating 14. I have sleeping disturbances The scale categories: not at all, a little, moderate, quite a bit, very much. 15. How is your perceived overall mental health? Very good, rather good, neither good nor poor, rather poor, very poor Effects on the quality of life as a whole. 16. To what extent do your back/leg complaints limit your way of living? Not at all, negligible, moderate, to a rather great extent, to a very great extent. 17. How much concern does your pain cause you? No, slight, moderate, great concern, a real misery. 18. How would you rate your overall quality of life? Very good, rather good, neither good nor poor, rather poor, very poor.

BIS are normally used for all patients presurgery and postsurgery at the clinic. Our study was designed to fit this presurgery occasion together with an additional presurgery occasion. The first set of questionnaires was completed the evening before going to the clinic. The next day at the clinic, the eligible patients were informed and consented to participate in the study, and in the evening they filled in the set of questionnaires again. This test-retest interval-contingent www.jspinaldisorders.com |

197

J Spinal Disord Tech

Svensson et al

design was used to avoid uncontrolled variations in the patients’ conditions.27 The risk of biased assessments owing to the designed change of location was in common for all patients, and we used a statistical method that could identify and measure presence of such systematic disagreement.25,28

Statistical Methods A self-assessment scale of high quality must have a high level of intra-individual agreement and lack of both systematic disagreement (bias) and of individual variability in the test-retest assessments.25,26,28 The statistical nonparametric method by Svensson is applied and demonstrated by a worked example. Figure 1 shows the frequency distribution of the pairs of data from the test-retest assessments regarding the variable feeling down (BIS item 10). The percentage agreement (PA) in the test-retest assessments was 52%. What is the main reason for disagreement? Presence of systematic disagreement, which is the disagreement ascribed for the group is revealed by different frequency distributions of assessments on the scale categories as evident by the 2 marginal distributions of the contingency table. Six patients responded not at all feeling down on the first and 13 did so on the second occasion, 4 of them were in common. Two measures of systematic disagreement were calculated; the relative position (RP) and the relative concentration (RC).28–30 The RP expresses the extent to which the marginal distribution of the second occasion (denoted Y) is shifted toward higher categories than the marginal distribution of assessments from the first occasion (denoted X) rather than the opposite. A theoretical description of this systematic shift in position is the difference between probabilities, Prob(X