The Assesstnent of Occ pational Functioning: A Screening Tool for Use in Long-Tertn Care (human occupation, long-term care, test construction)

Janet Hawkins Watts, Gary Kielhofner, David F. Bauer, Mark D. Gregory, Diane B. Valentine

This paper descriL'es the development of the Assessment of Occupational Functioning (AOF), a screening tool designed to assess the functional capacity of residents m long-term treatment settings who have physical and/ or psychiatric problems. The assessment is based on six variables of the Model of Human Occupation. A study of 83 community and institutionalized elderly subjects was conducted to examine the AOF's dimensionality, test-retest reliability, interrater reliability, concurrent validit)l, and ability to discriminate between healthy and institutionalized adults. 1tem analysis suggests that ratings tend to correspond with components of the theoretical model. Both testretest reliability and intermter reliability correlations for total test scores were above accepted minimum levels. Correlations of the screening tool with scores on the Life Satisfaction 1ndex-Z, a concurrent validity measure, yielded positive correlations. Correlations of the screening tool score with another concurrent validity measure, the Geriatric Rating Scale score, yielded mixed results. Discrimination results indicated that the instrument can distinguish be-

tween the adaptive performance of norma I and institutionalized populations.

"T

his paper describes the development of the Assessment of Occupational Functioning (. OF), a screening tool based on the J\fodel of Human Occupation (1S). The tool is intended to screen o\erall occupatIonal function of phYSICally disabled and/or p ychiatric p,nienrs/residenls in Jongterm selting~ such as state hospitals and Intermediate care resid lltial facilities. The purpose of the assessment is to provide the therapist \,'ith self-report IIlformation concerning the patient'S values, personal causation, interests, roles, h2bil~, and skills. The tool may thus help the therapisl to obtain a brief overview of overall functioning and to identif\ areas for further assessment and treatment.

The Need for Be ter Assessments Tile lack of occupational therapy assessments with empirically dem-

onstrated reliability and validity has recently been a topic of concern (6-9). Most existing assessments have unknown reliability and validity. Many therapists use informal assessments developed by (conlinued on page 235)

Janet Hawkins Watts, MS, OTR, is Assistant Professor, Department of Occupational Therapy, Virginia Commonwealth University, NIedical College of Virginia, Richmond, VA 23298-0001 Gar)' Kielhofner, DrPH, OTR, FAOTA, is Associate Professor, Department of Occupational Therapy, Sargent College of Allied Health Professions, Boston C'niversity, Boston, "'IA 02215. David F Bauer, PhD, is Associate Professor, Department of lvIathematical Sciences, Virginia Commonwealth University, Richmond, VA 23284. Lieutenant N1ark D. Gregory, MS, OTR, is an occupational therapist, Biomedical Science Corps, U.S. Air Force Regional Hospital, March Air Force Basf, CA 92518. DianeB. Valentine, OTR, is an occupational therapIst, The Virginia Home, Richmond, VA 23220.

The American Journal of Occupational Therapy Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

231

Figure 1

Figure 2

Data plan interview

Assessment of occupational functioning of adults in long-term care programs/facilities

Please interview the resident following this format. You may use a few follow-up questions or rewordings to elicit further information. Note responses on this form. Responses from this interview will provide the input for you to make ratings on the assessment form. For use in research, investigators are to rely only on information collected from these interview questions to determine ratings. VOLITION SUBSYSTEM Values

What activities are important to you? Name at least five. Do you have certain ideas about how you should carry out your daily activities? Do you think about what others expect of you? What were you doing last year at this time? Do you have plans for one year from now? For five years? What are they? Do you think about the future? Do you have goals for the future? What are your goals? Personal Causation

Do you feel in control of your life? For example, do you make your own decisions? Do other things or people (for example family, health, age, institutional rules, etc.) interfere with your life? Was it your decision to come to this program (facility)? Are you able to do the diHerent things you want to do (go diHerent places, meet diHerent people, try diHerent activities)? list these things. Can you do the things that you need to do each day? Can you do them well? Do you believe you will be able to do the things you need and/or want to do in the future? Interests

Name the things you like to do. Which of these do you like best? How often do you do each of these things? Are there things that you would like to do that you are not doing now? list them. HABITUATION SUBSYSTEM Roles

How do you spend most of your time? What do you do? With whom? How often? What are your major roles in life? What does that mean you have to do? How would you describe your involvement here in this program (facility) as a hobbyist, volunteer, active participant, patient, participant in social activities, friend, client, student, worker, or in another role? Habits

What is a typical weekday like for you? Weekend day? Do you have enough time to do the things you need to do? Do you have too much time? (rontinued on next page)

232

When making the ratings, choose the statement that best describes the patient. VOLITION SUBSYSTEM Values

1. Has values; has well-defined meaningful activities, goals, and standards; is oriented in time; and follows through on activities and goals (pursuit routine). 2. Has values; identifies activities, goals, and standards; is oriented in time; activity and goal pursuit is variable (specify whether the individual is unable or unwilling to pursue). 3. Has some diHiculty identifying values, goals, meaningful activities, standards, and time orientation (pursuit limited). 4. Has major diHiculty identifying values, goals, meaningful activities, standards, and/or time orientation (pursuit minimal). 5. Does not identify values, goals, meaningful activities, standards, or time orientation (pursuit absent). Personal Causation

1. Consistently makes own decisions independently. Expresses confidence that skills are adequate for attaining desired goals. Has appropriate outlets and displays skill competence. Characteristically wiling to seek out and engage in new activities. 2. Makes own decisions with limited support and encouragement. Occasionally lacks confidence in skills needed to pursue goals. Has appropriate outlets, but lacks some skill competence. Hesitates before engaging in new activities. 3. Makes own decisions only with much support and encouragement. Typically lacks confidence in skills needed to pursue goals. Has limited outlets and limited skill competence. Hesitates before engaging in routine, familiar activities. 4. Refuses to make own decisions, prefers having others decide. Articulates expectancy of failure in many areas, but still identifies a few (1 or 2) areas of limited skill. 5. Is passive, unable to make decisions. Expects failure and identifies no area of skill or negates, denies areas of skill. Lacks appropriate outlet and skill competence. Interests

1. Has clearly identified areas of interest; pursuit routine. 2. Has clearly identified areas of interest; pursuit variable. 3. Interests are ill defined; pursuit limited. 4. Interests are ill defined; pursuit minimal. 5. No interest in environment or activities; withdrawn; pursuit absent. HABITUATION SUBSYSTEM Roles

1. Identifies an adequate balance of major life roles (family, student, worker, hobbiest, etc.); has realistic concept of the demands and social obligations of those roles. 2. Identifies a limited number of major life roles; continues to have a realistic concept of the demands and social obligations of those roles. 3. Cannot clearly identify major life roles, but expresses some role involvement or interests in roles; unclear concept of the demands and social obligations of those roles. 4. Cannot clearly identify life roles; has little understanding of demands and social obligations of those roles. 5. Is disengaged from all life roles; may still identify with family role, but concepts of demands and social obligations unrealistic. Habits

1. Time is well organized; is flexible when necessary; performs autonomously in a variety of routine tasks in a variety of situations. 2. Time is well organized; is generally flexible; is able to perform tasks in a variety of situations with regularity when a minimum of structure is provided. 3. Performs within acceptable time frames only in familiar situations and with more structure; degrees of flexibility limited; performs (ADL, leisure, work) tasks, if able, routinely meeting environmental and social demands. (rontinued on next page)

April 1986, Volume 40, Number 4

Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

Figure 1 continued

Figure 2 continued

Do you feel you are flexible when the situation requires a change in your routine? How do you act when things change?

4. Organization of time unclear; is noticeably rigid or lax in meeting demands for routine performance in familiar situations, but can do so with much support, gUidance, and structure. 5. Unable to organize time; unable to perform routine tasks; may be rigid and unyielding.

Do you depend on encouragement or reminders from others to accomplish routine daily tasks?

PERFORMANCE SUBSYSTEM Skills Do you have trouble moving or getting around? Does it interfere with the things you need and/or want to do?

If you run into problems, can you usually figure them out for yourself? Do you ask others for help to figure out a problem? Do you feel you get along well with others? Do you ever have trouble understanding what others want you to do? Do you ever have trouble making others understand you?

PERFORMANCE SUBSYSTEM Skills 1. Motor, process, and interactions/communicational skills are adequate to pursue values and goals; no identified areas need improvement.

2. Motor, process, and interactional/communicational skills are generally good; some deficits in at least one area may need improvement.

3. Significant deficits exist of motor, process, and/or interactional/communicational skills; but there is possibility of goal attainment through therapeutic activity and/or adapted equipment. 4. Has major deficits of motor, process, interactional/communicational skills with little possibility of goal attainment; or refuses to accept deficits or use adapted equipment or special techniques.

5. Has extreme limitations of motor, process, and/or interactional/communicational skills, which cannot be altered by therapeutic activities or adapted equipment. ADL, activities of daily living.

Figure 3 Content validity chart Model Component THE VOLITION SUBSYSTEM Values

Temporal orientation

Meaningfulness of activities

Occupational goals

Personal standards

Personal causation

Belief in skill Belief in the efficacy of skill

Belief in internal/external control

Expectency of success/ failure Interests

Conceptual Definition

Personal images of what is good, right, and/or important. The way in which an individual interprets, views his or her own placement in time. It includes the degree of orientation or concern with past, present, or future, and the beliefs one holds about how time should be used. The individual's predisposition to find importance, security, sense of worth, and purpose in certain forms of occupation. Objectives for personal accomplishments or future occupational activities or roles. Commitments to performing in moral, excellent. efficient, or otherwise socially sanctioned ways.

A collection of beliefs and expectations that a person holds about his or her effectiveness in the environment. The person's conviction that he or she has a range of abilities. The person's conviction that his or her abilities are relevant to his or her current situation. The individual's conviction that outcomes in life are related to personal actions (internal control) versus the action of others, fate, etc. (external control). The person's anticipation for future endeavors and whether outcomes will be successful or not. Dispositions to find occupations pleasurable.

Data Plan/Interview

Assessment

What were you doing last year at this time? Do you have plans for one year from now? For five years? What are they?

Is oriented in time.

What activities are important to you?

Has well-defined, meaningful activities.

Do you think about the future? Do you have goals for the future? What are your goals? Do you have certain ideas about how you should carry out your daily activities? Do you think about what others expect of you?

Has well-defined, meaningful goals.

Are you able to do the different things you want to do? Can you do the things that you need to do each day? Can you do them well? Do you feel in control of your life? Do other things or people interfere in your life?

Do you believe you will be able to do the things you need and/or want to do in the future?

Has well-defined standards.

Skills are adequate for attaining desired goals. Has appropriate outlets and displays skill competence. Consistently makes own decisions independently.

Characteristically willing to seek out and engage in new activities. (continued on next page)

The American Journal of Occupational Therapy Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

233

Figure 3 (continued)

Discrimination of interest

Pattern of interest

Potency of interest

THE HABITUATION SUBSYSTEM

Roles

Perceived incumbency

Internalized expectations

Role balance

Habits

Degree of organization

Social appropriateness

Rigidity and flexibility

THE PERFORMANCE SUBSYSTEM Skills

234

The degree to which one diHerentiates a liking or expectation/pleasure in certain occupations and objects. The configuration of occupations toward which a person expresses interest. The degree to which interests are based on past experience and influence present action.

A collection of images that trigger and guide the performance of routine patterns of behavior. Two sets of these images exist and interrelate in gUiding everyday behavior; they are referred to as habits and roles. Images that persons hold of themselves as occupying certain status or positions in social groups and of the obligations or expectancies that accompany being in the role. The belief that one has status, rights, and obligations of a role and that others perceive one to be in that role. It also includes the image one holds of when (i.e. during the day, week) one is in a given role. The images one holds of what others expect one to do by virtue of being in a role (i.e., the perceived obligations of a given role). The degree of healthy integration of roles into one's daily life. The routine and typical ways in which one performs. It refers to both the temporal structuring of behavior and the style and manner of performance. The degree to which one has and can report a typical use of time. The degree to which one's typical behaviors are those expected, valued by the environments in which one performs. Refers to a continuum representing the degree a person changes routines of behavior to accommodate unusual contingencies.

A collection of images and biological components that are used in the production of skilled behavior. Abilities that a person has for various performances. Three broad types of skills are recognized: (a) motor skills are skills in manipulating self and objects; (b) problem-solving and planning skills are skills for managing events, processes, and situations; and (c) communication/ interpersonal skills are skills for interacting and cooperating with people.

Which of these do you like best? Are there things that you would like to do that you are not doing now? Name the things you like to do.

Has clearly identified areas of interest.

How often do you do each of these things?

Pursuit is routine.

What are your major roles in life? How would you describe your involvement here in this program (facility)?

Has realistic concepts of the demands of those roles.

What does that mean you have to do?

Has realistic concepts of the social obligations of those roles.

How do you spend most of your time? What do you do? With whom? How often?

Identifies an adequate balance of major life roles.

What is a typical day like? Do you have enough time ... ? Do you have too much time? Do you depend on encouragement or reminders from others to accom piish routine daily tasks?

Time is well organized.

Do you feel you are flexible when the situation requires a change in your routine? How do you act when things change?

Is flexible when necessary.

Do you have trouble moving or getting around? Does it interfere with the things ...?

Motor skills

If you run into problems, can you usually figure them out for yourself? Do you ask others for help to figure out a problem?

Process skills

Do you get along well with others? Do you ever have trouble understanding ... ? Do you ever have trouble making others understand ... ?

Interactional/communicational skills

Has identified areas of interest.

Performs autonomously in a variety of routine tasks in a variety of situations.

April 1986, Volume 40, Number 4

Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

(continued from page 231)

themselves or their colleagues for their own practice setting; others adapt existing instruments and use them with patients for whom they were not designed; still others use instruments developed by non-occupational therapists (10). Using such assessments amounts to employing procedures that may lack stability and may not yield the information the assessment is intended to collect (6). Developing and refining such instruments would improve the practice of occupational therapy by aiding in the collection of valid data for establishing our knowledge base more firmly (II). Instruments that are grounded in theory and have demonstrated reliability and validity also increase confidence in practice.

Clinical Impetus and Conceptual Basis for the Instrument The AOF grew out of the specific needs of one therapist to develop a brief, yet comprehensive assessment that could be used to establish funher assessment and treatment priorities in a setting with more than 100 physical1y disabled and/or aged residents fOIwhom she was responsible. A team of faculty members and clinicians knowledgeable about theory cooperated in these early stages of the instrument's development and empirical study. The AOF is based on a model that conceptualizes humans as open systems interacting with their en vironments through a cycle of intake, throughput, output, and feedback (1-5). Internal variables that influence the output or occupational functioning of the individual are conceptualized as constituting three subsystems. The volition subsystem, composed of personal

causation, values, and interests, influences choices a person makes for engaging in occupation. The habituation su bsystem, composed of habits and roles, regulates the patterning of occupational functioning. The performance subsystem, composed of skills, is responsible for the production of behavior. These six internal components of the three subsystems were identified as ta rgets for assesslllen t.

The Instrument (AOF) The AOF consists of two pans, a sel1l istructu red interview sc hedlile (see Figure I) and a series of 5point rating scale items (see Figure 2). Rating scale statements describe the range of function in five increments from absent to fully adaptive. These statements are given for each of the six model components (values, personal causation, interests, roles, habits, and skills). A total score between 6 and 30 may be obtained by adding the scores for all six components. The complete screening process involves a brief inteniew (approximately 20 minutes) followed by completion of the rating form, which takes less than 5 minut The interview yields useful qualitative data on the person, while the ratings help summarize and quantify the therapist's judgment about the patient.

The Process of Instrument Development The four-step process for instrLlment development described by Benson and Clark (12) was followed. Planning, the first step, involved recognizing the need for a screening tool for overall adaptive occupational functioning in longterm settings and selecting a theoretica I model to gu ide construction. The purpose ane! target groups were designated, and rele-

vant literature was reviewed. Thus, a screening instru ment was Ian ned that would assess occupational functioning as measured by ratings of personal causation, values, interests, roles, habits, and skills. In the next phase, construction, an interview format was chosen, and the questions and ratings were devised to match the selected components of the Model of Human Occupation. Succeeding versions were used with disabled subjects \"hose feedback on the questions' content and clarity was solicited. Two postprofessional graduate students with long-term care experience further tested the use of the instrument and revised it. Next, revisions were made after consultatioll with content experts who were versed in the theoretical model and experienced in interviewing long-term care patients. These revisions were made to achieve wording that most precisely and efficiently elicited the desired information and to make items fit more precisely. with the model's theoretical content. Figure 3 illustrates the correspondence of the modeJ's conceptual variables with the interview questions and the instrument's rating scale items. In a third phase, quantitative evalu.ation, the instrument was evaluated through a pilot test. Thirteen su bjects were rated by two therapists to examine interrater reliability. Each therapist interviewed subjects with the other therapist observing. Then ratings were COI11pleted independently by both therapists for each subject. A Pearson product-moment correlation of .85 \vas achieved between the two raters. The instrument. was also evaluated for the clarity of questions and I'd ting items. Fu nher revisions were then made to clarify rating items and to improve questions.

The American Journal of Occupational Therapy Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

235

Table 1 Subjects' Demographic Information (N = 83) Psychogeriatric State Hospital Subjects (N 25)

=

The final phase, validation, is an ongoing process of determining to what degree an instrument actually measures the content domain it claims to measure. To begin this process, the following study was designed to assess some aspects of reliability and validity of the instrument.

Subjects

Data were collected on a convenience sample of 83 subjects from two institutions and the community. Institutionalized subjects (n = 49) were from the geriatric center of a large state psychiatric hospital and from a long-term facility for physically disabled patients needing intermediate nursing care. Some of the psychogeriatric subjects had only psychiatric diagnoses, while others had combined physical and psychiatric diagnoses. All were 60 years old or older and capable of responding to interview questions. The average inpatient time was 13.9 years for the intermediate care subjects and 2.1 years for state hospital subjects. Community subjects (n = 34) were elderly individuals living independently, 60 years old or older, who were experiencing no physical or psychiatric problems requiring treatment at the time of the study or for three months prior. Information on the subjects' age, sex, and diagnoses is shown in Tables 1 and 2.

=

=

Combined Psychiatric and Physical Diagnoses (n 13)

10 (29%) 19 (56%) 5 (15%) 0

12 (50%) 9 (38%) 2( 8%) 1 (4%)

7(58%) 5 (42%) 0 0

8(62%) 3 (23%) 2 (15%) 0

26 (76%) 8(24%)

19 (79%) 5(21%)

6(50%) 6(50%)

7(54%) 6(46%)

causation, values, interests, roles, habits, and skills. According to the model, the status of these variables will affect the functional performance (output) of the system and any feedback (including life satisfaction). In addition, two measures of concurrent validity were used. The Geriatric Rating Scale (GRS) (13) measures the functional performance of geriatric persons. Interrater reliability of .87 has been reported for the scale, and it has evidence of concurrent and discriminant validity. The Life Satisfaction Index-Z (LSI-Z) (14) is a

=

self-report measure of life satisfaction or morale developed for use with older subjects. The split-half reliability for this instrument is .79, and it also has evidence of concurrent validity. Data Collection

Therapists in each of the two inpatient facilities conducted audiotaped AOF interviews with 49 subjects; they also arranged for administration of the GRS, and they administered the LSI-Z. All audiotapes were then rated by the interviewer and two other re-

Table 2 Demographic Information: Institutionalized Subjects' Diagnoses (N = 49)

Instruments

The AOF was used to measure the adaptive status of personal 236

=

Psychiatric Diagnoses Only (n 12)

Age 60-69 70-79 80-89 90-99 Sex Female Male

Methods This descriptive study examined the following aspects of the AOF: (a) dimensionality, (b) test-retest reliability, (c) interrater reliability, (d) concurrent validity, and (e) ability to discriminate between healthy and institutionalized adults.

Community Subjects (n 34)

Intermediate Care Facility Subjects (n 24)

April 1986, Volume 40, Number 4

Psychogeriatric State Hospital (N 25)

=

Diagnosis Psychiatric Schizophrenic disorders Major affective disorders Paranoid disorders Organic mental disorders Adjustment disorders Substance use disorders Physical Arthritis Multiple sclerosis Arteriosclerotic heart disease Hypertension Diabetes Cerebral palsy CVA Spinal cord injury Other physical diagnoses

Intermediate Care Facility Subjects (n 24)

=

o o o o o o

Psychiatric Diagnosis Only (n 12)

=

o 4 (33.3%) o 7 (58.3%) 1 ( 8.3%)

o

8 (33.3%) 7 (29.2%)

o o o

2( 8.3%) 2( 8.3%) 2( 8.3%) 3 (12.5%)

CVA, cerebral vascular accident.

Downloaded From: http://ajot.aota.org/pdfaccess.ashx?url=/data/journals/ajot/930421/ on 01/21/2017 Terms of Use: http://AOTA.org/terms

Combined Psychiatric and Physical Diagnoses (n = 13) 1 ( 7.6%) 5 (38.5%) 1 ( 7.6%) 4 (30.7%) 1 ( 7.6%) 1 ( 7.6%) 2 (15.4%)

o

2 (15.4%) 1 ( 7.7%) 3 (23.1%)

o o o

5 (38.5%)

Table 3 Standardized Scoring Coefficients for the 6 Rotated Factors Extracted From the 6 Ratings on the AOF (N = 83) Factor Component

searchers to examine interrater reliability. The AOF was readministered to the institutionalized subjects after 14 to 21 days to examine test-retest reliability. The community subjects were interviewed once with the AOF, and data were used to determine the assessment's ability to distinguish between institutionalized and non institutionalized healthy individuals. Since all three interviewers were familiar with the Model of Human Occupation, it is assumed that knowledge of the model is required to administer the AOF interview and do the ratings.

Results Item Analysis

All item ratings correlated highly with the total score on the AOF, with correlations between .70 and .94. The interitem correlations were all positive (between .57 and .91), indicating that each influences some aspect of overall occupational functioning. A factor analysis of the six component scores was done for each of the three raters to explore whether the six items measured separate domains. With each rater, when six factors were requested, the rotated factors cou Id be identified satisfactorily with the six components, as the standardized scoring coefficients for one rater show (see Table 3). The coefficients were similar for the other two raters. This indicates that the ratings tend to correspond to the components of the Model of Human Occupation. Reliability

To examine test-retest reliability, Pearson product-moment correlations were calculated for each of the six items and for the total score of the AOF (see Table 4). These coefficients were calculated separately for intermediate care fa-

Values Personal causation Interests Roles Habits Skills

1

2

3

4

5

6

1.83 -034 -0.32 -0.37 -0.15 -0.11

-0.07 -0.47 -0.14 -0.12 -029 1.66

-0.15 -012 -038 -0.62 2.17 -0.45

-0.31 1.84 -025 -0.15 -0.06 -0.60

-0.59 -0.14 -0.29 2.37 -0.94 -0.05

-075 -038 2.49 -0.35 -063 -0.15

AOF, Assessment of Occupational Functioning.

Table 4 Test-Retest Reliability (Pearson Product-Moment Correlations) (N = 49) Psychogeriatric State Hospital Subjects (N 25)

=

Intermediate Care Facility Subjects (n 24)

=

Psychiatric Diagnosis Only (n 12)

=

Combined Psychiatric and Physical Diagnoses (n 13)

60" .68" .76" 69" .48" .83"

.94" .90" .89" .79" .48 .83"

.75" .S4t .12 .57t .61t .60t

.85"

90"

.70"

Values Personal causation Interests Roles Habits Skills Total

=

"p < .01 . t P