Education and Training in Developmental Disabilities, 2009, 44(4), 471– 480 © Division on Developmental Disabilities

Differences Between Employees’ and Supervisors’ Evaluations of Work Performance and Support Needs Kyle Bennett, Michael Frain, and Michael P. Brady

Howard Rosenberg Florida International University

Florida Atlantic University

Tricia Surinak Broward County Public Schools Abstract: Assessment systems are needed that are sensitive to employees’ work performance as well as their need for support, while incorporating the input from both employees and their supervisors. This study examined the correspondence of one such evaluation system, the Job Observation and Behavior Scale (JOBS) and the JOBS: Opportunity for Self-Determination (JOBS: OSD), to establish whether performance and support evaluations administered by work supervisors would match self-determined perceptions of the same variables by sheltered and supported employees with disabilities. Results showed that JOBS ratings established by supervisors did not correspond closely to employees’ self-determined ratings for any of the JOBS’ subscales (Work-Required Daily Living Activities, Work-Required Behavior, and Work-Required Job Duties). Results also showed that employees consistently overestimated their work performance and underestimated their support needs relative to supervisors’ ratings. These results suggest that major discrepancies exist between supervisors’ and employees’ perceptions of their work performance and support needs. These discrepancies may be important as job coaches, teachers, rehabilitation professionals, and employees and students with disabilities participate in employment preparation and transition planning. For most adults, employment is intricately tied to culture and social values, and is central to the roles adults hold in their communities and within their families. Because most people spend the majority of each day working, work is significant in how adults define themselves (Abrams, DonAroma, & Karan, 1997; Grossi, Schaaf, Steigerwald, & Mank, 2002; Kolstoe, 1961). For most adults, employment has personal benefits including self-confidence, direction, and increased life satisfaction (Griffin, Rosenberg, Cheyney, & Greenberg, 1996; Skinner, 2003). Economic benefits include the opportunity to shape one’s own financial independence and to decrease reliance on families and others for economic support (Anthony, 1994; Kim & Morningstar, 2005; Reisman & Reisman, 1993). Work also provides

Correspondence concerning this article should be addressed to Kyle Bennett, Florida Atlantic University, Department of Exceptional Student Education, 777 Glades Road, Boca Raton, Florida 33431.

social benefits including the opportunity to establish meaningful relationships and the opportunity to contribute to one’s community (Mancuso, 1990; Storey, 2002). Unfortunately, the benefits of work have yet to reach many individuals with disabilities, and community employment continues to be an elusive goal (Brady & Rosenberg, 2002a). For example, nearly 2/3 of adults with disabilities remain unemployed or under-employed, or only have access to employment in sheltered workshops (Levy, Jessop, Rimmerman, Francis, & Levy, 1993; Murphy, Rogan, Handley, Kincaid, & Royce-Davis, 2002). Access to community employment is particularly troublesome for individuals with more complex disabilities and behavior challenges (Hurlbutt & Chalmers, 2004; Mank, Cioffi, & Yovanoff, 1998; McDermott, Martin, & Butkus, 1999; Muller, Schuler, Burton, & Yates, 2003). For community employment to be a successful outcome, many individuals with disabilities require explicit and purposeful employment

Evaluations of Work Performance and Support Needs

/

471

and transition planning. This includes active interventions to improve work performance (such as job coaching) and attention to the design and delivery of work supports (Brady & Rosenberg, 2002a; Mancuso, 1990; Stodden, 1998; Targett, Wehman, McKinley, & Young, 2004). In addition, employment evaluations need to be sensitive to these two separate, but inter-related dimensions. To date, evaluations of supported and sheltered employees have seldom been linked directly both to demonstrated work performance and employment support needs (Brady, Rosenberg, & Frain, 2008). As important as it is to evaluate both of these employment dimensions (performance and support needs), it is equally important that employment evaluations include the perceptions of both employers and the employees themselves. In most community employment settings, work supervisors are most responsible for these evaluations; employment factors such as promotions, raises, and retention rely heavily on the perceptions of performance by supervisors (Graffam, Shinkfield, Smith, Polzin, 2002; Hamilton & Shumate, 2005; Krantz, 1971). However, for individuals with disabilities and other employment challenges, gaining input from employees directly is critical when making employment decisions (Brady et al., 2008; Lynn, Sumsion, McWilliam, & MacKinnon, 2004; Wehman, 2006), particularly when planning interventions to promote job skills or developing employment and transition plans (Martin & Huber Marshall, 1996; Menchetti & Garcia, 2003; Olney & Salomone, 1992; Rogan, Banks & Howard, 2000). Unfortunately, few employment evaluations include self-determination when establishing instructional or support goals and transition plans for community employment (Thoma, Williams, & Davis, 2005; Wehmeyer & Schwartz, 1997). Fewer still include both employer and employee perceptions of work performance and support needs (Shaw, McMahon, Chan, & Hannold, 2004; Valenzuela & Martin, 2005). If employment evaluations are going to be helpful in developing employment and transition plans for individuals with disabilities, then assessment information is needed along all four of these dimensions. First, it is important to establish whether or not an employee’s work productivity matches the productivity of other employees without disabilities. This per-

472

/

formance dimension is typically the focus of employer and supervisor evaluations (Hamilton & Shumate, 2005; Morgan & Alexander, 2005). Second, for an employment evaluation to be useful in a supported employment context, it needs to incorporate input on the type and level of support an employee uses to establish that level of performance (Rogan et al., 2000; Targett et al., 2004). Third, since employers are responsible for the continued operation of the business or enterprise, the evaluation necessarily includes the employer’s perception of an employee’s performance and support needs (Graffam et al., 2002). Finally, if transition and employment plans are to be developmental in nature – that is, the plans are to promote growth and opportunity in employees with disabilities – then evaluations must include employees’ own self-determined perspectives of their performance and support needs (Brady et al., 2008). An evaluation model that incorporates these four employment dimensions will provide important information when developing transition and employment plans, and selecting potential job roles, work placements, performance interventions, and support. The evaluation data could establish, for example, that a supported employee’s work performance is not sufficient to maintain continued employment, and that some type of employment support (e.g., adapting materials or routines; restructuring a complex job into several single tasks) is necessary to improve that performance. Numerous models of supported and sheltered employment exist that use evaluation data to make employment decisions such as this (Brady & Rosenberg, 2002a; Nisbet & Callahan, 1987; Olney & Salomone, 1992; Wehman, 1996). However, few models of assessment exist which incorporate both employee and employer perceptions of performance and support, and the extent to which these different perceptions correspond is relatively unknown. The purpose of the study was to investigate the correspondence between employees’ and supervisors’ evaluations of work performance and support needs of people with disabilities in supported and sheltered employment. In areas where there was a lack of correspondence, we sought to establish the direction of any differences (over-estimations or under-

Education and Training in Developmental Disabilities-December 2009

estimations) of the two groups, and whether there were differences across the dimensions of performance vs. support. Method Participants and Setting Nineteen employees with developmental disabilities (11 males and 8 females) participated in this study. Fifteen of the participants were diagnosed with a cognitive impairment, and four were diagnosed with an autism spectrum disorder. All participated in supported (n ⫽ 14) or sheltered (n ⫽ 5) employment programs. The employees ranged in age from 20 years and 11 months to 54 years and 3 months (M ⫽ 36 years, 4 months). All individuals who gave consent participated in this study, and no compensation was provided for participation. All employees worked on a part-time basis, ranging from 6 hours to 30 hours per week on the job. Job tasks for the five sheltered employees included sorting, collating, and preparing mailings for local businesses; job tasks for the 14 supported employees included child and animal care, custodial and kitchen duties, bagging groceries, and working as a cashier and office assistant. At the time of this study, employees had been in their job roles from 5 months to 9 years (M ⫽ 4 years, 1 month). Dependent Measures The Job Observation and Behavior Scale (JOBS) (Rosenberg & Brady, 2000) was used to gather supervisors’ evaluations of the employees; the Job Observation and Behavior Scale: Opportunity for Self-Determination (JOBS: OSD) (Brady, Rosenberg, & Frain, 2006) was used to gather employees’ self-ratings. The two instruments comprise a system of evaluation in which both external evaluators’ (supervisors, job coaches, teachers) ratings and employees’ own self-determined ratings are used for employment and training decisions. Both instruments target critical vocational behaviors and support needs of people in community employment settings. The evaluation assists employment professionals to evaluate an individual’s work performance and support needs, then to compare these results to the norms for other peo-

ple who perform similar jobs in sheltered or supported employment. Validity, reliability, and standardization data are available for three groups: secondary students with disabilities, adults in supported and sheltered work settings, and adults in entry level community jobs (Brady & Rosenberg, 2002b; Brady et al., 2008). Both JOBS and JOBS: OSD contain the same 30 items organized into three subscales titled Work-Required Daily Living Activities, Work-Required Behavior, and Work-Required Job Duties. The three subscales and a summary of items are found in Table 1. Although the scales and items are identical on both instruments, there are alternate phrasings for the JOBS: OSD items for instances when an employee has difficulty determining what is being asked. In addition, scoring protocols differ slightly for the JOBS and the JOBS: OSD. For each item on the JOBS, the supervisor rates each employee on the Quality of Performance, Type of Support needed, and whether any Adaptive or Prosthetic Materials are needed. Quality of Job Performance is rated with a 5-point Likert-type scale with 5 indicating superior performance, 4 indicating above average performance, 3 indicating average performance, 2 indicating below average performance, and 1 indicating performance which is not acceptable for competitive employment. The Type of Support is also rated on a 5-point scale with a rating of 5 meaning that no unique supervision or support is needed beyond that provided to other workers. A rating of 4 indicates that intermittent supervision is needed from a co-worker and a rating of 3 indicates that intermittent supervision is needed from the job coach or supervisor. Ratings of 2 and 1 respectively indicate that supervision from the job coach or supervisor is needed either frequently or continuously. Because JOBS: OSD requires a self-assessment from individuals with a wide range of cognitive and communication abilities, several alterations exist for the scoring. First, each JOBS: OSD item is modified into a self-determination question format so that the information can be obtained through a standardized interview. This response format requires that data be obtained from individual employees (an internal source of information) rather than from work supervisors, job coaches, or reha-

Evaluations of Work Performance and Support Needs

/

473

TABLE 1 Summary of JOBS and JOBS: OSD Items and Subscales Subscale

Number of items

Item examples

Work-Required Daily Living

13

Attendance; Punctuality; Personal Hygiene; Travel; Communication; Money; Reading; Math; Self Identification; Work & Personal Scheduling; Work Facilities; Motivation Stress Tolerance; Interpersonal Work & Social Interactions; Changes in Routines; Honesty; Reaction to Criticism; Work Initiative & Endurance Quality & Quantity of Work; Speed of Learning New Tasks; Performance on Previously Learned & Multiple Tasks; Organization of Work Tasks; Safety; Cleanliness of Work Environment

Work-Required Behavior

8

Work-Required Job Duties

9

bilitation counselors (an external evaluation source). Second, the scoring protocol is simplified to a 3-point scale. For Quality of Performance ratings, respondents report (a) “Yes”, (b) “Sometimes”, or (c) “No, not really” to the interviewer’s query about how well the individual performs each item. For Type of Support ratings an interviewer asks respondents to select one of three options to indicate their need for support: “Can you do it by yourself?” “Can you do it with some help?” “Do you need a lot of help?”

were not present during the interviews. Employees were informed of the answer choices prior to the interview and items and answer choices were restated as needed. To administer the JOBS: OSD, the interviewer read each item to the employee. The alternate phrasing provided in the evaluation protocol was provided if an employee needed clarification on the initial wording of the item. The investigators then totaled the scores for each of the three subscales for Quality of Performance and Type of Support. Data Analysis

Procedure For each employee, a work supervisor or job coach (hereafter referred to as supervisor) completed a JOBS recording and scoring form. Each of the supervisors had worked directly with the employees previously, and had the knowledge necessary to complete the form. After the supervisors scored the individuals, the forms were returned to the investigators for data entry and analysis. The investigators tallied the total of points scored for each of the three subscales (Work-Required Daily Living Activities, Work-Required Behavior, and Work-Required Job Duties) for Quality of Performance and Type of Support. Next, each employee was interviewed by one of the authors to obtain the JOBS: OSD ratings. Interviews were conducted in a private area at the employees’ worksites. Supervisors

474

/

Data were hand scored by the lead author and then analyzed using SPSS 15.0. The means and standard deviations were computed for each of the subscales and for the overall scores for both JOBS and JOBS: OSD. Next, Pearson product-moment correlation coefficients were computed between the supervisors’ and employees’ ratings for each subscale, as well as for the overall scores, to determine the degree of correspondence between the groups. A related samples t-test was used to study any difference between employees’ and supervisors’ perceptions on employees’ Quality of Performance and Types of Support needs for each subscale for both JOBS and JOBS: OSD (WorkRequired Daily Living Activities, Work-Required Behavior, and Work-Required Job Duties). Prior to this analysis; however, scores were transformed to a standard score on a

Education and Training in Developmental Disabilities-December 2009

TABLE 2 Standardized Means, Standard Deviations, and Correlations for Subscale and Overall Scores N ⴝ 19 for Each Scale Supervisor JOBS Construct

Mean

QOP TOS

76.5 80.1

QOP TOS

68.7 70.5

QOP TOS

72.0 71.5

QOP TOS

73.3 75.4

Employee SD

Mean

SD

Work-Required Daily Living Activities subscale 15.9 83.0 12.0 16.1 74.5 18.8 Work-Required Behavior subscale 19.8 87.5 12.0 18.5 80.9 15.4 Work-Required Job Duties subscale 16.9 79.5 17.2 17.1 74.0 10.3 Total scores 14.7 82.2 7.3 14.5 75.6 13.5

Supervisor & employee correlation

⫺.13 ⫺.10 ⫺.08 .00 .19 .15 ⫺.09 ⫺.12

QOP ⫽ Quality of Performance TOS ⫽ Type of Support

linear scale from 0-100, with 0 being the lowest score and 100 being the best possible score. The formula for this transformation followed Holmes and Shea’s (1998) transformation formula: Transformed Standard Score ⫽ (100 [Total Possible Score ⫺ # of Items on Scale]) ⫻ [Actual Score ⫺ # of Items on Scale]. For example, the raw Quality of Performance score for Daily Living Activities for each employee was converted to a transformed standard score on a linear scale as follows: Transformed Standard Score for Quality of Performance for Daily Living Activities ⫽ (100 [65 ⫺ 13]) ⫻ [Actual Participant Score ⫺ 13]. Using this formula, each employee’s JOBS and JOBS: OSD raw scores were transformed to a standardized score so that their ratings could be compared. Results Nineteen pairs of surveys were completed for a total of 38 surveys (19 JOBS; 19 JOBS: OSD). Means and standard deviations for each of the three subscales, and for overall scores for each

of the instruments are provided in Table 2. The means and standard deviations for this sample were similar to the normative sample for both JOBS and JOBS: OSD (Brady et al., 2006; Rosenberg & Brady, 2000). For instance, the raw scores for individuals on the JOBS:OSD Work Related Activities of Daily Living QOP scale in our sample was 34.6 (3.1); very similar to the normative sample of Brady et al. which found raw scores of 34.3 (4.1). (For more information on normative samples please refer to Brady & Rosenberg, 2002b, and Brady et al., 2008). Also seen in Table 2 are the Pearson product-moment correlation coefficients between the two groups for each subscale. These coefficients indicated generally low correlations, and in three instances (Quality of Performance ratings for both Daily Living Activities and Work-Required Behavior, and Type of Support scores for Work-Required Behavior) showed a reverse correlation. That is, as employees’ scores improved on JOBS: OSD, the supervisors’ scores for these employees decreased on JOBS. This trend was also seen in overall scores for both scales. The subscale correlations ranged from ⫺.13 to .19 Correlation coefficients for the overall scores were ⫺.09 for Quality of Performance and ⫺.12 for

Evaluations of Work Performance and Support Needs

/

475

TABLE 3 Related Sample t-test Analysis Summary (N ⴝ 38) 95% confidence interval Measures

Mean difference Supervisor-employee

Total Score DLA BEH JD

⫺8.8 ⫺6.4 ⫺18.7 ⫺2.5

Total Score DLA BEH JD

⫺.2 ⫹5.6 ⫺10.4 ⫺7.6

Upper

Lower

Quality of Performance ⫺1.2 ⫺16.5 2.7 ⫺15.6 ⫺8.0 ⫺29.5 1.7 ⫺16.9 Type of Support 9.0 ⫺9.4 17.1 ⫺5.9 .81 ⫺21.6 1.7 ⫺16.8

t

sig

ES

⫺2.3 ⫺1.4 ⫺3.5 ⫺.5

.02 .16 .001 .66

⫺.75 ⫺.46 ⫺1.18 ⫺.55

⫺.04 ⫺.98 ⫺1.8 ⫺1.7

.97 .33 .07 .11

⫺.01 .32 ⫺.62 ⫺.14

DLA ⫽ Work-Required Daily Living Activities BEH ⫽ Work-Required Behavior JD ⫽ Work-Required Job Duties

Type of Support. These findings indicate that there was generally a very small correlation between the supervisors’ scores and the employees’ scores of themselves. None of the correlations were statistically significant (p ⬍ .05). Results of the related samples t-test comparing employees’ and supervisors’ perceptions of the employees’ ratings of their Quality of Performance and Type of Support needs for the three subscales are included in Table 3. This table shows that significant differences exist between employees’ and supervisors’ ratings in the employees’ overall Quality of Performance scores, as well as on the Work-Required Behavior subscale. Given the small sample size in this study, the statistical power necessary to find a significant difference between the two sets of ratings was low; the power to find statistical significance at the .05 level, if it existed at a medium effect size was .69. Therefore, an alternate way of looking at the results was also pursued. This involved establishing the effect sizes for the differences. Effect sizes present the differences between scores in standard deviation terms [where the Difference ⫽ (Mean of Supervisors’ Scores – Mean of Employees’ Scores/Standard Deviation)]. This allowed for more practical significance to be examined. In social science research, Cohen (1988)

476

/

indicated that the practical significance of the size of an effect should be compared to the following standards: (a) for a large effect, d ⫽.80; (b) for a medium effect, d ⫽ .50; and (c) for a small effect, d ⫽.20. In this study, effect sizes show that employees rated themselves higher in every scale, but one, when compared to their supervisors. Effect sizes ranged from a low of .14 (small effect) for the Type of Support for Work-Required Job Duties to a large effect size of 1.18 for the Quality of Performance Work-Required Behavior subscale. In all, five of the eight paired samples had effect sizes that would be considered a medium to large effect; this included Quality of Performance for Work-Required Daily Living Activities, Work-Required Behavior, WorkRequired Job Duties subscales; the Total Quality of Performance rating; and the Type of Support rating for the Work-Required Behavior subscale. Two ratings fell in the small effect size range, including the Type of Support rating for the work required Job Duties, and the Type of Support rating for the Work-Required Daily Living Activities subscale (.14 and .32, respectively). These findings indicate that supervisors’ ratings were approximately half of a standard deviation below employees’ selfdetermined ratings. Only the Type of Support overall ratings failed to show an effect. It is important to note that all comparisons indi-

Education and Training in Developmental Disabilities-December 2009

cate effect size differences in the direction indicating that supervisors were always significantly lower in their perceptions than employees, with one exception. The Type of Support rating for the Work-Required Daily Living Activities subscale was the only one in which employers scored higher than employees. Discussion Overall, these results indicate that employees and their supervisors have a very different opinion concerning the quality of work accomplished by employees with disabilities. Since the number of comparison pairs was relatively small in this initial study, significance levels are better addressed by effect sizes than statistical significance testing. In this case, most of the pair comparisons for the six subscales and two overall scores showed important differences between how supervisors rated employees and how employees rated themselves in terms of Quality of Performance. There is a similar disconnect on the Type of Support that is needed by employees to do their jobs, also demonstrated by the effect sizes. It may be that employees are overconfident of their abilities, or employers might focus more on aspects of the job with which they are not satisfied. It also may be that employees with disabilities are unaware of how their work performance is judged by employers. Alternatively, it is possible that many individuals with disabilities have inherent difficulty with skills associated with self-determination (such as self-assessment) and may not be proficient in evaluating their strengths and needs accurately. In one study that supports this notion, Wehmeyer and Schwartz (1998) reported that educational plans rarely targeted self-determination skills for secondary students with cognitive impairments, and recommended that these skills be given priority for development. In the current study, it is important to note that the results (both the large effect sizes and the low correlations) show that employers rated the quality of performance differently and lower in most every category compared to employees. Overall and subscale scores in this study were very similar to the normed sample for JOBS: OSD on all of the Quality of Performance, and Type of Support measures. This

indicates that in terms of work performance, the employees in this study were similar (all scores well within one standard deviation of the normative group) to the sample on which this measure was normed. Although when comparing this sample to others who were in work experience or supported employment (in the normed sample), the Type of Support scores for the Job-Related Duties subscale and the overall scores were somewhat higher for those in this study. Thus, it may be that this sample had slightly less work related support needs than those in the past sample. There are many uses for employment and transition assessments, particularly when assessment systems incorporate the multiple dimensions targeted in this study (productivity vs. support needs; employees’ vs. supervisors’ perceptions). The JOBS evaluation system (Brady et al., 2006; Rosenberg & Brady, 2000) is one such example, and this study indicates a new area of use for such assessments. The discrepancy scores found in this study indicate that employers and employees disagreed about the quality of work and the support needs of entry-level workers with disabilities. While not knowing who was “right” in this debate, different theories emerge. It may be that employees with disabilities overestimate their ability level and need professionals in the field to give them honest feedback concerning how their performance will be judged by future employers. One way of using JOBS to achieve this aim is to ask individuals in the early days of employment (or before being employed) to evaluate for themselves how other workers are doing, and then compare those ratings to their own self-determined ratings. An alternate explanation for the discrepancy in scores could be employers underestimating the skills of these employees. It may be that employers have preconceived ideas about how well employees with disabilities will perform on a job, and unknowingly hold a rater bias when judging the quality of employees’ work performance. It is also possible that the mere presence of a disability predisposes supervisors to watch these employees differently or more closely. It may be that any employee who is watched closely could obtain many suggestions from an employer on how to improve one’s work. Employment professionals can

Evaluations of Work Performance and Support Needs

/

477

share this information with employers, and perhaps have them rate other very successful and marginally successful employees on the job. These ratings could be compared to the individual with a disability to understand whether there is a need for improved performance, or whether the employee is performing at an acceptable or normative rate. The different ratings obtained by employees and supervisors can be used to identify changes needed for successful employment to be maintained. Employment professionals, for example, could use these scores to create specific target goals based on these assessment scores. For instance, if an employee reported that his or her job was conducted with a high degree of safety, but the supervisor reported that this only happens some of the time, the two can work with a job coach so that both understand what constitutes safe job performance. They could then develop an intervention to reach that goal. Support interventions can be developed in a similar fashion, with the goal of educating community employers about the types of supports that are available to help employees successfully perform their jobs. For instance, if employees identify that they always need help to get back from breaks on time, the employment professional can discuss with the supervisor and employees the various ways to get the help needed to get back to work on time. For example, a coworker can be recruited to remind an employee when break time is over, or an alarm watch might be used as an adaptation to remind an employee that it is time to return to work. Finally, employment professionals can think about using supervisors’ and employees’ scores on the JOBS system as a measure of when successful employment has been achieved. Currently, many people with disabilities only receive employment support on a temporary basis, and lose such support when a rehabilitation counselor affirms that the employment status of the person is stable. This leads to many cases being closed before the individual is in a successful long-term employment situation. By including JOBS and JOBS: OSD ratings into an Individual Plan for Employment (IPE), these data could be used to determine whether the individual with a disability is stable in his or her work behavior,

478

/

and likely to continue success on the job (as indicated by minimum performance scores, and the support scores needed to maintain success). By inputting these criteria into an IPE, employment professionals can be more confident that employees are not losing jobs due to inaccurate perceptions of work skills and premature termination of services (Martin et al., 2006). Historically, people with disabilities have had difficulty obtaining and maintaining community employment (Brady & Rosenberg, 2002a; Levy et al., 1993; Murphy et al., 2002). There are many factors that contribute to this situation. These factors include providing and fading support (Rogan, Luecking, & Grossi, 2007), funding for supported employment efforts (Rusch & Braddock, 2004), and integrating employees with disabilities into the culture of the workplace (Rogan, Callahan, Griffin, & Hammis, 2007). An additional variable that may impact the employment status of people with disabilities is the degree to which their self-determined evaluations of their own work performance and support needs correspond with those of their supervisors’ evaluations. Thus, it seems prudent for supervisors, employment professionals, and related service providers to obtain the self-evaluations of employees with disabilities (see Brady et al., 2008; Lynn et al., 2004), and compare those evaluations to supervisors’ evaluations. In addition to learning the employees’ perspective, such information may highlight any existing discrepancies and allow supervisors and employees to address these areas of concern prior to any changes to the individual’s employment status.

References Abrams, K., DonAroma, P., & Karan, O. C. (1997). Consumer choice as a predictor of job satisfaction and supervisor ratings for people with disabilities. Journal of Vocational Rehabilitation, 9, 205–215. Anthony, W. A. (1994). The vocational rehabilitation of people with severe mental illness: Issues and myths. Innovations and Research, 3(2), 17–24. Brady, M. P., & Rosenberg, H. (2002a). Modifying and managing employment practices: An inclusive model for job placement and support. In K. Storey, D. Hunter, & P. Bates (Eds.), The road ahead: Transition to adult life for persons with disabil-

Education and Training in Developmental Disabilities-December 2009

ities (pp. 119 –135). St. Augustine, FL: Training Resource Network. Brady, M. P., & Rosenberg, H. (2002b). Job Observation and Behavior Scale: A supported employment assessment instrument. Education and Training in Mental Retardation and Developmental Disabilities, 37, 427– 433. Brady, M. P., Rosenberg, H., & Frain, M. (2008). A self-evaluation instrument for work performance and support needs. Career Development for Exceptional Individuals, 31, 175–185. Brady, M. P., Rosenberg, H., & Frain, M. (2006). Job Observation and Behavior Scale: Opportunity for SelfDetermination. Wood Dale, IL: Stoelting Co. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Graffam, J., Shinkfield, A., Smith, K., & Polzin, U. (2002). Factors that influence employer decisions in hiring and retaining an employee with a disability. Journal of Vocational Rehabilitation, 17, 175– 181. Griffin, D., Rosenberg, H., Cheyney, W., & Greenberg, B. (1996). A comparison of self esteem and job satisfaction of adults with mild mental retardation in sheltered workshops and supported employment. Education and Training in Mental Retardation and Developmental Disabilities, 31, 142–151. Grossi, T., Schaaf, L., Steigerwald, M., & Mank, D. (2002). Adult employment. In K. Storey, P. Bates, & D. Hunter (Eds.), The road ahead: Transition to adult life for persons with disabilities (pp. 101–120). St. Augustine, FL: Training Resource Network. Hamilton, M., & Shumate, S. (2005). The role and function of certified evaluation specialists. Journal of Rehabilitation, 71(1), 5–19. Holmes, W. C., & Shea, J. A. (1998). A new HIV/ AIDS-Targeted quality of life (HAT-QoL) instrument. Medical Care, 36, 138 –154. Hurlbutt, K., & Chalmers, L. (2004). Employment and adults with Asperger Syndrome. Focus on Autism and Developmental Disabilities, 19, 215–222. Kim, K., & Morningstar, M. E. (2005). Transition planning involving culturally and linguistically diverse families. Career Development for Exceptional Individuals, 28, 92–103. Kolstoe, O. P. (1961). An examination of some characteristics which discriminate between employed and not employed mentally retarded males. American Journal of Mental Deficiency, 66, 472– 482. Krantz, G. (1971). Critical vocational behaviors. Journal of Rehabilitation, 37(4), 32–34. Levy, J., Jessop, D., Rimmerman, A., Francis, F., & Levy P. (1993). Determinants of attitudes of New York state employers towards the employment of persons with severe handicaps. Journal of Rehabilitation, 59(1), 49 –54. Lynn, S., Sumsion, T., McWilliam, C., & MacKin-

non, J. (2004). Service provider perspectives on including consumers in the vocational rehabilitation process. Journal of Vocational Rehabilitation, 21, 123–136. Mancuso, L. (1990). Reasonable accommodations for workers with psychiatric disabilities. Psychosocial Rehabilitation Journal, 14(2), 3–19. Mank, D., Cioffi, A., & Yovanoff, P. (1998). Employment outcomes for people with severe disabilities: Opportunities for improvement. Mental Retardation, 36, 205–216. Martin, J. E., & Huber Marshall, L. H. (1996). Choicemaker: Infusing self-determination instruction into the IEP and transition process. In D. J. Sands & M. L. Wehmeyer (Eds.), Self-determination across the lifespan (pp. 215–236). Baltimore: Paul H. Brookes. Martin, J. E., Van Dycke, J. L., Christensen, W. R., Greene, B. A., Gardner, J. E., & Lovett, D. L. (2006). Increasing student participation in IEP meetings: Establishing the self-directed IEP as an evidence-based practice. Exceptional Children, 72, 299 –316. McDermott, S., Martin, M., & Butkus, S. (1999). What individual, provider, and community characteristics predict employment of individuals with mental retardation? American Journal on Mental Retardation, 104, 346 –355. Menchetti, B. M., & Garcia, L. A. (2003). Personal and employment outcomes of person-centered career planning. Education and Training in Developmental Disabilities, 38, 145–157. Morgan, R. L., & Alexander, M. (2005). The employer’s perception: Employment of individuals with developmental disabilities. Journal of Vocational Rehabilitation, 23, 39 – 49. Muller, E., Schuler, A., Burton, B. A., & Yates, G. B. (2003). Meeting the vocational support needs of individuals with Asperger Syndrome and other autism spectrum disabilities. Journal of Vocational Rehabilitation, 18, 163–175. Murphy, S. T., Rogan, P. M., Handley, M., Kincaid, C., & Royce-Davis, J. (2002). People’s situations and perspectives eight years after workshop conversion. Mental Retardation, 40, 30 – 40. Nisbet, J., & Callahan, M. (1987). Achieving success in integrated work places: Critical elements in assisting persons with severe disabilities. In S. Taylor, D. Biklen, & J. Knoll (Eds.), Community integration for persons with severe disabilities (pp. 184 – 201). New York: Teacher’s College Press. Olney, M., & Salomone, P. (1992). Empowerment and choice in supported employment: Helping people to help themselves. Journal of Applied Rehabilitation Counseling, 23(3), 41– 44. Reisman, E. S., & Reisman, J. I. (1993). Supervision of employees with moderate special needs. Journal of Learning Disabilities, 26, 199 –206.

Evaluations of Work Performance and Support Needs

/

479

Rogan, P., Banks, B., & Howard, M. (2000). Workplace supports in practice: As little as possible, as much as necessary. Focus on Autism and Other Developmental Disabilities, 15, 2–11. Rogan, P., Callahan, M., Griffin, C., & Hammis, D. (2007). Toward full citizenship: New directions in employment for people with significant disabilities. In P. M. Walker & P. Rogan (Eds.), Make the day matter!: Promoting typical lifestyles for adults with significant disabilities (pp. 35–56). Baltimore, MD: Paul H. Brookes Publishing Co. Rogan, P., Luecking, R., & Grossi, T. (2007). Preparing for meaningful adult lives through school and transitions experiences. In P. M. Walker & P. Rogan (Eds.), Make the day matter!: Promoting typical lifestyles for adults with significant disabilities (pp. 15–34). Baltimore, MD: Paul H. Brookes Publishing Co. Rosenberg, H., & Brady, M. P. (2000). JOBS: Job Observation and Behavior Scale: A work performance evaluation for supported and entry level employees. Wood Dale, IL: Stoelting Co. Rusch, F. R., & Braddock, D. (2004). Adult day programs versus supported employment (1988 – 2002): Spending and service practices of mental retardation and developmental disabilities state agencies. Research and Practice for Persons with Severe Disabilities, 29, 237–242. Shaw, L. R., McMahon, B. T., Chan, F., & Hannold, E. (2004). Enhancement of the working alliance: A training program to align counselor and consumer expectations. Journal of Vocational Rehabilitation, 20, 107–125. Skinner, A. L. (2003). Using locus of control to improve self-determination and outcomes for consumers with learning disabilities. Journal of Applied Rehabilitation Counseling, 34(2), 27–32. Stodden, R. (1998). School-to-work transition: Overview of disability legislation. In F. R. Rusch & J. G.

480

/

Chadsey (Eds.), Beyond high school: Transition from school to work (pp. 60 –76). Belmont, CA: Wadsworth Publishing. Storey, K. (2002). Strategies for increasing interactions in supported employment settings: An updated review. Journal of Vocational Rehabilitation, 17, 231–237. Targett, P. S., Wehman, P. H., McKinley, W. O., & Young, C. L. (2004). Successful work supports for persons with SCI: Focus on job retention. Journal of Vocational Rehabilitation, 21, 19 –26. Thoma, C. A., Williams, J. M., & Davis, N. J. (2005). Teaching self– determination to students with disabilities: Will the literature help? Career Development for Exceptional Individuals, 28, 104 –115. Valenzuela, R. L., & Martin, J. E. (2005). Self-directed IEP: Bridging values of diverse cultures and secondary education. Career Development for Exceptional Individuals, 28, 4 –14. Wehman, P. (2006). Individualized assessment planning: Putting self-determination into action. In P. Wehman (Ed.), Life beyond the classroom: Transition strategies for young people with disabilities. Baltimore: Brookes. Wehman, P. (1996). Supported employment: Inclusion for all in the workplace. In W. Stainback & S. Stainback (Eds.), Controversial issues confronting special education: Divergent perspectives (pp. 293– 304). Needham Heights, MA: Allyn & Bacon Wehmeyer M. L., & Schwartz, M. (1997). Self-determination and positive adult outcomes: A follow-up study of youth with mental retardation or learning disabilities. Exceptional Children, 63, 245– 255.

Received: 19 March 2008 Initial Acceptance: 17 May 2008 Final Acceptance: 26 September 2008

Education and Training in Developmental Disabilities-December 2009