Community Mental Health Journal, Vol. 33, No. 6, December 1997

A Comparison of Clinical and Structured Interview Diagnoses in a Homeless Mental Health Clinic Carol S. North, M.D. David E. Pollio, Ph.D. Sanna J. Thompson, M.S.W. Daniel A. Ricci, CM., L.C.S.W. Elizabeth M. Smith, Ph.D. Edward L. Spitznagel, Ph.D.

ABSTRACT: Objective. This study compared psychiatric diagnoses ascertained by independent clinicians with structured research interviews of homeless psychiatric patients assessed in a mental health clinic and in the community. Problems of both overdiagnosis and underdiagnosis in structured research interviews compared to clinician assessment were predicted. Method. Over a period of a year, 97 patients referred to a mental health clinic for homeless people were assessed with the Diagnostic Interview Schedule (DIS) administered by a clinical social worker who then completed a full clinical psychiatric social work assessment. These same patients received a thorough and systematic clinical psychiatric evaluation by a psychiatrist or psychologist, both experienced with this population. These clinicians gathered data from multiple sources, often with extended observation over time. The DIS and clinician diagnoses were made blind to one another and then compared; the clinician was often made aware of some of the symptoms that the social worker had elicited, but not whether the elicited material was from the DIS or from the clinical assessment. Diagnoses of 33 clinic patients previously assessed by trained nonclinician DIS interviews in an epidemiologic study of the homeless population in the community were also compared to clinician diagnoses, All the authors are affiliated with Washington University, St. Louis, Missouri, except for Daniel A. Ricci, who is affiliated with the Grace Hill Neighborhood Health Center, St. Louis, Missouri. Dr. Smith died March 7, 1997. Address correspondence to Carol S. North, M.D., Washington University, School of Medicine, Department of Psychiatry, 4940 Children's Place, St. Louis, MO 63110. This research was supported by National Institute on Alcohol Abuse and Alcoholism Grant #AA007549 to Dr. Smith. 531

© 1997 Human Sciences Press, Inc.

532

Community Mental Health Journal

and no information from these patients' survey DIS interviews was made available to the clinicians. Results. Compared to clinician assessment, structured interviews underdiagnosed antisocial personality disorder (ASPD) and overdiagnosed major depression. Alcohol use disorder and schizophrenia showed only small discrepancies by assessment method. Drug use disorder revealed no bias according to method of ascertainment, but showed very discrepant kappa levels comparing DIS to clinician assessment in the two different comparison contexts. Conclusions. If structured research methods assessing the homeless population actually overestimate depression, underestimate ASPD, and misclassify drug abuse, then policies stemming from structured interview research recommendations may call for levels and types of services not optimally suited to the reality of this population's needs. Because mental illness and substance abuse are thought to be critical factors in the generation and perpetuation of homelessness, the issue of accurate diagnosis is tantamount to understanding and providing workable solutions to the problem of homelessness. Further research is needed to untangle potential confounders of the homeless situation to psychiatric diagnosis.

INTRODUCTION The last decade of research into homelessness has benefitted from increased attention to systematic study of psychiatric disorders in this population. Random, representative sampling methods and systematic assessment of psychiatric disorders of the homeless have greatly improved scientific confidence in the findings. For example, rates of schizophrenia in this population, previously estimated at 50% to 75% (Torrey, 1986; Lipton, Sabatini, & Katz, 1983) by less sophisticated methods, have subsequently been honed to a more realistic 5% to 15% (Bassuk, Rubin, & Lauriat, 1986; Koegel, Burnam, & Farr, 1988; Breakey, et al. 1989; Susser, Struening, & Conover, 1989; Herrman, McGorry, Bennett, van Riel, & Singh, 1989). Although methodologic improvements have advanced the field significantly, current emphasis on use of structured instruments has preceded investigation into the validity of such instruments in this special population. Much of the existing knowledge about mental illness in the homeless population has derived from studies based on structured interviews (Koegel, Burnam, & Farr, 1988; Koegel & Burnam, 1988; Smith, North, & Spitznagel, 1992; Smith, North, & Spitznagel, 1993) such as the Diagnostic Interview Schedule (DIS) (Robins, Helzer, Croughan, Williams, & Spitzer, 1981). Questions have been raised about potential lack of sensitivity of the DIS to psychiatric disorders such as schizophrenia and depression in homeless populations (Fischer, Shapiro, Breakey, Anthony, & Kramer, 1986; Breakey, et al. 1989; Fischer, 1989). Drake and colleagues emphasize that valid diagnostic assessment is critical for study of the homeless population. Both Drake's group

Carol S. North, M.D., et al.

533

(Drake, Osher, & Wallach, 1991) and Fischer (1989) have pointed out that standardized instruments need to be normalized for homeless populations. Drake and colleagues (Drake, Osher, & Wallach, 1991) stressed the need to aggregate observations over time and situation, collect information from collateral sources, and modify standard instruments for assessment of behavioral dimensions that are relevant to this population to maximize validity of diagnosis. Susser and Struening (1990) identified a need to validate structured interview assessment through longitudinal observation and consideration of information from multiple sources who are familiar with homeless subjects. Fischer (1989) observed that prevalence estimates from standardized scales overdiagnose psychiatric disorders relative to psychiatric examination in homeless populations. Although concern has been expressed that antisocial personality disorder (ASPD) may be overdiagnosed in homeless populations due to confounders of the homeless situation (Koegel & Burnam, 1992), North and colleagues (1993) have documented the validity of the diagnosis of ASPD ascertained by DIS interview in this population. Potentially confounding effects of stressors inherent in homelessness may generate difficulties in distinguishing stress-related behavior from psychopathology. For example, distinguishing state-dependent distress (e.g., odd appearance, deteriorated hygiene, dysphoria and demoralization) from enduring psychopathology (e.g., psychosis, major depression) may be very difficult (Susser, Conover, & Struening, 1989). North (North, 1995) reported that homelessness-related situational factors such as miserable weather on the day of interview increase the likelihood of a diagnosis of major depression. Therefore, use of structured interviews and assessment scales devised for other populations may account for some portion of overdiagnosis of psychiatric illness in homeless settings. Structured instruments may also have problems with underdiagnosis. Fischer and colleagues (1992) have observed substantial utilization of psychiatric services among homeless people with no history of major mental illness or substance use disorder. North and coworkers (1994) found that more than half of homeless people reporting a past suicide attempt described no history of major depression. Koegel and Burnam (1992) documented an 8.5 times increase in risk for suicide attempts among nondepressed homeless people over a domiciled population. While these findings might be considered evidence of underdiagnosis of major psychiatric disorders in homeless populations, alternatively, suicidality and utilization of psychiatric services could also

534

Community Mental Health Journal

be considered indicators of the magnitude of psychological distress and general misery in this population that does not fit traditional definitions of psychiatric disorder. Further study is needed to compare the performance of structured interviews such as the DIS with clinician ratings in the homeless population. Therefore, the purpose of the present study was to compare clinician diagnoses with diagnoses made by a standardized instrument to determine differences in rates of diagnoses as well as disagreement on individual cases. Based on evidence in the literature, it was hypothesized that certain disorders might be overdiagnosed by standardized instruments relative to clinical assessment, while others might be underdiagnosed, and both forces might be operative in some diagnoses. Specifically, it has been predicted (North, 1995) that major depression in homeless subjects will be overdiagnosed by structured instruments. The current study also predicted that ASPD and substance use disorders will be underdiagnosed by a single cross-sectional research interview due to subjects' desire to create positive impressions. The diagnosis of schizophrenia, because of misperception of adaptation to homelessness as psychosis, as well as denial of symptoms on a crosssectional structured interview assessment, is expected to show elements of both overdiagnosis and underdiagnosis relative to clinician assessment, manifested by agreement in rates but disagreement on which individuals constitute the cases. METHODS Two homeless samples, in which structured interviews were compared with clinician assessments, comprise the data base for this study. In the first data set, both assessments occurred in close temporal proximity (2 weeks) of one another in a mental health clinic. In the second, assessments in the mental health clinic were compared with structured interview diagnoses made in a separate homeless population study.

Sample 1: Comparison of DIS and Clinician Assessment in a Mental Health Clinic In 1988, the Health Care for the Homeless Program at the Grace Hill Neighborhood Health Center in St. Louis set up a mental health clinic to serve homeless individuals. The mental health team for the clinic, composed of three of the authors of this paper (C.S.N., E.M.S., D.A.R), evaluated patients in the clinic and on visits to shelters. These patients were referred from a variety of medical and social service sources in the city, and referrals were not connected with the survey study to be described later in this paper. During the first year of this program's operation, patients were initially screened and assessed by the team social worker (D.A.R.) who administered the DIS as part of the routine evaluation. Depending on whether the services of a psychiatrist (C.S.N.) or a

Carol S. North, M.D., et al.

535

psychologist (E.M.S.) were more immediately appropriate, the patients were then referred respectively for treatment. Clinician diagnoses were made according to DSM-III-R criteria by the psychiatrist or the psychologist, who were blind to diagnoses from the DIS research interview. The clinicians, unlike the social worker, had considerable experience in psychiatric assessment and treatment of homeless individuals. They also had the luxury of assessment over time (usually with multiple clinic visits, sometimes over a period of years from the initiation of the study) and the advantage of multiple sources of information. Clinical data were gathered from as many sources as possible, including medical records from other treatment centers and information from shelter and social service personnel who often had extensive contact with the individuals. In the first year of the clinic's existence, 97 consecutive patients received a DIS interview and a clinician interview in the clinic.

Sample 2: Comparison of Clinician Assessment with DIS Data from a Community Homeless Population Study In 1989-1990, 600 men and 300 women were randomly sampled for an epidemiologic study of the homeless population from all overnight and daytime shelters located in the city of St. Louis that serve the homeless, as well as locations on the street or other public areas where the homeless are known to congregate. Sampling was conducted proportionally to the numbers of persons in the various programs. Lists of guests of shelters and day centers were randomized, and sets of random numbers were generated by computer to select subjects proportionately to center size from daily logs of attendance. This random sampling procedure provided this study with a sample that is believed to be truly representative of homeless men and women in St. Louis who utilize shelters and day centers. The majority (70%) of the 600 male subjects and all of the 300 women were sampled from shelters. From overnight shelters, 195 men and 251 women were interviewed; 150 men and 29 women were interviewed from day centers; 76 men were sampled from specialized rehabilitation programs; and 20 women were sampled from 24-hour emergency shelters. The remaining male subjects (N = 179) were recruited systematically from streets, parks, and other public areas. Women were not available in large enough numbers in these settings to permit sampling there. The DIS was administered in the field survey sample by trained nonclinician interviewers. The completion rate for the men was 91% and for the women was 96%. More details of the sampling methods utilized in this study are provided elsewhere (Smith, North, & Spitznagel, 1992; Smith, North, & Spitznagel, 1993; Smith, North, & Spitznagel, 1991). Comparison of names from the list of study subjects to the roster of clinic patients found that 33 subjects from the Grace Hill mental health clinic had participated in the homeless population study. Clinical diagnoses in this group of 33 made by the psychiatrist and the psychologist were blind to all data from the DIS interviews in the population study.

Data Analysis Diagnoses included in the data analyses were schizophrenia, major depression, alcohol and drug use disorders, and ASPD, which were selected for their prevalence and significance in homeless populations in the published literature. Clinician diagnoses were compared with DIS diagnoses separately for the 97 clinic patients who received the DIS in the clinic and the 33 clinic patients who had received the DIS in the community survey study. Reliability estimates on diagnoses were made

536

Community Mental Health Journal

by kappa comparisons. (Strictly speaking, kappa is a measure of test-retest reliability, meaning that the two measurement methods should be equivalent; nonetheless, kappa has been used successfully in a variety of contexts in reporting agreement between trained nonclinician and clinician raters.) Where prevalence rates of diagnoses were below 20%, the kappa statistic is negatively affected and considered by some to be an unacceptable measure. When this occurred, following Spitznagel and Helzer (1985), a Y statistic (coefficient of colligation) was also calculated (Yule, 1912). Y scores approximate what the kappa value would have been had the prevalence been within optimal range. Kappa values above .75 are considered excellent, .4 to .75 fair to good, and less than .4 poor (Fleiss, 1981). To examine partiality of DIS diagnosis relative to the clinician as standard, McNemar's chi square was calculated for each diagnosis. In the absence of a gold standard for psychiatric diagnostic validity in the homeless population, clinician diagnoses yielded more opportunities for the inclusion of data from multiple sources and over time, and were therefore used as the comparison standard for this study. If the proportion of differences between the DIS and clinician showed directional tendencies (i.e., either the clinician or the nonclinician DIS tended to make a diagnosis more often than the other), then the McNemar test would find a differential diagnostic prevalence. For example, if the DIS was consistently positive and clinician consistently negative, negative clinician partiality would be described. McNemar's chi square does not register disagreement if it is evenly distributed in positive and negative directions. Statistical significance for the McNemar's x2 was set at the .05 level.

RESULTS Demographic characteristics of the two samples were quite similar. Because the survey sample was prospectively designed to be twothirds male, the gender rates differ from the clinical sample, which was two-thirds (68%) female. Both samples were predominantly young (men about 35 and women about 30 years of age), African-American (75% of survey and 69% of clinic subjects), and unmarried (>90% of both samples), and averaged an eleventh grade level of education. Table 1 presents comparison of DIS diagnoses with clinician diagnoses both made in the clinic. Schizophrenia and substance use disorders had acceptable (Y and kappa .5 to .6 range) reliability, ASPD only fair (kappa = .4), and major depression poor (kappa < .4) reliability. ASPD and depression were the only two diagnoses showing significant differences in diagnostic prevalence by diagnostic method. The clinician diagnosed many more cases of ASPD (McNemar's x2 = 12.96, df = 1, p < .001), and the DIS diagnosed more cases of major depression (McNemar's x2 = 17.45, df = 1, p < .001). Diagnoses by the clinician versus nonclinician DIS survey diagnoses on the 33 population study subjects who were seen in the clinic are summarized in Table 2. In these comparisons, alcohol use disorder and schizophrenia had acceptable (kappa and Y >.6) reliability and the

538

Community Mental Health Journal

other three diagnoses had poor (kappa < .4) reliability. The one diagnosis with differential diagnostic prevalence was ASPD, which again was much more often diagnosed by the clinician (McNemar's x2 = 5.82, df = 1, p < .05). The nonclinician-administered DIS identified a marginal but nonsignificant excess of major depression. While the reliability of drug diagnoses was very poor in this comparison context, the two diagnostic methods identified similar rates of diagnoses, although identifying different subjects for diagnosis, and without directional partiality. DISCUSSION This study compared psychiatric diagnoses of homeless clinic patients made by clinicians who have considerable experience in working with homeless populations with diagnoses ascertained by the DIS (by a clinician DIS rater inexperienced with the homeless in one comparison, and by trained nonclinician DIS administrators in another). Compared to experienced clinician assessment of homeless psychiatric patients, structured interviews underdiagnosed ASPD and overdiagnosed major depression. Alcohol use disorder and schizophrenia showed only small discrepancies by assessment method. Drug use disorder showed no directional partiality according to method of assessment, but reliability of the diagnosis comparing DIS to clinician assessment yielded very discrepant kappa levels in the two different comparison contexts. The hypothesis that ASPD would be underdiagnosed by the DIS relative to the clinician was upheld, ais demonstrated by the low kappas and significant directional partiality detected across both comparisons. It is therefore likely that the experienced clinician procedure of obtaining information from multiple sources and by observation over time yields significant gains in data not obtained in a single face-to-face interview that depends entirely upon the subject's candidness. Similarly, the hypothesis that major depression would be overdiagnosed by the DIS was strongly supported by the findings of low kappas found in both comparisons and significant directional partiality found in the first comparison. This finding agrees with previous speculation that the diagnosis of major depression in this population may be confounded by the miserable conditions of homelessness that the structured nature of the DIS on cross section cannot sort out in the way that a clinician experienced with this population can. Clinical experience of this team has been that homeless people presenting for treat-

Carol S. North, M.D., et al.

539

540

Community Mental Health Journal

merit of depressive complaints seem to present with a context-oriented set of symptoms that temporally parallels the course of homelessness. To determine whether this syndrome clinically simulating major depression in cross section represents a valid case of major depression as it presents in other populations, external validators such as consistency on follow-up especially after stable housing is achieved, family history, and response to treatment for depression should be sought in future studies. The hypothesis that substance use disorders are systematically underdiagnosed by the DIS was not supported by the data. Alcohol use disorder showed the highest kappa of all five disorders examined, and no directional partiality of method of diagnosis was ascertained. Drug use disorder also showed no directional partiality, and in the first comparison the kappa was relatively good; however, in the comparison of nonclinician DIS diagnoses made in the field with clinician diagnoses, the kappa was the lowest of all reliability statistics across either set of comparisons. This lack of directional partiality by method for this diagnosis could indicate that subjects were equally likely to conceal their drug history from the DIS interviewer in the field (potentially due to fear of being expelled from programs if this history is learned) as they were to conceal it from the clinician (due to desire to make a positive impression). In both comparisons, the clinician and the DIS assessments diagnosed similar overall rates of drug use disorder, but the rates of disagreement indicate that the two methods of assessment often identified different cases. Part of the discrepancy in drug diagnoses may be due to the length of time between interviews, sometimes up to four years apart in the field-DIS assessed sample (while the clinic-DIS assessed sample received both interviews within a week or two of one another). This is supported by the 41% rate of disagreement in the field-DIS assessed sample, compared to only 16% disagreement in the clinic-DIS assessed sample. The combined rate of cases diagnosed by the clinician and/or DIS in the field-DIS assessed sample was 56%, much higher than the clinic-DIS assessed rate of 39%. This suggests that in different settings and in separate time periods subjects' stories change, and the yield may be optimized by combining information from as disparate of sources and time periods as possible. The final hypothesis, that schizophrenia would show no significant bias, was upheld, but the expectation of unacceptably low reliability was not borne out in either set of comparisons. Therefore the DIS assessment of schizophrenia in this population may be functionally

Carol S. North, M.D., et al.

541

equivalent to clinician diagnosis. Situational factors of homelessness do not appear to confound structured diagnostic assessment of schizophrenia any more than they confuse the seasoned clinician. Strengths of this study included the standardized definition of homelessness and random sampling of field subjects from shelters, day centers, and street settings, as well as the systematic interviewing with the DIS. The expertise of the clinicians with this population and the opportunity to collect information over time and from many sources argue in favor of their role as the standard of comparison in this study. Probably the greatest limitation to this study is that because it assessed psychiatric disorders only among clinic attenders, the findings may not generalize to assessment of other homeless populations. The study was also limited in its assessment of only five psychiatric diagnoses, due to limited prevalence of diagnoses, and in assessment of lifetime as opposed to current diagnosis. Finally, the sample was relatively small and inter-rater reliability of clinicians was not ascertained. Therefore, attention to these limitations is needed in future research. The findings of this study suggest the need for more research in diagnostic assessment of homeless populations to further clarify the effects of potential confounders of the homeless situation to psychiatric diagnosis, especially for the diagnoses of major depression and ASPD, and possibly drug use disorder. Because it cannot be determined whether the diagnostic discrepancies found in this study were due to differences in evaluator expertise with the homeless population or to differences in assessment method, future studies might consider controlling for level of clinician expertise with homeless populations. Studies to collect repeated diagnoses over time across changes in the individuals' homelessness status might help clarify the impact of homelessness on validity of diagnosis. A potentially fruitful next step in diagnostic methodology might be to study the effects of the findings of incorporation of methodological elements of multiple sources of data, longitudinal observation of diagnostic information and its interplay with homelessness, and degree of assessors' clinical experience with this population. Drake and colleagues (Drake, Alterman, & Rosenberg, 1993) have pointed out that diagnostic dilemmas created by unrecognized presence of substance abuse include misdiagnosis of other disorders, inappropriate treatment with medications, neglect of appropriate interventions for substance abuse, and inappropriate treatment planning and referral. These researchers concluded that when substance abuse is

542

Community Mental Health Journal

not addressed, mental health treatment for dually diagnosed individuals is "markedly ineffective." In the homeless population, where mental illness and substance abuse are thought to be critical factors in the generation and perpetuation of homelessness, the issue of proper diagnosis is tantamount to understanding and providing workable solutions to the problem of homelessness. If nonclinician structured assessment of the homeless population actually overestimates depression, underestimates ASPD, and misclassifies drug abuse, then policies stemming from these research recommendations may call for levels and types of services not optimally suited to the reality of this population's needs. Overestimation of major mental illness in the homeless population may lead to policies overemphasizing and isolating mental illness in service development while ignoring the very stressors that may be causing the overestimation by standardized instruments. Underestimation of Axis II disorders and substance abuse in this population will interfere with therapy of other disorders and delay recognition of need for substance abuse treatment, probably the greatest psychiatric treatment need of this population. The findings of this study suggest the need for interpretation of findings by clinicians experienced with this population in recommending program designs. Further work is needed by clinicians who are both experienced in homelessness and in methods of academic research on psychiatric diagnosis. REFERENCES Bassuk, E.L., Rubin, L., & Lauriat, A.S. (1986). Characteristics of sheltered homeless families. American Journal of Public Health, 76(9), 1097-1101. Breakey, W.R., Fischer, P.J., Kramer, M., Nestadt, G., Romanoski, A.J., Ross, A., Royall, R.M., & Stine, O.C. (1989). Health and mental health problems of homeless men and women in Baltimore. The Journal of the American Medical Association, 262(10), 1352-1357. Drake, R.E., Alterman, A.I., & Rosenberg, S.R. (1993). Detection of substance use disorders in severely mentally ill patients. Community Mental Health Journal, 29, 175-192. Drake, R.E., Osher, EC., & Wallach, M.A. (1991). Homelessness and dual diagnosis. American Psychologist, 46(11), 1149-1158. Fischer, P.J. (1989). Estimating the prevalence of alcohol, drug and mental health problems in the contemporary homeless population: A review of the literature. Contemporary Drug Problems, 16(3), 333-390. Fischer, P.J., Drake, R.E., & Breakey, W.R. (1992). Mental health problems among homeless persons: A review of epidemiological research from 1980 to 1990. In H.R. Lamb, L.L. Bachrach, & F.I. Kass (Eds.), Treating the Homeless Mentally III: A Report of the Task Force on the Homeless Mentally III. Washington, DC: American Psychiatric Association, (pp. 7593). Fischer, P.J., Shapiro, S., Breakey, W.R., Anthony, J.C., & Kramer, M. (1986). Mental health and social characteristics of the homeless: A survey of mission users. American Journal of Public Health, 76(5), 519-524.

Carol S. North, M.D., et al.

543

Fleiss, J. (1981), Statistics for rates of proportions. New York: John Wiley & Sons. Herrman, H., McGorry, P., Bennett, P., van Riel, R., & Singh, B. (1989). Prevalence of severe mental disorders in disaffiliated and homeless people in inner Melbourne. American Journal of Psychiatry, 146, 1179-1184. Koegel, P., & Burnam, A. (1988). Alcoholism among homeless adults in the inner city of Los Angeles. Archives of General Psychiatry, 45, 1011-1018. Koegel, P., Burnam, A., & Farr, R.K. (1988). The prevalence of specific psychiatric disorders among homeless individuals in the inner city of Los Angeles. Archives of General Psychiatry, 45, 1085-1092. Koegel, P., & Burnam, M.A. (1992). Problems in the assessment of mental illness among the homeless: An empirical approach. In M.J. Robertson & M. Greenblatt (Eds.), Homelessness: A National Perspective. New York: Plenum, (pp. 77-99). Lipton, F.R., Sabatini, A., & Katz, S.E. (1983). Down and out in the city: The homeless mentally ill. Hospital and Community Psychiatry, 34, 817-821. North, C.S. (1995). Homelessness: Psychiatric and Cultural Dimensions. American Psychiatric Association presentation in Miami, Florida. North, C.S., Smith, E.M., & Spitznagel, EX. (1993). Is antisocial personality a valid diagnosis among the homeless? American Journal of Psychiatry, 150, 578-583. North, C.S., Smith, E.M., & Spitznagel, EX. (1994). Violence and the homeless: an epidemiologic study of victimization and aggression. Journal of Traumatic Stress, 7(1), 95—110. Robins, L.N., Helzer, J.E., Croughan, J., Williams, J.B.W., & Spitzer, RX. (1981). NIMH Diagnostic Interview Schedule: Version III (May 1981). National Institute of Mental Health. Smith, E.M., North, C.S., & Spitznagel, EX. (1991). Are hard-to-interview street dwellers needed in assessing psychiatric disorders in homeless men? International J of Methods in Psychiatric Res. 1, 69-78. Smith, E.M., North, C.S., & Spitznagel, EX. (1992). A systematic study of mental illness, substance abuse, and treatment in 600 homeless men. Annals of Clinical Psychiatry, 4(2), 111120. Smith, E.M., North, C.S., & Spitznagel, EX. (1993). Alcohol, drugs, and psychiatric comorbidity among homeless women: An epidemiologic study. Journal of Clinical Psychiatry,34, 82-87. Spitznagel, EX., & Helzer, J.E. (1985). A proposed solution to the base rate problem in the kappa statistic. Archives of General Psychiatry, 42, 725-728. Susser, E., Conover, S., & Struening, EX. (1989). Problems of epidemiologic method in assessing the type and extent of mental illness among homeless adults. Hospital and Community Psychiatry, 40, 261-265. Susser, E., Struening, EX., & Conover, S. (1989). Psychiatric problems in homeless men. Archives of General Psychiatry, 46, 845-850. Susser, E.S., & Struening, EX. (1990). Diagnosis and screening for psychotic disorders in a study of the homeless. Schizophrenia Bulletin, 16, 133-145. Torrey, E.F. (1986). Forced medication is part of the cure. The New Physician, December, 34-37. Yule, G.U. (1912). On the methods of measuring association between two attributes. Journal of the Royal Statistical Society, 75, 581-642.