Title

Author(s)

Citation

Issued Date

URL

Rights

Validation of the insomnia severity index, athens insomnia scale and sleep quality index in adolescent population in Hong Kong

Kan, Ka-ki, Katherine.; 簡嘉琪 Kan, K. K. [簡嘉琪]. (2008). Validation of the insomnia severity index, athens insomnia scale and sleep quality index in adolescent population in Hong Kong. (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR. Retrieved from http://dx.doi.org/10.5353/th_b4073363 2008

http://hdl.handle.net/10722/52015

The author retains all proprietary rights, (such as patent rights) and the right to use in future works.

Validation of the Insomnia Severity Index, Athens Insomnia Scale and Sleep Quality Index in adolescent population in Hong Kong

Katherine Kan 2007944281 Master of Medical Science

1

Declaration

I, Katherine Ka Ki Kan declare that this thesis represents my own work, except where due acknowledgement is made, and that it has not been previously included in a thesis, dissertation or report submitted to this University or to any other institution for a degree, diploma or other qualifications.

______________________ Katherine Ka Ki Kan

2

Abstract

This study examines the validity and reliability of the Insomnia Severity Index (ISI), Athens Insomnia Scale (AIS) and Sleep Quality Index (SQI) in the Chinese adolescent population in Hong Kong. A sample of 1376 students from 3 secondary schools with age 10-25 and mean age of 14.6 completed a set of questionnaire. Seven to fourteen days after their first attempt, 256 (19%) of the whole sample agreed to do the questionnaire again to evaluate the test-retest reliability of the scales. A clinician’s diagnosis of insomnia was done on 282 (20.5%) of the 1376 students to measure the agreement between the results obtained from the insomnia scale scores with clinician diagnosis of insomnia disorder. The SQI, ISI and AIS showed reliable Cronbach’s alpha of 0.64, 0.83 and 0.81 respectively. Pearson’s correlation from test-retest reliability ranged from 0.66 to 0.77, therefore demonstrated good stability. The concurrent validity of the 3 screening scales all showed a kappa value which fell between 0.21 and 0.40 that showed fair agreement. Upon evaluating the internal consistency, test-retest reliability, convergent and concurrent validity of the 3 scales, we have come to a conclusion that all the 3 sleep scales showed good results to be used as screening tools for insomnia in adolescents for clinical practices. ISI demonstrated to be the most suitable tool out of the 3 as it showed best concurrent validity and high internal consistency and reliability.

3

Introduction

Definition of insomnia

The classification of insomnia can also be defined as experiencing sleep related problems like difficulty initiating sleep, inability to stay asleep, early awakening and unsatisfactory sleep in a person who has sufficient condition and opportunity to sleep (Buysse et al., 2006). The DSM-IV diagnostic criteria for primary insomnia includes: 1. Complaint of difficulty initiating or maintaining sleep or nonrestorative sleep for at least one month. 2. The sleep disturbance (or associated daytime fatigue) causes clinically significant distress or impairment in social, occupational, or other important areas of functioning. 3. The sleep disturbance does not occur exclusively during the course of narcolepsy, breathing-related sleep disorder, circadian rhythm sleep disorder, or a parasomnia. 4. The disturbance does not occur exclusively during the course of another mental disorder. 5. The disturbance is not due to the direct physiological effects of a substance.

4

Prevalence of insomnia-

It is worthwhile to look at the general patterns of sleep disturbances and prevalence of insomnia across adolescents in different countries to get an idea of the insomnia disorder in the general adolescent population.

A 12 month prospective study was done in Texas to estimate prevalence and chronicity of insomnia and the impact of chronic insomnia on health and functioning of adolescents (Roberts et al., 2008). In that study, over one-fourth reported one or more symptoms of insomnia at baseline and about 5% met diagnostic criteria for insomnia. Almost 46% of those who reported one or more symptoms of insomnia in wave 1 continued to be cases at wave 2 and 24% met DSM-IV symptom criteria for chronic insomnia. Roberts and colleagues (2008) came to a conclusion that insomnia is both common and chronic among adolescents.

Another study done on high school students from Cuenca, Spain reported that poor quality sleep was reported by 38.55%, 23.1% had difficulty in getting to sleep, 38.2% woke up during the night and 15.9% woke up too early. 17.7% reported some kind of sleep complaint plus some other insomnia-related symptom; the prevalence of insomnia was found to be 9.9% (García-Jiménez et al., 2004)

Ohida et al. (2004) did a study on self-reported sleep problems among Japanese students and came out with results showing that out of the total, 16% of the students had difficulty initiating sleep, 35% had excessive daytime sleepiness and 39% had subjectively insufficient sleep (Ohida et al., 2004). Another study again done on

5

Japanese students but on a national wide basis revealed a prevalence of insomnia of 23.5% with prevalence of difficulty initiating sleep, difficulty maintaining sleep, and early morning awakening was 14.8%, 11.3%, and 5.5%, respectively (Kaneita et al., 2006).

A previous study on self-reported sleep problems was done on Chinese adolescents in Jinan China. Five high schools from Shangdong, Province of Mainland China were given self-administered questionnaires and reports of insomnia symptoms included difficulty initiating sleep of 10.8%, difficulty maintaining sleep of 6.3%, and early morning awakening of 2.1% by the students (Liu et al., 2000)

A local report looked into problems of sleep disturbances in Hong Kong adolescents and reported that 5.6% had difficulty falling asleep, 7.2% had woken up during the night, 10.4% woke up too early in the morning (Chung and Cheung , 2008). From the past studies, the prevalence of sleep insomnia appeared to be roughly similar in both Chinese adolescents in China and Hong Kong.

We could see that the actual prevalence of insomnia symptoms recorded varies depending on the methodology and country sampled, but most estimates ranged from 10% to 30%.

Possible causes of insomnia in adolescents-

Reports have shown several factors which may be responsible for causing insomnia in adolescents. Studies have shown that negative family life events, together with

6

academic stress, predicted the highest levels of insomnia in young adults in Florida (Bernert et al., 2007). The Hong Kong Federation of Youth Groups have done interviews with Forms 5 and 7 examination candidates in the past and revealed that 33.8% of the respondents had a high stress index. The main sources of stress reported by the students included expectations they had on themselves, not being able to further their studies in F.6 or university and disappointing their parents. Insomnia, was one of the common symptoms that the students complained of, indicating high levels of stress. Also during adolescence (13-22 years of age), changes occur both physically and mentally which can quite drastically change their sleeping patterns, and subsequently influence their sleep quality and quantity (Millman, 2005). Evidence from a Canadian report showed that sleep patterns in early adolescence demonstrated decreased nocturnal sleeping times and bedtimes were delayed (Laberge et al., 2001). In addition, teenagers are easily accessible to high technology and media use such as TV and computer, it can have influence on their sleeping habit as well.

A report on 2 middle school in Brazil invited students from the age of 15-18 to do a self-reported questionnaire on their sleeping habits and computer use and finally came to a conclusion that irregular sleep patterns were associated with nightly computer use which eventually led to a deterioration in sleep quality (Mesquita and Reimao, 2007)

From the studies above, we could see that adolescent sleep is characterized by widespread sleep restriction, irregular sleep schedules, daytime sleepiness, and elevated risk for sleep disturbances .Adolescents in Hong Kong, like the rest of other countries are under intrinsic factors such as puberty and extrinsic factors such as pressure from school and media use can lead them to develop symptoms of insomnia.

7

These factors are likely to put the adolescents in a situation more prone to decreasing total sleep time, a tendency to delay their time of sleep, an increased level of daytime sleepiness and eventually, insomnia disorder.

Consequences of insomnia

From looking at previous studies that are noted above, the most obvious negative consequence we see in adolescents in insomnia is excessive daytime sleepiness. A Canadian report did a cross-sectional survey on a total of 3235 high school students to determine the association between sleepiness and performance in both academic and extracurricular activities. Final results showed that sleep deprivation and excessive daytime sleepiness were common in high school students and were associated with a decrease in academic achievement and extracurricular activity (Gibson et al., 2006). As mentioned above (Roberts et al., 2002), chronic insomnia increased subsequent risk for somatic health problems, interpersonal problems, psychological problems, and daily activities.

A longitudinal cohort of 12 months was done in Staffordshire, UK with 2662 postal returned questionnaire. Of those without insomnia at baseline, the incidence of insomnia at 12 months was 15%, and this was significantly associated with baseline anxiety, depression, and pain. Of those who did have insomnia at baseline, 69% had insomnia at 12-month follow-up. Their results provided evidence that the common problems of insomnia and psychological distress are intertwined and suggest that combined approaches to treatment may be needed to reduce the onset and persistence of the problems (Morphy et al., 2007).

8

People with insomnia had greater depression and anxiety levels than people not having insomnia and were 9.82 and 17.35 times as likely to have clinically significant depression and anxiety, respectively. Increased insomnia frequency was related to increased depression and anxiety, and increased number of awakenings was also related to increased depression (Taylor et al., 2007).

A 20 year long-term study of almost 600 young adults found that those with annual one-month periods of insomnia found the incidents of sleeplessness gradually increased over two decades (Buysse et al., 2008) Study author Dr. Daniel J. Buysse (2008) showed in his results that insomnia seems to be followed by depression more consistently than the other way around and insomnia tends to be a chronic problem that gets more persistent over time, whereas depression was a more intermittent problem (McKeever, 2008). Buysse (2008) says that there is growing evidence that insomnia is not just a symptom of depression, but that it may actually precede depression. In other words, people who have insomnia but no depression are at increased risk for later developing depression.

Results from Bailly et al. (2004) showed that persistent sleep disorders are significantly associated with physical, psychological or social difficulties, hence it is essential to take an interest in the quality of sleep in adolescents, which may be a way to approach their psychosocial difficulties (Bailly et al., 2004)

A more extreme psychological problem of sleep and suicide has also been conducted. A study on sleep and youth suicidal behavior by Liu and colleagues concluded that suicidal psychiatric patients had more sleep disturbances including insomnia,

9

hypersomnia, or nightmares than nonsuicidal patients and also sleep loss or disturbances are likely to signal an increased risk of future suicidal action in adolescents. (Liu et al., 2006)

In view of the high prevalence of insomnia and its immediate and long-term negative effect on adolescents, it is important to detect insomnia early and provide intervention.

Assessment methods of insomnia

A paper by Buysse (2006) clearly presented recommendations for assessment methods of insomnia. The most conventional way to diagnose specific areas of insomnia and psychiatric disorders is to use structured interviews or the International Classification of Sleep Disorders 2nd edition criteria by a clinician.

For global sleep and insomnia symptoms, the self administered questionnaires which captures subjective aspects of sleep experience like the Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Athen’s Insomnia Scale (AIS) and the Sleep diary was recommended.

For diagnosis of consequences of insomnia such as fatigue, mood and quality of life and function and psychiatric disorder, the Beck Depression Inventory and Fatigue Severity Scale and Inventory of Depressive Symptomatology can be used (Buysee et al., 2006)

10

Hence we could see that there are many ways to assess insomnia of different severity and also targeted for a range of symptoms of insomnia.

Self-reported scales of insomnia

Self-report instruments are ideal because it is convenient for both the patients and clinicians. However, before these scales can be used for screening, it must be validated first to show that they are reliable and accurate within the population.

The ISI was tested for validity and reliability in a group of insomniac patients. The results showed that the ISI has adequate internal consistency and is a reliable selfreport measure to evaluate perceived sleep difficulties. The results also indicated that the ISI is a valid and sensitive measure to detect changes in perceived sleep difficulties. In addition, there is a close convergence between scores obtained from the ISI patient's version and the clinician's so it has met the gold standard validation (Bastien et al., 2001).

The AIS was tested twice on adults for validity and reliability by Soldatos and colleagues. One was validation based on ICD-10 (Soldatos et al., 2000) and the other one was done on diagnostic validity (Soldatos et al., 2003). In both studies, results show that the high measures of consistency, reliability, and validity of the AIS made it a very good tool in sleep research and clinical practice.

The SQI was used in an epidemiological study on a population of 31140 Finnish adults and showed a good internal consistency (Urponen et al., 1991) but on a local

11

report on sleep-wake patterns and sleep disturbances on adolescents in Hong Kong, the SQI showed a lower internal consistency (Chung and Cheung, 2008). Hence, the actual validity on the adolescents in Hong Kong is still not very clear. The ESS had been used more widely and a study done in Taiwan, where the Chinese version of the ESS had been proved to be reliable in both a linguistic and a test-retest sense, and valid to act as a standardised screening test of sleepiness in daily life (Chen et al., 2002). A local report also demonstrated that the Chinese version of the ESS in adults had satisfactory reliability and validity was useful to separate patients with and without pathological degree of objective daytime sleepiness (Chung, 2000).

The internal consistency, validity and factor structure of the 12-item General Health Questionnaire (GHQ-12) were investigated in a homogeneous sample consisting of 18-year-old males in Italy. The GHQ-12 was proved to be useful and reliable in providing information about the well-being in the study (Politi et al., 2004).

The testing of the GHQ-12 on adolescents between the age of 12-19 in Australia was also concluded that the validity of the GHQ for adolescents in populations other than the UK and Hong Kong remains to be demonstrated as validity outside these 2 areas showed sporadic results (Tait et al., 2002)

The review of self-administered screening scales above has shown that the ISI, AIS and SQI have been shown to be reliable and accurate in the adult population, but we are still lacking validity of these scales for the adolescent population.

12

Problems with screening scales Screening scales are usually done in a self-reported way, so there is always a possibility that the subject may have misinterpretation of the scale items. Subjective perception of the disease my also lead to under or over-report of severity. Screening scales are usually rated by looking at the extent of the severity in the symptoms, therefore it can be quite difficult to assess functional impairment by analyzing the symptoms. Self-report scales may not follow the same diagnostic criteria used in diagnostic system, so there could be disagreement between the outcome of the selfreported scales and diagnostic system.

Cross-cultural research is inevitable for researchers because research projects frequently need more than one cultural or ethnic group therefore one instrument is normally required to be translated into many different languages.

Problems that often arise with screening scales is that when an instrument developed in one country is translated and used in another language and culture, even when all the words are changed exactly, the meaning of the original expression may not be kept.

One way to ensure that the translation is done properly is to translate from the original language in the new language and then translate it back to the original language to imply the equivalence of the two versions (Tait et al., 2001). Another way to translate it properly is to have teams of experts or bilinguals to translate it professionally.

13

Aim of study

Upon emphasizing the need to identify insomnia symptoms because of the negative effects it may have on adolescents, it has given us a very good motivation to develop a more easily accessible and faster way to screen for insomniac adolescents. This study would focus on validating 3 self-administered questionnaires for use in the adolescent population as a screening tool for insomnia. This is because most selfreported scales of insomnia have been confirmed and demonstrated in the adult population therefore it would be sensible to validate scales that could be used for adolescents especially in present of the high prevalence of insomnia symptoms. The main objective of this paper is to validate three self-reported scales: Insomnia Severity Index (ISI), Sleep Quality Index (SQI) and the Athens Insomnia Severity (AIS). We did not use the Pittsburgh Sleep Quality (PSQI) Index and sleep diary as suggested by Buysse because PSQI is too long and it does not focus on the problems of insomnia alone and sleep diary requires a week to complete usually 7-day sleep diary, so it was not easy to administer. Another incentive to carry out this study is that we want to identify self-report instruments that can be easily administered and at the same time be a valid screening tool. This study will allow us to come up with a decision whether or not these 3 scales are reliable enough to act as a screening tool for insomnia and which one would be the most applicable. From previous studies on the validity of the 3 scales in the adult population, I predict that the ISI, SQI and AIS will be valid in the adolescent population in Hong Kong. I will test the internal consistency, test-retest reliability and the covergent and concurrent validity of each scale.

14

Method

Subject

Chinese secondary students were chosen to fill in a questionnaire that consists of the 3 scales that were validated (ISI, AIS and SQI). Secondary schools in Hong Kong are separated in 3 bands based on the academic achievement of the students in the school. One co-educational school was chosen from each band so we end up with 3 preselected schools that cover a range of students with different academic results. Students from Forms 1-4 and Form 6 were recruited to fill in the questionnaire and Forms 5 and 7 were excluded because they were having public examinations. Participating schools are co-educational schools so it consists roughly same number of boys and girls.

Assessment Procedure

Translation

We selected self-reported sleep-wake questionnaires that could be understood by adolescents as young as 12 years in the local cultural setting. The questionnaires used in this study were forward and backward translated and results were checked by 2 Chinese language teachers, who verified they would be suitable for Form 1 students and above.

15

Pilot test The questionnaires were piloted in a class of Form 1 students in a school with marginal academic results. The pilot test is to ensure that the level of understanding of the scales were satisfactory so it doesn’t hinder with the results obtained from the participants as the scales were translated from English to Chinese.

Instead of answering the questions, the students were asked to only rate each question in the whole questionnaire based on their understanding on a 7 point scale (1 = extremely easy to understand; 4 = not sure; 7 = not at all understandable); any item that has a mean score > 2.5 will be retranslated and retested until the highest level of understanding is obtained.

From the pilot test results, 3 questions from the questionnaires had to be reworded as the mean score of understanding was over 2.5. The 3 questions that did not satisfy our translation requirements were amended before the main study.

Main Study

All Forms 1-4 and Form 6 students from the 3 secondary school of different academic standings (Band 1, Band 2 and Band 3) that were asked to fill out a self-reported questionnaire that includes the 3 sleep scales of interest (SQI, ISI and AIS) and other scales and information that was used for data analysis. The questionnaires were in Chinese to meet the requirements of their mother tongue of the students.

Written consents were obtained from the students and their parents prior to the survey.

16

We arranged a time convenient to the schools to carry out the survey. Two of the schools had their students fill out the questionnaires that were given out by teachers during Humanities or General Studies lesson in class. These 2 schools were given one week to make sure all the questionnaires are filled out. The other school had all their students in the school to fill out the questionnaire together at the same time during school assembly.

Ten to 14 days after the students completed the whole questionnaire, twenty-five percents of the students who have completed the first phase of the study were randomly selected to repeat the questionnaires and receive a face-to-face interview with a sleep specialist. We chose 10-14 days because there will be less chance to have significant change in sleep habits and also the time interval is enough to reduce the chance of the students remembering what they have answered in the first attempt. An experienced sleep clinician interviewed the students; those who satisfy the criteria A and B of DSM-IV Primary Insomnia are considered as suffering from insomnia disorder.

Measures The questionnaire included the following for measurements: 1. Clinical and demographic features – gender, age, family condition (more specific), parents’ occupations and educational level, living condition and medical history. 2. Sleep-habits questionnaire 3. Sleep Quality Index (SQI)

17

4. Insomnia Severity Index (ISI) 5. Athens Insomnia Scale (AIS) 6. Epworth Sleepiness Scale (ESS) 7. General Health Questionnaire 12 Item (GHQ-12)

The students had to fill in their class and student number and they was emphasized to do so as we had to retrieve their files for test-retest and their class and student number through this identity. Gender, age, academic results and parents’ education level and occupation were recorded for analysis on their backgrounds.

Due to time restriction, this study would omit the analysis of the sleep-habits questionnaires.

Description of the Scales

Sleep Quality Index

The SQI is self-administered questionnaire that has 8 questions which evaluates the quality of sleep of the patients based on the majority of days and nights in the past month. Studies on SQI done in adults in the past (Uroponen et al., 1991) have showed good results and reliability. Out of the 8 questions, 6 asked about how frequent they suffered from insomnia, difficulties falling asleep, disturbed night sleep, waking up during the night, waking up too early and the use of hypnotics. The other 2 questions asked how much time it takes for them to fall asleep and the degree of morning tiredness they experience The final score for this scale range from 0 to 16 as each

18

question is rated on a 3-point scale (0, 1, 2). A higher score will show a lower quality of sleep for an individual.

19

Insomnia Severity Index

The ISI is offered in 3 different forms: self administered by patients, significant other and clinician. ISI had successfully been used and validated on insomnia adults in the past (Bastien et al., 2001). The ISI consists of 7 questions which evaluates the 3 stages during sleep (initial, middle and terminal), satisfaction of sleep pattern, the degree of interference with daily functioning, how noticeable their sleep pattern problems are to others and how worried are they about their sleep problem.

The total score range from 0 to 28 because the 7 questions is rated on a 5-point scale with 0= not at all and 4=extremely. The higher the score obtained from this scale, the more greater the severity of their insomnia.

Studies by Bastien et al. (2001) have shown that the ISI scale is reliable for the adult population and it is supported by concurrent validations with other instruments such as the sleep diary and polysomnography.

Athens Insomnia Severity

The AIS is a self-reported questionnaire which consists of 2 parts. The first 5 questions are related to the quality of sleep of the patient and the second part has 3 questions which are more related to the interference of their sleeping problems in their daily activities.

20

Each question is scored on a 4 point scale (0, 1, 2 and 3) and patients are advised to respond positively if they experience the sleep difficulty listed for each question so a higher score will mean that their state of insomnia is more severe.

There are 2 versions of the AIS: AIS-8 and AIS-5. AIS-8 uses the entire scale hence the final score for this scale range from 0-24. The shorter 5 item scale, AIS-5, only uses the first part of the scale where the quality and quantity of sleep will be evaluated. Soldatos et al. (2000) have shown that the high measure of reliability, consistency and validity of the AIS in the adult population in Greece, makes it a useful tool for clinical practices and sleep research.

The AIS was also successfully used in the Japanese population on studies to see the association between work and insomnia (Yoshioka et al., 2008).

Epworth Sleepiness Scale (ESS)

The ESS consists of 8 questions which evaluates the possibility of the patient to doze off in 8 different daily activities during the day. Each question is scored on a 4-point scale (0, 1, 2, 3), with 0=would never doze and 3= high chance of dozing. The patient’s final score range from 0-24 and a higher score will indicate that they have a higher tendency to fall asleep during their daily activities. A paper by Chung (2000) shows that the ESS has shown satisfactory reliability and validity in the Chinese adult population and suggested that it should be included as one of the methods for assessing sleepiness (Chung, 2000).

21

General Health Questionnaire-12 (GHQ-12)

The GHQ-12 is a 12 question questionnaire that evaluates the mental well being of the patient. Each question is scored by ticking off a 4-point scale (0, 1, 2, 3) where 0=much less than usual and 3= much more than usual. The Chinese version of the GHQ-12 was proved to be reliable and valid. (Pan PC, 1990).

This scale has been validated in the Australian adolescent population (Tait et al., 2003) and proved to be a valid index of psychological wellbeing in this population. Therefore, it is a reliable tool to test for the mental health of adolescents.

Data Analysis

All statistical analysis were done by SPSS 16.0. for Windows. Descriptive results were drawn from the sleep-wake habit questionnaire and other general questions regarding student’s personal information such as socioeconomic status and academic achievements.

Prevalence of sleep disturbances were analyzed based on the findings from the SQI, ISI and AIS.

The reliability of the results were tested in 2 methods. The first one was from the testretest method, which is the agreement of measuring instruments over time. To determine stability, a measure or test is repeated on the same subjects at a future date. In this study, the same questionnaires were filled out again by 256(18.6%) students

22

after 2 weeks. Results were compared and correlated with the initial test to give a measure of stability by Pearson’s correlation.

The second method to test the reliability of the results was to examine the homogeneity of the items in the scale by finding the Cronbach’s alpha coefficient, which is testing the internal consistency. Cronbach's alpha measures how well a set of items (or variables) measures a single dimension construct, therefore it is a coefficient of reliability (or consistency). Cronbach’s alpha ranges from 0 to 1 and the higher the score, the more reliable the scale is. The 3 scale of interest, the SQI, ISI and AIS within the questionnaire will be evaluated on its own. Each question will be correlated with each other to test for the homogeneity of the items within the scale by Cronbach’s alpha.

The convergent validity was done by performing Pearson’s correlation analysis on the final scores of the 3 scales. The AIS, ISI and SQI were inter-validated with each other to determine its extent of association they have with each other. The more related the measurements are, the higher the convergence.

The external validity was done by correlating the SQI, ISI and AIS separately with the 2 other scales (ESS and GHQ-12) by Pearson correlation.

The concurrent validity was tested by examining the agreement between the results obtained from the insomnia scale scores with clinician diagnosis of insomnia disorder. The clinician’s report was used as a gold standard to find the best diagnostic cutoff

23

score for the 3 scales to determine the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the SQI, ISI and AIS.

This was done by calculating the Cohen’s Kappa coefficient (Sim et al., 2005), which is a measure of 2 rater’s agreement. One rater was the clinician’s report and the other one was the self-reported questionnaire from the student. The kappa coefficient helped us to determine the best diagnostic cutoff score for each of the 3 scales.

The study was approved by ethics board before anything was carried out for research.

24

Results

A total of 1964 copies of consent form were sent out to the 3 pre-selected secondary schools in Hong Kong and the total number of usable questionnaires for our analysis comes to a final total of 1376 copies. There was a response rate of 70.1%. Questionnaires that were completed incorrectly either by leaving parts of it blank or wrongly filled in were not included for analysis. Students who have failed to identify themselves by their class and class numbers were also omitted from the analysis.

A total of 344 students (25% of the 1376 students who completed the first phase of study) were invited to repeat the questionnaires and receive a face-to-face interview by a sleep specialist. The data of 256 (74.4%) of the 344 students was used for the test-retest analysis and 282 (82.0%) received the face-to-face interview. Again, students who completed the questionnaire incorrectly in the second try or those who did not even have a first attempt were omitted from the test-retest analysis. All students who were given consent to do the face-to-face interview were included only.

Socio-demographics and personal background

Table 1 presents the students’ characteristics. There were 617 male and 759 female students in this analysis. Their ages range from 10 to 25 with a mean age of 14.6 and a standard deviation of 1.5 years.

25

Around 53% of their father received Secondary education as their highest level of education, and 4% and 3% of their fathers and mothers respectively received tertiary education or higher.

Almost half of the students fall into the category of “very little” alcohol consumption and 27% of them have never consumed alcohol while 29.5% have an occasional tendency of consuming alcohol. Only 1% of the students reported to have smoking habit.

From their self reported academic standings, 382 of them belonged to the top third of the class, 574 of them were in the middle range of the class and 420 are in the lower third of the class.

Prevalence of sleep disturbances

Table 2 presents the results derived from the SQI which relates to their sleeping habit over the last 1 month. About 13% of them took more than 30 min to fall asleep and 3% of them claimed to have suffered from insomnia of more than 3 times per week. 75% of them did not have a disturbed night sleep. Morning tiredness occurs in almost 44% of the adolescents. Students with disturbed night sleep or waking up too early in the morning of more than 3 times per week accounts for 4.87% and 5.09% respectively. Almost all of them have not used hypnotics over the past 3 months.

Table 3 presents the results from ISI. Roughly 15% to 25% of students showed moderate to very severe insomnia. Less than 25% were unsatisfied with their current

26

sleep. Around 43% of the students sensed that their sleep problems may cause an interference with day time functioning to a certain extent. The majority of students (92%) perceived that there is a certain extent of noticeable impairment and at least 93% of them were somewhat concerned with their sleep problem.

Table 4 shows the results obtained from AIS. Well over 80% of the students showed no major problem in sleep induction, disturbed night sleep and waking up in the morning. Around 10% of them showed markedly or very decreased sense of wellbeing and functioning during the day. Twenty percent of them were not satisfied with their total sleep duration and around 30% of the students showed considerable amount of sleepiness during the day.

Internal consistency

The internal consistency of the 3 insomnia scales SQI, ISI and AIS was measured based on Cronbach’s α. Results from table 5 show that the internal consistency of the 3 scales are quite high as their Cronbach’s α of the SQI, ISI and AIS were 0.64, 0.83 and 0.81 respectively. The Cronbach’s α did not show drastic change when any one of the items was removed from the calculation for all the 3 scales.

Internal consistency is lowest in SQI. The inter-item Pearson’s correlation of SQI was presented in Table 6. The inter-item total correlation was not too high as it ranges from 0.01 to 0.57.

27

Cronbach’s α for both ISI and AIS are very high but the inter-item total correlation was only slightly higher than the SQI (Table 7 and 8).

Test-retest reliability

Questionnaires that had missing data and students who did not have a first attempt were excluded from this analysis. After all the exclusions, 256 students’ questionnaires that were repeated 10-14 days after their first attempt were used to compare with their first attempt for the test-retest reliability. The test-retest Pearson’s correlation coefficient correlation for SQI total score is 0.67 (Table 5), 0.75 for ISI total score (Table 6) and 0.77 for AIS total score (Table 7). The results were all significant at the 0.01 level hence, all of them showed a good level of satisfaction.

The significant mean-item correlation for SQI ranges from 0.39 to 0.63 except the last item on the scale which correlates to -0.008 (Table 5). The mean-item correlation ranges from 0.46 to 0.64 for ISI (Table 6) and 0.45 to 0.62 for AIS (Table 7), in which all are significant at the 0.01 level.

Convergent validity between SQI, ISI and AIS

The SQI, ISI and AIS were correlated with each other with the Pearson’s correlation coefficient to test for the convergent validity. Table 11 shows that the correlation values ranges from 0.59 to 0.74, with correlation between SQI and AIS being the lowest and ISI and AIS the highest. The correlations of the convergent validity were all significant at 0.01 level.

28

Correlation between SQI, ISI and AIS with ESS and GHQ-12

The 3 scales were also correlated with the ESS and GHQ-12 separately. SQI showed correlations of 0.29 and 0.44 with ESS and GHQ-12 respectively (Table 12), while ISI showed correlations of 0.38 and 0.52 and AIS showed correlations of 0.44 and 0.51. Here, SQI showed the lowest correlation with the 2 scales ESS and GHQ-12 and AIS showed the highest correlation. The correlation coefficients were all significant at the 0.01 level.

29

Validation against clinician diagnosis

Two-hundred eighty two students received an interview by a sleep specialist to devise the DSM-IV diagnosis of insomnia disorder. A total 25 of the 282 (8.9%) students were assessed to have insomnia disorder and the results were used to measure against the outcome from their self-reported questionnaire.

Results show that the kappa coefficient is the highest at the score of 7, which means that it is the best diagnostic cutoff value for the SQI with 60% sensitivity, 89% specificity, PPV of 35% and NPV of 96% (Table 13).

The kappa coefficient demonstrates that the best diagnostic cutoff value is 14 with 48% sensitivity, 91% specificity, 35% PPV and 95% NPV for the ISI (Table 14).

The kappa coefficient was the highest at a diagnostic cutoff at a score of 10 with 52% sensitivity, 84% specificity, 25% PPV and 95% NPV for AIS (Table 15).

30

Discussion

The purpose of this paper is to validate 3 insomnia scales, SQI, ISI and AIS in the adolescent population in Hong Kong to look for the best screening tool for insomnia in adolescent. The moderate to high internal consistency, test-retest reliability, and convergent and concurrent validity of the SQI, ISI and AIS suggested that psychometric properties of these 3 scales are acceptable and satisfactory. The satisfactory level of these scales are sufficient to act as a screening tools intended for research and diagnostic purposes of insomnia disorder.

Prevalence of sleep disturbance

From the results of the 3 scales, the prevalence of sleep disturbance in our students shared some similarity with a previous study that was done in China, Jinan city (Liu et al, 2008). Liu et al (2008) showed about 25% of their students were not satisfied with their sleep which is similar to the results from our current study, where retrieved information from ISI showed that around 25% of the students were dissatisfied or very dissatisfied with their sleep.

For SQI there are about 5% of students who complained of difficulties initiating and maintaining sleep, 4% with wakening up too early, but over 40% complained of tiredness in the morning. The prevalences of initial and middle insomnia symptoms in the current study are similar to a recent local survey (Chung and Cheung, 2008), which are in the range of 4-7%, the prevalence of early awakening and morning

31

tiredness is higher in the previous study, which is about 10% and 50%, respectively.

The AIS showed a higher prevalence of initial, middle and late symptoms of insomnia than SQI and ISI. The reason to this is unknown

In comparison, there was a higher percentage of students that showed rather great concern regarding their functioning and well-being during the day in contrast to the actual percentage of students who claim to have real sleep disturbances. This may suggest that there could be other external factors that caused the hindrance to their daily activities during the day.

Internal consistency

The SQI showed a Cronbach’s alpha of 0.64, which is marginally acceptable to be an adequate scale (Nunnaly, 1978). This shows that there is moderately low level of homogeneity between the items and the items are not too correlated with the overall scale.

Studies done with the SQI on adults of ages 35-59 in the past (Urponen et al.) showed higher Cronbach’s alpha value of 0.73. But studies done on sleep-wake pattern and sleep disturbances among Hong Kong Chinese adolescents by Chung et al. (2008) demonstrated an even lower Cronbach’s alpha of 0.57. This may suggest that SQI tends to have lower internal consistency when the scale is applied to adolescents compared to adults.

32

The ISI and AIS both showed a Cronbach’s alpha of higher than 0.80, which is a good indicator that there is high level of homogeneity between the items in the scale and good reliability. The inter-item correlation of both the scales show similar value to the SQI only, but it is not uncommon to have a scale with high Cronbach's alpha to have one or more items with low item-total correlations.

Each item was deleted separately to test if there were any considerable change in the Cronbach’s alpha. When single items were removed from the scales, the deleted item Cronbach’s alpha did not show too much fluctuation in all the 3 scales. This means that there are no single items in the scale that can be held responsible for the overall internal consistency.

Hence, in terms of internal consistency based on Cronbach’s alpha, the ISI and AIS demonstrated more homogeneity in the items than in SQI.

Test-retest reliability

Test-retest reliability is an index of score consistency over a brief time period, which is 10-14 days for our in study. The test-retest reliability was highest in AIS, followed by ISI, and SQI had the lowest reliability of less than 0.7. SQI also have the lowest mean-item correlation (which ranges from 0.39 to 0.63) but both the ISI and AIS show similar range and values. The moderate value showed in the test-retest reliability means that the students demonstrated fair stability in the scales over the 1014 days gap between their first and second time in filling out the scales.

33

The test-retest reliabilities of the AIS in previous studies with adults (Soldatos et al., 2000) showed a much higher correlation of 0.89 in the overall scale and correlation ranged between 0.70 to 0.86 for each individual item. However, there is a time difference between our test-retest time and that of Soldatos et al, (2000). Soldatos et al’s study did their test-retest with their patients after 1 week of their first trial, which may account for the discrepancy to our results.

Convergent validation and external validation

Convergent validation between the 3 scales and external validation with ESS and GHQ-12 all showed high correlations. The purpose of the convergent validation here is to measure the degree of association between the 3 different scales. High correlations between the test scores would be evidence that measures that should be related are in reality related. The AIS and ISI showed the highest correlation followed by ISI and SQI while AIS and SQI had the lowest correlation.

The external validation done with ESS and GHQ-12 illustrated that AIS and ISI had similarly value of Pearson’s correlation coefficient with them and SQI had the lowest. Although the correlations were all significant, the results shown here indicates that in the comparison of the 3 scales, AIS and ISI would be the best screening tool for insomnia and SQI would be the least favored.

Soldatos et al. (2000) also did their validation of AIS by correlating with the Sleep problem scale and the Pittsburg sleep quality index in 2 different studies both validating AIS. Results showed high Pearson’s correlation coefficient of around 0.9. It

34

would worthwhile to compare the 3 scales with the Sleep problem scale or the Pittsburg sleep quality index for further study to test if their Pearson’s correlation coefficient would be significantly higher.

Validation against clinician diagnosis

Cohen’s kappa is a measurement of agreement between 2 raters. In our study, the two raters would be the clinician diagnosis and the self-reported questionnaire from the students. Kappa values for the 3 scales all ranged from 0.21-0.40, which demonstrates fair agreement (Landis and Koch, 1977).

At a diagnostic cutoff score of 7 or higher in SQI, the NPV was found to be well above 90% which means that almost all of the students who score less than 6 can be regarded as not suffering from insomnia. The high specificity value indicates that most students who are not suffering from insomnia are successfully identified as not having insomnia after screening. If a cutoff score at 6 was chosen in SQI, there would be an even higher NPV, which gives us more certainty to determine that those who score less than 6 are not insomniacs and at the same time, a higher sensitivity, so less false positive results. However, if we do so, the trade-off would be a lower specificity and PPV, consequently there would be higher false negative.

A diagnostic cutoff score of 12 based on kappa gives ISI similar results to SQI as mentioned above with high NPV and specificity. Similarly this meant that students who score less than 12 are almost all non-insomniacs and if they have a negative test, there is an 84% chance that it was showing true negative results. If a cutoff score of

35

11 was chosen, there would be higher sensitivity and NPV but again this trade off would give us lower specificity and PPV.

Based on kappa, a score of 10 for AIS is the cutoff between an insomniac and a noninsomniac. Very similar to SQI and ISI, high NPV and specificity showed accurate true negative results. From looking at analysis of AIS, if a cutoff score of 9 was chosen,

All 3 scales shared the same characteristic of lower sensitivity and PPV values at their cutoff scores. Low sensitivity meant that there are high false positive rates, where many screened positive are not actually having insomnia disorder. In this case, ISI has the highest sensitivity, so lowest false positive rates.

SQI, ISI and AIS have a PPV of around 30% which meant that roughly 1/3 of positively tested patients are actually insomniacs.

From the gold standard validation of the 3 scales, it pointed out that these scales were quite accurate in identifying students who are not insomniacs. On the other hand, all the 3 scales were moderate at identifying students who are suffering from insomnia. This means that a positive result in these scales would need more reliable tests and examination to accurately determine whether an individual has insomnia disorder or not. Positive cases were advised to seek help from their family doctors. High specificity and NPV values indicate that negative results from these tests would most likely be enough to confirm that the student does not have insomnia disorder. Low

36

PPV may also indicate that the prevalence of insomnia is low in adolescents hence, insomniacs are harder to identify.

It is common to have high specificity and lower sensitivity in screening tools as students may show more concern with their poor sleep that had happened for a few nights and reported to have poor sleep but on the other hand, a clinician’s assessment would only focus on the sleep problem that occurred in the majority of nights in the past 1 month, hence a high false positive.

Limitations

Our study had a few places for improvement and area for further investigation. The scales were translated from English to Chinese and a pilot test was done initially in hope to try our best effort to preserve the same meaning in the 2 versions. However, there can never be perfect translation so a way to improve for future studies can include using professional translator, larger number of students in the pilot or obtaining more teachers' comments.

The three Secondary schools were chosen by a convenient sample selection, hence this could be a limitation of the district for this study. In addition, we assumed that 1500 to 2000 students should be sufficiently large to examine the psychometric properties of the scales. 200-300 clinician interview should identify 10-15 clinically significant insomnia cases for calculation of sensitivities and specificities but precise statistical analysis of the sample size requirements were not done exactly.

37

This study can also be further carried out on adults in Hong Kong to compare results due to the age difference and see if age could be the limiting factor to the validity to these scales.

Due to the omission of the sleep-habits questionnaire, there was also a limitation of our testing of the concurrent validity because we could analyze the correlation of the 3 insomnia scales with the findings obtained from the sleep-wake habits questionnaire. This could be included in future analysis.

The SQI had a lower internal consistency in comparison to the other two scales probably because there were 2 items which could not correlate very well. One is morning tiredness which occurs frequently and the other one is use of hypnotics which rarely occurs. It would be worthwhile to calculate the internal consistency without the 2 items to compare the differences in further analysis.

In addition, our sample had neglected Forms 5 and 7 students as they were away from school either preparing or having public examinations. This means that we couldn’t randomly select adolescents from the whole school population.

Summary

To sum up, the SQI, ISI and AIS used on adolescents in Hong Kong showed significant and reasonable results to act as screening tools for insomnia. However, the 3 scales are very good indicators of students of not having the condition (noninsomniacs) but weak in identifying students who has insomnia. All 3 of them

38

demonstrates excellent performance as a screening tool to pick out the students who do not need further examinations to determine that they do not have insomnia disorder. On the whole, the SQI, ISI and AIS are all useful as screening tools for clinical practice and research purposes on adolescents in Hong Kong as they showed good internal consistency, test-retest reliability and convergent and concurrently validity in this paper. In comparing the 3 scales, ISI would be the best, followed by AIS and finally SQI. This is because, ISI showed highest kappa coefficient, high internal consistency from Cronbach’s alpha and high reliability from evaluation of the testretest reliability.

39

Table 1. Sample description (n=1376) (1 decimal place) Variables

N (%)

Female Gender

759 (55.2)

Age, years 12 or under 13 14 15 16 17 or above

106 (7.7) 253 (18.4) 326 (23.7) 326 (23.7) 227 (16.5) 138 (10.0)

Father's education No formal education Primary Secondary Tertiary

328 (23.8) 258 (18.8) 738 (53.6) 52 (3.8)

Mother's education No formal education Primary Secondary Tertiary

331 (24.1) 284 (20.6) 719 (52.3) 42 (3.1)

Have smoking habit 18 (1.3) Alcohol consumption Sometimes Very little Never

337 (24.5) 670 (48.7) 369 (26.8)

Self reported academic result Top one-third Middle one-third Lower one-third

382 (27.8) 574 (41.7) 420 (30.5)

40

Table 2. Prevalence of sleep disturbance by Sleep Quality index. (n=1376) Sleep quality index item

N (%)

Time to fall asleep 10 min or faster

647 (47.0)

11-30 min

554 (40.3)

30 min or longer

175 (12.7)

Suffered from insomnia No

968 (70.4)

Less than 3 days/ week

366 (26.6)

3 or more days/ week

42 (3.1)

Difficulties falling asleep No

911 (66.2)

Less than 3 days/ week

407 (29.6)

3 or more days/ week

58 (4.2)

Disturbed night sleep No

1033 (75.1)

Less than 3 days/ week

276 (20.1)

3 or more days/ week

67 (4.9)

Waking up during night Less than once/month

826 (60.0)

Less than 3 days/week

480 (34.9)

Every or almost every night

70 (5.1)

Morning tiredness Very or rather alert

425 (30.9)

Cannot say

349 (25.4)

Very or rather tired

602 (43.8)

Waking up too early No

847 (61.6)

Less than 3 days/ week

469 (34.1)

3 or more days/ week

60 (4.4)

Use of hypnotics No

1358 (98.7)

Occasionally

15 (1.1)

At least once/week

3 (0.2)

41

Table 3. Prevalence of sleep disturbance by Insomnia severity index. (n=1376) Insomnia Severity index item

N (%)

Perceived severity of insomnia (initial) None

626 (45.5)

Mild

409 (29.7)

Moderate

258 (18.8)

Severe

71 (5.2)

Very severe

12 (0.9)

Perceived severity of insomnia (middle) None

858 (62.4)

Mild

307 (22.3)

Moderate

160 (11.6)

Severe

37 (2.7)

Very severe

14 (1.0)

Perceived severity of insomnia (late) None

751 (54.6)

Mild

394 (28.6)

Moderate

183 (13.3)

Severe

35 (2.5)

Very severe

13 (0.1)

Degree of satisfaction Very satisfied

222 (16.1)

Satisfied

340 (24.7)

Neutral

476 (35.9)

Dissatisfied

280 (20.4)

Very dissatisfied

58 (4.2)

Interference with daytime functioning Not at all interfering

153 (11.1)

A little

628 (45.6)

Somewhat

398 (28.9)

Much

151 (11.0)

Very much interfering

46 (3.3)

Noticeability of impairment

42

Not at all noticeable

360 (26.1)

A little

594 (43.2)

Somewhat

326 (23.7)

Much

70 (5.1)

Very much noticeable

26 (1.9)

Concern of sleep problem Not at all worried

539 (39.2)

A little

507 (36.9)

Somewhat

234 (17.0)

Much

69 (5.0)

Very much worried

27 (2.0)

43

Table 4. Prevalence of sleep disturbance by Athens insomnia scale. (n=1376) Athen’s insomnia scale

N (%)

Sleep induction No problem

550 (40.0)

Slightly delayed

595 (43.2)

Markedly delayed

201 (14.6)

Very delayed or did not sleep at all

30 (21.8)

Awakenings during the day No problem

653 (47.5)

Minor problem

632 (45.9)

Considerable problem

82 (6.0)

Serious problem or did not sleep at all

9 (0.7)

Final awakening earlier than desire Not earlier

671 (48.8)

A little earlier

570 (41.4)

Markedly earlier

121 (8.8)

Much earlier or did not sleep at all

14 (1.0)

Total sleep duration Sufficient

441 (32.1)

Slightly insufficient

654 (47.5)

Markedly insufficient

252 (18.3)

Very insufficient or did not sleep at all

29 (2.1)

Overall quality of sleep (no matter how long you slept) Satisfactory

597 (43.4)

Slightly unsatisfactory

603 (43.8)

Markedly unsatisfactory

154 (11.2)

Very unsatisfactory or did not sleep at all

22 (1.6)

Sense of well-being during the day Normal

702 (51.0)

Slightly decreased

545 (39.6)

Markedly decreased

113 (8.2)

Very decreased

16 (1.2)

Functioning (physical and mental) during the day

44

Normal

654 (47.5)

Slightly decreased

575 (41.8)

Markedly decreased

129 (9.4)

Very decreased

18 (1.3)

Sleepiness during the day None

142 (10.3)

Mild

773 (56.2)

Considerable

376 (27.3)

Intense

35 (2.5)

45

Table 5. Internal consistency of the SQI, ISI and AIS. (Cronbach’s alpha)

Sleep Quality Index

Insomnia Severity index

Athens Insomnia Scale

Whole scale

.64

.83

.81

Item 1 removed

.60

.80

.80

Item 2 removed

.57

.80

.80

Item 3 removed

.56

.82

.82

Item 4 removed

.60

.80

.77

Item 5 removed

.60

.79

.77

Item 6 removed

.68

.81

.76

Item 7 removed

.62

.78

.77

Item 8 removed

.65

SQI item

.79

46

Table 6. Inter-item Pearson’s correlation of the Sleep Quality Index (n=1376) Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

1. Time to fall asleep 2. Suffered from insomnia

.39

3. Difficulties falling asleep

.48

.57

4. Disturbed night sleep

.18

.27

.30

5. Waking up during night

.16

.27

.23

.32

6. Sleepiness in the morning

.10

.14

.10

.13

.13

7. Waking up too early

.14

.18

.19

.20

.30

.08

8. Use of hypnotics

.07

.17

.13

.20

.14

.01

47

.13

Item 8

Table 7. Inter-item Pearson’s correlation of the Insomnia severity index. (n=1376) Item 1a Item 1b Item 1c

Item 2

Item 3

Item 4

1a. Perceived severity of insomnia (initial) 1b. Perceived severity of insomnia (middle)

.56

1c. Perceived severity of insomnia (late)

.38

.53

2. Degree of satisfaction

.43

.38

.25

3. Interference with daytime functioning

.33

.35

.25

.47

4. Noticeability of impairment

.26

.26

.19

.37

.57

5. Concern of sleep problem

.46

.42

.32

.53

.62

48

.54

Item 5

Table 8. Inter-item Pearson’s correlation of the Athens insomnia scale (n=1376). Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

1. Sleep Induction 2. Awakenings during the day

.31

3. Final awaking earlier than desired

.24

.37

4. Total sleep duration

.31

.17

.14

5. Overall quality of sleep

.38

.31

.21

.57

6. Sense of wellbeing during the day

.30

.27

.21

.55

.52

7. Functioning (Physical and mental) during the day

.30

.23

.19

.54

.49

.71

8. Sleepiness during the day

.17

.21

.12

.44

.38

.49

49

.47

Item 8

Table 9. Two-week test-retest reliability of Sleep Quality Index (n=256).

Pearson’s correlation coefficient Time to fall asleep

0.63**

Suffered from insomnia

0.62**

Difficulties falling asleep

0.54**

Disturbed night sleep

0.37**

Awakening during the night

0.39**

Sleepiness in the morning

0.49**

Waking up too early

0.39** -0.01

Use of hypnotics

0.66**

Total score ** p < 0.01 (2-tailed).

50

Table 10. Two-week test-retest of Insomnia severity index (n=256)

Pearson’s correlation coefficient 0.64**

Perceived severity of insomnia (initial)

0.65**

Perceived severity of insomnia (middle) Perceived severity of insomnia (late)

0.46**

Degree of satisfaction

0.54**

Interference with daytime functioning

0.58**

Noticeability of impairment

0.51**

Concern of sleep problem

0.62**

Total score

0.75**

** p