Building a Multi-Level Model of Individual E-Learning Effectiveness Dr. Chieh-Ling Lo, Assistant Professor, Mingdao University, Taiwan

ABSTRACT In the era of the knowledge economy and lifelong learning, many enterprises with foresight have realized that high-quality manpower is an important strategic resource that ensures their competitiveness. Therefore, they expect to improve staff performance and enhance manpower quality by training activities, and finally achieve the goal of a long-lasting operation. Due to the fast development of information technologies and the accessibility of network technologies, E-learning has become a new and important training method. Therefore, it is necessary to evaluate the effectiveness of E-learning and understand the related influencing factors. After reviewing the related literature, it was discovered that although the factors that influence the effectiveness of E-learning have multiple levels from a theoretical perspective, when it comes to empirical analysis, most studies deal with variables on a single level, which may result in incorrect inference. It is impossible to understand the operational situations of E-learning systematically and comprehensively without a multi-level analysis. Therefore, this study will develop a model to discuss the influences of multi-level factors on individual E-learning effectiveness. Based on this model, empirical research will be conducted in the future. Keywords: E-learning effectiveness, multi-level model, personal factors, technical environment factors, social environment factors INTRODUCTION The 21st century is the era of the knowledge economy and lifelong learning. For an individual, brainpower is a key to success, and knowledge learning is an important approach to changing ideas (Chou, 2008). Many enterprises with foresight have realized that, in this constantly changing business environment, high-quality manpower has become an important strategic resource that can ensure their competitiveness. Therefore, enterprises expect to improve staff performance and enhance manpower quality by training activities, and finally achieve the goal of a long-lasting operation (Chao, Hsu, Lin & Yeh, 2004; Huang, Wang & Lee, 2009). The introduction of E-learning is one of the methods used to assist enterprises’ innovation, and has caused significant changes to the processes of learning and training. According to a report from Taiwan’s Common Wealth Magazine, in 2006, the gross output value of enterprises’ E-learning reached USD 23 billion, and the value was estimated to reach USD 50 billion in 2010 (Chou, 2008). This demonstrates the growing acceptance of E-learning by different social sectors. Therefore, it is necessary to evaluate the effectiveness of E-learning and understand related influencing factors. Evaluating the effectiveness of E-learning is helpful for understanding whether learners really gain something, whether the overall plan of the learning activity shows benefits, whether the goal of the education has been achieved and what adjustments are needed. All of these will enable the refinement of the training processes and make them more suitable for the development of the enterprise.

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

71

After reviewing the study content of related literature, factors that influence E-learning effectiveness are classified into three categories. The first category is related to technologies. These are topics related to the platforms of the E-learning systems and designs of the curriculum content, such as the usability of systems, the simplicity of systems, the richness of the media, the system quality, the system service quality, and the quality of the curriculum content (Yu, 2007; Johnson, Gueutal, & Falbe, 2009; Ozkan, Koseler, & Baykal, 2009; Wu & Huang, 2010; Garavan, Carbery, O＇Malley, & O＇ Donnel, 2010). The second category is related to behavior, referring to the traits of E-learning learners, such as study strategies, E-learning involvement, computer self-efficacy, flow experience, the computer anxiety of learners and study attitudes (Chao, Hsu, Lin, & Yeh, 2004; Johnson, Gueutal, & Falbe, 2009; Ozkan, Koseler, & Baykal, 2009; Huang, Wang, & Lee, 2009; Wu & Huang, 2010; Garavan, Carbery, O ＇Malley, & O＇Donnel, 2010; Yu, Hwang, Yu, Cheng, & Yang, 2010). The third category is related to organizational and social factors, including organization cultures (culture of learning and knowledge sharing), motivation factors, attitudes of teachers, environmental universality and support level, peer-interactions, collectivism, team force and ability to adapt to changes (Yu, 2007; Ozkan, Koseler, & Baykal, 2009; Wu & Huang, 2010; Hassan, 2010). These E-learning effectiveness influencing factors have multiple levels from a theoretical perspective, but when it comes to empirical analysis, most studies deal with variables at a single level, which may result in incorrect inferences. A multi-level analysis can resolve biased errors caused by multi-level variations, and analyze the influences of multi-level variables on the learning results. Falconer (2006) points out that learning activities in organizations should be divided into different levels, and it is impossible to understand the operational situations of E-learning systematically and comprehensively without multi-level analysis (Ozkan, Koseler, & Baykal, 2009; Chu & Chu, 2010; Wu & Huang, 2010). Therefore, in order to avoid the above mentioned errors caused by multiple levels and due to the maturing of multi-level analysis skills, this study will develop a model to discuss the influences of multi-level factors on individual E-learning effectiveness. Based on this model, empirical research will be conducted in the future. LITERATURE REVIEW Computer Self-Efficacy Computer self-efficacy refers to an individual’s awareness of their ability to implement computer-related tasks with different information media tools. It is a very important factor in E-learning because all curriculum materials and interactions are completed by information technology (Johnson, Gueutal, & Falbe, 2009). Computer self-efficacy stands for an individual’s judgment of their ability to use computers. It doesn’t refer to simple skills, such as disk formatting or inputting formulas on spreadsheets, but to an integrated judgment on whether an individual can apply these skills to wider and more complicated tasks. Computer self-efficacy can assist an individual to work with computers correctly and efficiently (Hsu, Shen, Shiau, Chuang, & Chen, 2008). Scholars believe that network self-efficacy can play a considerably important role in intelligence acquirement and the communication of internet activities (Yukselturk & Bulut, 2007; Chu & Tsai, 2009). It presents an individual’s confidence or self-belief, the belief that they are capable of using network functions, and has an influence on the success of online curriculums (Chu & Chu, 2010). Considering the above ideas, this study defines computer self-efficacy as an individual’s awareness and judgment of their level of ability to use related information technology tools, such as computers and networks. It is an important influencing factor for the implementation of information technology related tasks.

72

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

E-learning Involvement Antil (1984) believes that involvement refers to the levels of importance and interest that an individual feels under certain stimulations in certain circumstances. Zaichkowsky (1985) believes that involvement can explain the level of an individual’s feeling of relevance and attention towards something (Yu, Hwang, Yu, Cheng, & Yang, 2010). Hsu (2001) tends to define involvement as the importance and relevance of something perceived by an individual in a certain circumstance and is based on the user’s demands, values and interest. With respect to an information system, user involvement refers to the level of participation of potential users during the development of a system (Hsu, 2001). Therefore, when staffs show a higher level of involvement in E-learning, they will store more knowledge, memory, information and experience during the learning processes, which will lead to more specific effectiveness in information sharing and thinking in the future. The level of involvement in E-learning can be defined as the level of relevance and attention the staffs feel towards E-learning (Yu, Hwang, Yu, Cheng, & Yang, 2010). Flow Experience Flow theory was first proposed by psychologist Csikszentmihalyi. It mainly discusses humans’ psychological states when they give full commitment to a certain behavior or activity, which is considered to be flow experience. Flow is a kind of psychological state caused when an individual is completely immersed in an activity because of their own interest. They focus on things they are interested in, and lose irrelative consciousness. It seems as if they are sucked into the activity. Therefore, it is also often called “immersion” (Chiu, 2008). Currently, the flow theory has been widely applied to studies in many fields, such as sport, work, consumption, games, hobbies and computer usage. After the emergence of networks, scholars started to discuss flow phenomena during network interactions successively. However, few studies have been carried out to discuss learners’ behavior and performance in E-learning using the flow theory (Chao, Hsu, Lin, & Yeh, 2004). Hoffman and Novak (1996, 1997) defined flow experience in the environment of computer communication as the process starting from the sense of joy gained from interaction with the computer. With the disappearance of self-perception, an individual is completely immersed in the related environment (Huang, Chiu, Sung, & Farn, 2011). When staffs produce a flow experience during E-learning, they will be deeply involved in the E-learning situation and exploration. Such a state will encourage users, bring about positive feelings and satisfaction, and lead to further exploration (Chao, Hsu, Lin, & Yeh, 2004). Ghain and Deshpande (1994), Chou and Ting (2003), and Chiu (2008), believe that “concentration” and “joy” are connotations of flow experience and, when applying them to E-learning, concentration refers to the level of an individual’s commitment to the E-learning environment and joy refers to an individual’s level of happiness experienced in the E-learning environment. Technical Environment Factors In addition to learners’ traits, some scholars believe that technical environment factors will influence E-learning effectiveness. Related technical environment factors include: reliability, social presence and media synchronicity (Johnson, Gueutal, & Falbe, 2009); technology reliability and conditions of peer-interactions (Johnson, Hornik, & Salas, 2008); technical infrastructure or skills, methods of scientific technology and specialized knowledge (Trist, 1981; Pasmore, 1988; Molleman, & Broekhuis, 2001); ease of use of the information science and technology, media richness and the usability of scientific technologies (Wu and Hwang, 2010); information quality, system quality and service quality

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

73

(DeLone & McLean, 2003; Holsapple & Lee-Post, 2006); system interactivity (the level of communication and sharing when an individual is experiencing electronic learning), interface designs (in addition to concerns on basic information elements, such as characters, images, audio signals, animation and video signals, more attention is paid to learners’ tendencies) and the quality of scientific technologies (quality of the curriculum content of the distance education system) (Chen & Hsu, 2007). On the other hand, Ozkan, Koseler and Baykal (2009) believe that a learning management system should be examined in a systematic way, and proposed a learning management framework with six levels, including three scientific technology factors, two social factors and one support factor. This study believes the scientific technology factors proposed are quite complete, including ideas for the theory of planned behavior and the technology acceptance model, and are seldom studied. Therefore, we will carry out further discussion in this current study on connotations of these scientific technology factors, including: 1. system quality (ease of use, safety, stability, providing options of assistance programs, easy to use teaching tools, good organization and fast speed); 2. service quality (learner tracing function, curriculum authorization, tools for teaching designs, curriculum management, richness in knowledge and safety); 3. content quality (teaching material management, curriculum flexibility, interactions, providing tests, providing case discussion, well-written and expressed, providing slides, personal tutorials available, timely updating of teaching materials, proper teaching length and well-organized teaching materials). Social Environment Factors After the literature collection and sorting, this study plans to discuss the influences of two social environment factors, external incentive measures and learning culture, on E-learning effectiveness, as these two variables have not been discussed much in previous studies. With regard to external incentive measures, this study will use the study results of Liu and Tan(2008) as the basis for development. Liu and Tan(2008) divided the methods that will effectively encourage staff E-learning into two levels, namely, substantial level (including increasing benefits, providing well-being and punishment) and psychological level (including satisfying demands, pursuit of fairness and fulfilling expectations). In addition, in terms of learning culture, Piers (2003) believes that the learning culture is an important influencing factor for the success of E-learning strategy (Kuo, 2008). Watkins and Marsick (1993) discuss an organization’s learning culture from the perspective of human resources, and emphasize that a learning organization should have its own learning culture. They also believe that a learning organization stays in a continuously learning and evolving process, and learning is the combination process between strategy and work. The participating parties include organization members, working groups and the whole organization. The results of learning bring about changes to knowledge, beliefs and behavior, and strengthen the innovation and growth of the organization. Hsieh (2005) believes that the learning culture of an organization is the culture of continuous learning, and, in such an organizational culture, staffs are both employees and learners, and whether individuals, groups or the whole organization, they will carry out continuous learning. Lien (2002) and Cheng (2007) believe that a learning culture should include three levels; individuals, groups and the organization. In order to conform to the aim of this study, only the connotation of the learning culture at an organizational level will be mentioned in the following discussion. The learning culture at organizational level in this study includes the four dimensions of system establishment, system connection, strategy guidance and empowerment.

74

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

Individual E-learning Effectiveness Different from traditional enterprise training, E-learning features cross-fields, is simultaneous, has cost benefits, convenience and is open for learning. It can assist enterprises to deliver study resources flexibly, and guide staff to carry out learning activities under quickly-changing information technologies. Currently, more and more enterprises have been active in introducing E-learning. Therefore, it is worthwhile to deeply explore the effectiveness of E-learning. Chou and Liu (2005) believe that the measurement of learning effectiveness can be divided into three parts; learning accomplishment (the production function), satisfaction of learning and the level of peer-support, and group's perception of well-being. Yu (2007) carried out measurements of learning effectiveness using learners’ personal beliefs, including three variables of involvement and participation, usage intention and comparative advantages. Johnson, Gueutal and Falbe (2009) measured individual learning effectiveness by course scores, estimation of course utility and satisfaction with curriculums. Chu and Chu (2010) measured individual learning effectiveness through learning awareness, continuous commitment and satisfaction. Lim, Lee and Nam (2007) believe that the performance of E-learning refers to the levels of learners’ working knowledge, skills, and attitude improvements after receiving training courses, and point out that the measurement of training effectiveness should not only include the level of accomplishment learning, but also the transference level of learning. After reviewing the literature collected, we found that Chung & Yang (2006), and Chang & Lo (2009) have proposed comparatively complete indices. As this study focuses on the discussion of E-learning effectiveness, criterions will be selected related to the level of individuals from the above two studies as a measurement of variable connotations, and will include the dimensions of learning and behavior. DEVELOPMENT OF PROPOSITIONS In a study on adopting new computer technologies, Hill, Smith and Mann (1987) demonstrated that self-efficacy influences an individual’s intention to take a certain action. In studies related to computer acceptance behavior and E-learning, scholars (Yu, 2007; Chiu, Lin, & Shih, 2007; Huang, Wang, & Lee, 2009) clearly point out that self-efficacy is an important influencing factor. The higher the computer self-efficacy is, the higher the learning performance is (Joo et al., 2000; Tasi & Tasi, 2003). Learners with a high level of computer self-efficacy have comparatively positive computer learning experience and are optimistic about E-learning environment. They believe they will succeed in an E-learning environment and will show comparatively high satisfaction. This shows that there is a positive relationship between computer self-efficacy and E-learning effectiveness, and the first study proposition is proposed as follows: P1: Computer self-efficacy has a significantly positive influence on individual E-learning effectiveness. Involvement usually refers to potential users’ behavior or activities during the development of a system. With respect to social science research, involvement mainly reflects an individual’s subjective beliefs and feelings towards some topics (Barki & Hartwick, 1994). When applying the concept of the involvement level into E-learning, it represents the levels of relevance and attention that the staffs of an organization feel towards E-learning (Yu, Hwang, Yu, Cheng, & Yang, 2010). Therefore, we propose that the involvement level of E-learning will also significantly influence the individual E-learning effectiveness, and the second study proposition is proposed as follows: P2: E-learning involvement has a significantly positive influence on individual E-learning effectiveness.

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

75

Flow is a kind of temporary and subjective experience and is why people are willing to continue certain activities (Chao, Hsu, Lin & Yeh, 2004). In E-learning related studies, few studies have included flow experience in their study targets and, according to literature, flow experience will also significantly influence individual E-learning effectiveness. Therefore, the third proposition is proposed as follows: P3: Flow experience has a significantly positive influence on individual E-learning effectiveness. Wu and Huang’s (2010) study takes college students as study samples, and establishes the relationship between “learning attitude influences the usage of E-learning, and the usage of E-learning further influences learning performance” at the personal level. They believe that learning attitude represents learners’ love of taking courses, and that they are immersed in the joy of participating curriculum activities. Good learning attitude can be regarded as an internal motivation and enthusiasm. Saade et al. (2007) point out that when students have stronger internal motivation, they will be more willing to use E-learning. Therefore, this study proposes that flow experience is a kind of learning attitude, and will influence an individual’s learning performance by influencing their usage of E-learning, and thus the fourth proposition is proposed as follows: P4: Usage of E-learning has a mediating effect on the relationship between flow experience and individual E-learning effectiveness. According to ideas of the social-technical system, factors at the technological and social levels will influence E-learning effectiveness. Many scholars (Islas et al., 2007; Liaw, Huang, & Chen, 2007; Ozkan, Koseler, & Baykal, 2009; Wu & Huang, 2010) have demonstrated this, and thereby the fifth proposition and the sixth proposition are proposed as follows: P5: Technical environment factors have significant influences on individual E-learning effectiveness. P6: Social environment factors have significant influences on individual E-learning effectiveness. Wu and Huang (2010) took college students as study respondents, and adopted a multi-level statistical analysis to understand the influences of learning environment factors and personal environment factors on E-learning effectiveness. Study results show that E-learning effectiveness is directly influenced or moderated by factors of the technical system and social system at both the learning environment level and personal environment level alternatively or simultaneously. Thereby, the seventh to the ninth propositions are proposed as follows:

76

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

Technical Environment ．System Quality ．Service Quality ．Content Quality

Social Environment ．External Encouragement ．Learning Culture

H5

Organizational Level H7

H6

---------------------------------------------------------------------------------------------------Individual Level

H8 H1

H9

Computer Self-Efficacy H2

Individual E-Learning Effectiveness ．Learning ．Behavior

E-Learning Involvement H4 Flow Experience

Usage of E-Learning H3 Figure1: Research Model

P7: Technical environment factors have significantly positive influences on the usage of E-learning. P8: Social environment factors have significantly positive influences on the usage of E-learning. P9: Social environment factors have moderating effects on the relationship between usage of E-learning and individual E-learning effectiveness. Figure1 shows the multi-level model of this current research which is the base to conduct empirical test in the future. CONCLUSION Due to the development of information technologies and the accessibility of network technologies, great changes have taken place in the processes of learning and educational training, and E-learning has now become a new and important method. Webster and Hackley (1997), McCray (2000), and Hiltz and Turoff (2005) believe that using networks and technological media owned by an enterprise to carry out E-learning will satisfy the demands of the enterprise and the staff with regards to learning and gaining knowledge more efficiently, and reducing the expenses of on-the-job training (Yu, 2007). Liaw, Huang, & Chen (2007), and Hassan (2010) have all compared E-learning with traditional teaching, and discovered the following advantages: provided flexibility with time and space, reducing enterprises’ costs for money and time; speeds up self-oriented learning where learners are centered; constructs a cooperative learning environment; creates wide-ranging learning groups; E-learning can be delivered by different channels, and can effectively update or maintain knowledge. The US has also verified the One-Third Principle, i.e., the application of electronic technologies can reduce teaching costs by one third, reduce the teaching

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

77

duration by one third, and increase the knowledge or skills acquired within the same period by one third (Chou & Liu, 2005; Lim, Lee, & Nam, 2007; Liu & Tan, 2008). According to the above explanation, E-learning has become a learning method for accumulating human capital and intelligence capital, and has attracted more and more attention, which further demonstrates the necessity to understand the influencing factors for E-learning effectiveness. This study has analyzed and sorted the related literature, and has proposed related propositions. In the future, empirical materials will be collected by questionnaire survey, and multi-level analysis techniques will be adopted to explore the influences of correlation factors at individual and organizational levels on individual E-learning effectiveness, aiming to make further contributions to e-learning related theories and operation practice. ACKNOWLEDGMENT This article was supported by a grant from the National Science Council, Taiwan, NSC 101-2511-S-451-001. REFERENCES Antil, J. H. (1984). Conceptualization and operationalization of involvement. Advances in Consumer Research, 11, 203-209. Barki, H., & Hartwick, J. (1994). Measuring user participation, user involvement, and user attitude. MIS Quarterly, 18(1), 59-82. Chou, T. J., & Ting, C. C. (2003). The role of flow experience in cyber-game addiction. Cyberpsychology & Behavior, 6, 663-675. Chao, R. M., Hsu, H. Y., Lin, M. K., & Yeh, Y. H. (2004). The study of using key account management to analyze kibs' workers of e-learning--aspects of knowledge sharing type, flow experience and human research performance. Journal of E-Business, 6(1), 27-52. Chou, S. W., & Liu, C. H. (2005). Learning effectiveness in a Web-based virtual learning environment: a learner control perspective. Journal of computer assisted learning, 21, 65–76. Chung, R. G., & Yang, P. W. (2006). To develop the evaluation indicators for effectiveness of corporate e-learning. Journal of Human Resource Management, 6(1), 123-140. Chen, M. S., & Hsu, K. P. (2007). The effects of asynchronous e-learning situation on learning perception and usage intention. Journal of Human Resource Management, 7(3), 25-44 Cheng, K. H. (2007). The impact of organizational learning culture and employees'' motivation to transfer on transfer of training. Unpublished Master’s Thesis, National Chung Cheng University, Chaiyi, Taiwan. Chiu, Y. W, Lin, Y. M., & Shih, D. H. (2007). The effects of technology characteristics, task characteristics and computer self-efficacy on behavioral intention of electronic learning. Journal of E-Business, 9(2), 235-266. Chiu ,H. G. (2008). Exploring the factors influencing the behavioral intention of using instant messenger. Journal of Information, Technology and Society, 8(1), 1-19 Chou, C. Y. (2008). The e-learning effectiveness of Taiwan E-Learning center. Personnel Monthly, 46(2), 36-41. Chang, C. S., & Lo, P. H. (2009).

To explore the e-learning effectiveness evaluation indices in the public sectors-take civil service

development center as an example. Educational Research, 177, 63-77 Chu, R. J., & Tsai, C. C. (2009). Self-directed learning readiness, Internet self-efficacy, and preferences toward constructivist Internet-based learning environments among adult learners. Journal of Computer Assisted Learning, 25, 489–501. Chu, R. J., & Chu, A. Z. (2010). Multi-level analysis of peer support, Internet self-efficacy and e-learning outcomes–The contextual effects of collectivism and group potency. Computers & Education, 55, 145-154. DeLone, W. H., McLean, E. R. (2003). The Delone and Mclean model of information systems success: a ten-year update. Journal of Management Information System, 19(4), 9-30.

78

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

Garavan, T. N., Carbery, R., O’Malley, G. & O’Donnell, D. (2010). Understanding participation in e-learning in organizations: a large scale empirical study of employees. International Journal of Training and Development, 14(3), 155 -168. Ghani, A. J., &. Deshpande, P. S (1994). Task Characteristics and the experience of optimal flow in human – computer interaction, The Journal of Psychology, 4(128), 381-389. Hassan, M. S. A. (2010). Hybrid e-learning acceptance model: learner perceptions. Decision Sciences Journal of Innovative Education, 8(2), 313-346. Hill, T., Smith, N. D., & Mann, M. F. (1987). Role of efficacy expectations in predicting the decision to use advanced technologies: The case of computers. Journal of Applied Psychology, 72 (2), 307-313. Hsu, H. Y. (2001). The study in relation between internet involvement and contents of internet recruitment. Unpublished Master’s Thesis, National Taiwan University of Science and Technology, Taipei, Taiwan. Hiltz, S. R., & Turoff, M. (2005). Education goes digital: The evolution of online learning and the revolution in higher education. Communications of the ACM, 48 (10), 59-64. Hsieh, H. L. (2005). A study of knowledge management and the fifth discipline among universities: the moderating effect of organizational learning culture. Journal of Humanities and Social Science, 1(7)，45-73. Holsapple, C. W., & Lee-Post, A. (2006). Defining, assessing, and promoting e-learning success: an information systems perspective. Decision sciences journal of innovative education, 4(1), 67-85. Hsu, P. Y., Shen, Y. H., Shiau, W. L., Chuang, C. H., & Chen, S. S. (2008). User's computer self-efficacy and erp successful implementation. Journal of E-Business, 10(2), 379-402 Huang, J. T., Wang, Y. C., & Lee, T. C. (2009).

A study of the relationship of employee self-efficacy, learning strategy, and

e-learning effectiveness. K.U.A.S. Journal of Humanities and Social Sciences, 6(2), 283-306. Huang, L. T., Chiu, C. A, Sung, K., & Farn, C. K. (2011). A Comparative Study on the Flow Experience in Web-Based and Text-Based Interaction Environments. Cyberpsychology, Behavior, and Social Networking, 14(1-2), 3-11. Islas, E., Perez, M., Rodriguez, G., Paredes, I., Avila, I., & Mendoza, M. (2007). E-learning tools evaluation and roadmap development for an electrical utility. Journal of Theoretical and Applied Electronic Commerce Research, 2(1), 63-75. Joo, Y., Bong, M., & Choi, H. (2000). Self-efficacy for self-regulated learning, academic self-efficacy, and internet self-efficacy in web-based instruction. Educational Technology Research and Development, 48(2), 5–17. Johnson, R. D., Hornik, S. R., & Salas, E. (2008). An empirical examination of factors contributing to the creation of successful e-learning environments. International Journal of Human-Computer Studies, 66, 356-69 Johnson, R. D., Gueutal, H., & Falbe, H. C. (2009). Technology, trainees, metacognitive activity and e-learning effectiveness. Journal of Managerial Sychology, 24(6), 545 – 566. Kuo, C. T. (2008). Relationships among e-learning culture, learner control, e-learner satisfaction, perceived network externalities and self-efficacy. Unpublished Master’s Thesis, National Taipei University, Taipei, Taiwan. Lien, Y. H. (2002). Dimensions of the learning organization and organizational performance. Commerce & Management Quarterly, 3(4), 337-358. Liaw, S. S., Huang, H. M. & Chen, G. D. (2007). Surveying instructor and learner attitudes toward e-learning. Computers Education, 49(4), 1066-1080. Lim, H., Lee, S. G., & Nam, K. (2007). Validating E-learning factors affecting training effectiveness. International Journal of Information Management, 27, 22–35. Liu, C. C., & Tan, Y. J. (2008). Barriers to employees' participation and motivation methods of e-learning. Electronic Commerce Studies, 6(2)，207-228. McCray, G. E. (2000). The hybrid course: Merging on-line instruction and the traditional classroom. Information Technology and Management, 1, 307-327. Molleman, E., & Broekhuis, M. (2001). Sociotechnical systems: towards and organizational learning approach. Journal of Engineering and Technology Management, 18, 271-294.

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue

79

Ozkan, S., Koseler, R., & Baykal, R. N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 111–130. Pasmore, W. A. (1988). Designing Effective Organizations: The Socio-technical Systems Perspective, New York, NY: Wiley. Saade, R. G., He, X., & Kira, D. (2007). Exploring dimensions to online learning. Computers in Human Behavior, 23, 1721-1739. Trist, E. (1981). The Evaluation of Sociotechnical Systems, Toronto: Quality of Working Life Center. Tsai, M. J., & Tsai, C. C. (2003). Information searching strategies in web-based science learning: The role of Internet self-efficacy. Innovations in Education and Teaching International, 40, 43-50. Watkins, K. E., & Marsick, V. J. (1993). Sculpting the learning organization: Lessons in the art science of systemic change. SF: Jossey-Bass Publishers. Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40 (6), 1282-1309. Wu, W. C., & Huang, L. Y. (2010). The effectiveness of e-learning for blended courses in colleges: a multi-level empirical study. International Journal of Electronic Business Management, 8(4), 312-322. Yukselturk, E., & Bulut, S. (2007). Predictors for student success in an online course. Educational Technology & Society, 10( 2), 71-83. Yu, T. K. (2007). Technology-mediated learning performance: a comparison of technological and vocational education and private enterprise. Bulletin of Educational Psychology, 39(1), 69-90. Yu, C., Hwang, I. H., Yu, T. F., Cheng, Y. S., & Yang, Y. (2010). A study of the impact of s bank organizational memory on organizational learning--e-learning involvement inventory of employees as the adjustment variable. Operation Management Reviews, 6(2), 23-44.

80

The Journal of Human Resource and Adult Learning, Vol. 10, Num. 2, December, 2014 issue