AN EXPERIENCE OF A COURSE MANAGEMENT SYSTEM IN CONSTRUCTION SCHEDULING FRAMED WITHIN TAC-ABET ACCREDITATION CRITERIA Tulio Sulbaran, Ph.D 1

Abstract – The delivery media used in construction engineering technology (as in many other majors) in higher education is evolving to take advantage of new information technology resources. However, there is lack of data supporting the educational impact of these new information technology resources in construction and engineering education. The work presented herein describes an experience in implementing one of the many information technology resources available - Course Management System (CMS). The CMS was implemented in a construction scheduling class and its evaluation was framed within TAC-ABET criteria to support the accreditation process. This work followed a qualitative research methodology. The construction scheduling content was disseminated to the students using the CMS during a complete semester. At the end of the semester the students received a survey that was developed to mainly assess the impact of CMS on the students’ ability to demonstrate Criterion 1 of the TAC-ABET accreditation. The results from the evaluation indicate that the students benefit from the CMS and that this type information technology resource can be used to support the TAC-ABET accreditation process. Therefore, the work described in this paper has the potential to directly benefit construction education and serve as an example of good practice in engineering education. Keywords: Course Management System, Scheduling, TAC-ABET, Survey

INTRODUCTION TO COURSE MANAGEMENT SYSTEMS The delivery media used in construction engineering technology (as in many other majors) in higher education is evolving to take advantage of new information technology resources. Among the many information technology resources available are the Course Management Systems (CMS). A CMS is a tool that allows instructors, universities, and corporations to develop and support online education [UMUC 2005]. Currently, there are several CMS available with a wide range of features and cost associated. Table 1 presents some of the CMS currently available. As stated by Morgan, CMS are an increasingly important part of academic systems in higher education [Morgan, 2003]. The importance of CMS is particularly accentuated in construction education where there is a great demand for construction professionals. Therefore, construction programs are beginning to take advantage of new information technology resources such as CMS to meet the demands of the construction industry. Thus, the work presented in this paper describes the experience in implementing a CMS for a course in a construction program. The CMS was implemented in a construction scheduling class and its evaluation was framed within TAC-ABET criteria to support the accreditation process.

1 Assistant Professor – School of Construction at the University of Southern Mississippi, Box 5138, Hattiesburg, MS, 39406. E-mail: [email protected].

2006 ASEE Southeast Section Conference

Table 1. Sample of Course Management Systems Angel ATutor Avilar WebMentor Bazaar BlackBoard Bodington BSCW CentraOne CHEF Claroline ClassWeb Click2learn Colloquia COSE Coursemanager CourseWork

Desire2Learn eCollege Educator EduSystem Eledge Embanet ETUDES FirstClass Fle3 Groove Workspace HTMLeZ ILIAS Internet Course Assistant IntraLearn Janison Toolbox Jenzabar Internet Campus Solution

Jones education KEWL KnowEdgeeLearning Suite Learnwise LON-CAPA ManhattanVirtual Classroom MimerDesk Moodle Sakai Teknical Virtual Campus TeleTop The Learning Manager Unicon Academus Virtual U WebCT Whiteboard

METHOD The Participants In an effort to provide a more enriching learning experience to students in the Construction Engineering Technology (BCT) program at the University of Southern Mississippi, this project involved student participating in the Construction Planning and Scheduling course (BCT 458). This project took place during Fall 2004 with nineteen (19) students participating. Most of these students had junior or senior standing. The Course Management System in Construction Scheduling The implementation of the Course Management System (CMS) in construction scheduling was designed to encourage students to access the class material from any place at any time to foster students’ capacity to demonstrate their abilities in their corresponding field as outlined by Criterion 1 of TAC-ABET. Each BCT student was assigned an individual access code to the CMS for scheduling, which allowed the student to retrieve class information at any time from any location inside or outside the university campus. The information and activities in the CMS for scheduling were grouped in eight categories/segments as shown in table 2. Table 2. Categories within the CMS for Scheduling 12345678-

Syllabus Lecture Material Assignments Quiz/Exam Student Guide & Instructions Mail (User to User) Discussions (Board) Related Links

The “Syllabus” segment contained information such as: class textbook, class objectives, assessment tools/weights, policies, class topics, among others. The “Lecture Materials” provided the students with worksheet with some

2006 ASEE Southeast Section Conference

information about each lecture and spaces for the student to take notes during class. The “Student Guide & Instructions” included detailed help regarding the use of the CMS. The “Mail (User to User)” allow a student to send private messages to another student. The “Discussion (Board)” provided an avenue for the student to share comments and questions will all other students. The “Related Links” provided access to scheduling related materials located in the Internet. The students were encouraged to print from the “Lecture Materials” the student worksheet prior to coming to class. The worksheets allow the student to take notes in a structured way. The assignments were submitted electronically prior to the class meeting time and the faculty downloaded the presentation into a central computer before the class. The quizzes were taken on the CMS outside the classroom and the exams done on a computer during class time. Throughout the semester, it was emphasized that the CMS was intended as a course supplement (not a substitution of the classes) to enhance the educational experience of the active participants of the scheduling course. The Survey During the sixteen (16) weeks course, the students were encouraged to give their perspective to the faculty in a free flow format regarding the progress of the course. Additionally, at the end of the course a survey was given to each student. This survey was developed to assess the students’ perspective of the CMS framed within Criterion 1 of the TAC-ABET accreditation. Criterion 1 of the TAC-ABET accreditation states that the institutions are responsible to demonstrate that the graduates have (TAC-ABET 2000): a. an appropriate mastery of the knowledge, techniques, skills, and modern tools of their disciplines, b. an ability to apply current knowledge and adapt to emerging applications of mathematics, science, engineering, and technology, c. an ability to conduct, analyze, and interpret experiments and apply results to improve processes, d. an ability to apply creativity in the design of systems, components, or processes appropriate to program objectives, e. an ability to function effectively on teams, f. an ability to identify, analyze, and solve technical problems, g. an ability to communicate effectively, h. a recognition of the need for, and an ability to engage in lifelong learning, i. an ability to understand professional, ethical, and social responsibilities, j. respect for diversity and a knowledge of contemporary professional, societal, and global issues, and k. a commitment to quality, timelines and continuous improvement. In addition to frame the survey within the Criterion 1 of the TAC-ABET accreditation, parameters for the development of a scholarly survey were considered. There are a variety of good references for scholarly survey and assessment. The reference that most influenced the survey used in this work was “Designing and Conducting Survey Research, by Rea, L., Parker, R. Jossey-Bass Publishers, 1997” (Rea 1997). Following are the main parameters considered for the survey: a. Length of the survey: The survey was limited to 8 minutes. This time was selected, because the longer the survey, the less willing is the participant to provide information or even worse, the information could be adversely affected. b. Number of pages: The survey was limited to 2 pages. This number of pages was selected, because it plays an important psychological effect on the participants. Participants tend to think that fewer pages require less time to complete. This was achieved by reducing the text size and listing the options across rather than vertical. c. Minimum demographic questions: The survey had a minimum number of demographic questions. Standard questions such as marital status and salary were considered not relevant. Therefore, they were not included. This allows the survey to focus more on the multidisciplinary learning experience of the students. d. Use of categories: The survey used categories for questions related to grades (Overall Grade Average, Class Grades, and Anticipated Project Grade). This was done because some students might be reluctant

2006 ASEE Southeast Section Conference

e. f. g. h.

i. j.

to provide their grades, however they will be willing to provide the range of their grade. Additionally, the use of categories also facilitated the analysis of the data. Use of scales: The survey used Likert scale. This was used to attempt to measure students’ perspective on the multidisciplinary project framed within the Criterion 1 of TAC-ABET. Pre-testing: The survey was pre-tested with 5 people. This was done to screen the survey before the data-collection. Simple wording: The survey used simple, clear, straightforward and brief wording. Additionally, the wording was carefully selected within the framework of criterion 1 of TAC-ABET. Close and open-ended questions: The survey was mainly composed of close-ended questions (questions that provide a fixed number of alternative responses). This was done because this type of questions allows easier comparison and analysis of the responses. The survey also included some open-ended questions. These questions were included to overcome the constraints of fixed alternative responses in the close-ended questions. Questions sequencing: The survey was carefully prepared to place questions in a logical sequence. The questions were also grouped to facilitate the answers without distorting the data. Reliability: The survey had split-half for internal consistency. This was used, because, as stated by Gay, it is a commonly used internal consistency procedure that requires only one administration of the survey. This one time administration eliminates certain sources of error, such as difference in survey conditions, which can occur in a test-retest situation (Gay 1996)

RESULTS AND DISCUSSION At the end of the project, the survey was administered to students. Fifteen out of the 19 participating students filled out the survey. The participants’ demographic information is presented in Table 1 discriminated by discipline. It can be observed in Table 3 that most of the participants were senior, Caucasian students in their early 20s. Table 3. Demographic Information of the Multidisciplinary Project Participants. Gender

Male Female

Students 14 (93%) 1(7%)

Age (Years)

Minimum Average Maximum

22 23.2 26

Race

Caucasian African American Asian Oriental Hispanic No Response

10 (66%) 1 ( 7%) 4 (27%)

Class Standing

Freshman Sophomore Junior Senior Graduate No Response

2 (13%) 9 (60%) 1 ( 7%) 3 (13%)

2006 ASEE Southeast Section Conference

To check for the survey reliability, the twenty (20) questions framed within TAC-ABET were divided in two equivalents halves. A correlation coefficient of both halves was calculated using Pearson. After calculating the coefficient, they were corrected using the Spearman-Brown prophecy formula to consider the reliability of the complete survey and not only half of it. The final correlation coefficient obtained for the survey was 0.88. Considering that for a positive correlation the coefficient could range from 0 (no correlation) to 1 (exact correlation), the correlation coefficients obtained for this survey indicate that there is a very high level of consistency within the survey. Meaning that the survey gathered the students’ perspective consistently. The students’ perspective regarding the opportunity given by the CMS to foster their competences within the Criterion 1 of the TAC-ABET are presented in table 4. Table 4 presents in the first column the sub-criteria within criterion 1 of the TAC-ABET considered on this work. The second shows the distribution of students’ answers for each sub-criteria. And the last column, presents the total number of answers obtained for each sub-criteria. In Table 4, it can be observed that most of the students strongly agreed or agreed that the CMS gave them the opportunity to enhance their learning experience. This learning experience directly correlates with the TAC-ABET accreditation criterion 1. It is interesting to notice that regarding sub-criteria ‘e’ (an ability to function effectively on teams), 47% of the students strongly disagree or disagree that the CMS gave them the opportunity to effectively function on a team. Similarly, a good percentage of the students (34%), strongly disagree or disagree that the CMS gave the opportunity to effectively communicate (sub-criteria ‘f’). This is very reasonable because most of the communication among the students happened in person or through personal e-mails not through the CMS discussion (board) or mail (user to user), limiting the students view of the use of CMS to develop team collaboration and facility communications. Regarding sub-criteria ‘j’, it is observable that the students had the strongest disagreement, 50% of the students strongly disagree or disagree that the CMS enhanced their respect for diversity and knowledge of contemporary professional, social and global issues (sub-criteria ‘j’). It is possible that this was due to the fact that most of material presented in the CMS was scheduling content without explicitly addressing this sub-criteria. In addition to the TAC-ABET criteria several other questions were ask to the students. So, when the students were asked the question “How helpful was IN GENERAL the on-line material? 47% of the students answered either “Extremely Beneficial” or “Very Beneficial”, and none answered “Not Very or Not at all”. When the students were asked the question “How helpful was the SYLLABUS section of the on-line material?” 53% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the LECTURE MATERIAL section of the on-line material?” 94% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the ASSIGNMENTS section of the on-line material?” 60% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the QUIZ/EXAM section of the online material?” only 26% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the STUDENT GUIDE&INSTRUCTIONS section of the online material?” only 33% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the MAIL (USER TO USER) section of the online material?” only 26% of the students answered either “Extremely Beneficial” or “Very Beneficial”. When the students were asked the question “How helpful was the DISCUSSION (BOARD) section of the on-line material?” only 26% of the students answered either “Extremely Beneficial” or “Very Beneficial”. It is interesting to notice that when the students were asked the question “Should this on-line material be accessible to the upcoming students? ” 100% of the students answered either “Yes”.

2006 ASEE Southeast Section Conference

Table 4. Students’ Level of opinion of the opportunity given by the CMS as they pertain to the TAC-ABET Criterion 1.

TAC-ABET Criterion 1 a. an appropriate mastery of the knowledge, techniques, skills, and modern tools of their disciplines b. an ability to apply current knowledge and adapt to emerging applications of mathematics, science, engineering, and technology,

Distribution of Students’ Answers BCT Participants SA A U D SD 8 13 8 1 (27%) (43%) (27%) (3%)

Total 30

8 11 8 3 (27%) (37%) (27%) (10%)

30

c. an ability to conduct, analyze, and interpret experiments and apply results to improve processes,

8 14 6 1 (28%) (48%) (21%) (3%)

-

29

e. an ability to function effectively on teams

14 18 19 9 (23%) (30%) (32%) (15%)

60

f. an ability to identify, analyze, and solve technical problems,

7 8 10 5 (23%) (27%) (33%) (17%)

30

g. an ability to communicate effectively,

9 11 5 5 (30%) (37%) (17%) (17%)

30

h. a recognition of the need for, and an ability to engage in lifelong learning

11 10 7 2 (37%) (33%) (23%) (7%)

-

30

j. respect for diversity and an knowledge of contemporary professional, societal, and global issues, and

6 9 10 5 (20%) (30%) (33%) (17%)

30

k. a commitment to quality, timelines and continuous improvement

8 12 8 2 (27%) (40%) (27%) (7%)

30

-

Note: Strongly Agree (SA), Agree (A), Undecided (U), Disagree (D), Strongly Disagree (SD) Top Number represents numbers of students' answers Bottom (Between parenthesis) represents percentage within the discipline

2006 ASEE Southeast Section Conference

SUMMARY This attempt at a University in Mississippi to implement a Course Management System (CMS) in a construction scheduling class proved to be an excellent learning experience for the students and for the faculty participating. The students had the opportunity to have their regular classes supplement with on-line material to enhance the educational experience of the active participants in the scheduling course. The CMS enhanced the students learning experience by fostering participants’ capability to access the material at any time for any location. Additionally, this type of project can be used to support the TAC- ABET accreditation process by addressing Criterion 1. Finally, the wealth of feedback given by the participants will help improve this project for future deployment

ACKNOWLEDGEMENT The author would like to recognize the valuable feedback given by the students.

2006 ASEE Southeast Section Conference

REFERENCES ADE (2005). “ Accessibility in http://www.umuc.ed/ade/glossary.html

Distance

Education:

Glossary”,

University

of

Maryland,

URL:

Gay, L.R. (1996) Educational Research Competencies for Analysis and Application (5th Ed.). New Jersey: Merril Morgan, G (2003). “Key findings: Faculty Use of Course Management Systems”, Educause Center for Applied Research. Rea, L.,& Paker., (1997). Designing and Conducting Survey Research. San Francisco: Jossey-Bass Publishers TAC-ABET (2000). Criteria For Accrediting Engineering Technology Programs. Baltimore: ABET

2006 ASEE Southeast Section Conference

Tulio Sulbaran Ph.D Tulio Sulbaran is an Assistant Professor at the University of Southern Mississippi’s School of Construction and is the director of the Innovation for Construction and Engineering Enhancement (ICEE) center. He received his BS in Civil Engineering from the University Rafael Urdaneta in Venezuela and his Ph.D in Civil Engineering from the Georgia Institute of Technology. His research interest is on the impact of information technology resources on construction and engineering education and training.

2006 ASEE Southeast Section Conference