Florida State College at Jacksonville

Florida State College at Jacksonville Detailed Assessment Report 2011-2012 Insurance Claims Adjuster (PSAV) (POS 5728) (DE) As of: 2/07/2013 03:22 PM ...
Author: Brendan Ford
2 downloads 0 Views 208KB Size
Florida State College at Jacksonville Detailed Assessment Report 2011-2012 Insurance Claims Adjuster (PSAV) (POS 5728) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the Insurance Claims Adjuster Program is to assist students in their quest for an Adjuster License by providing qualifying pre-licensing education to potential Adjusters. This program is based on State-mandated curriculum and class hours and will earn the student an Accredited Claims Adjuster (ACA) designation and exemption from the state examination. By offering in class and online as well as in Spanish, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Property Insurance Students will demonstrate an understanding of property insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 1: Unit 3 Quiz Students will be given a quiz on Units 3 to determine their mastery of the information presented on Property Insurance. At the completion of the class, ordered software will grade and analyze a minimum of 30% of chapter 3 quizzes for the Fall Term classes. This chapter quiz evaluates the student's learning outcomes on Property Insurance. The software will provide the department with a report of the quiz results on all the components of Property Insurance. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on Unit 3 quiz. Finding (2011-2012) - Target: Met 84 out of 121 Unit 3 quizzes (69% of total) were assessed using the Remark software. 70 were online and 14 were in a classroom setting. According to the software reports, 93% of combined online and in class scored 70% or greater on questions directly related to Property Insurance. (99% online and 64% in class.) These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this. We found that the quizzes were not administered consistently in each class and we had very few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams. Related Action Plans (by Established cycle, then alpha): Train adjunct Instructors, Change SLO for next cycle. Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 3 Quiz | Outcome/Objective: Property Insurance Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Projected Completion Date: 07/2012 Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager to communicate with the adjunct instructors.

M 2: Final Exam Students will answer specific questions related to the topic of Property Insurance on the final exam. At the completion of the Fall Term classes, purchased software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Property Insurance. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of the final exam scores will result in a 70% or higher on the questions related to Property Insurance. Standardized test of subject matter knowledge questions #13-19 (online) and questions 9, 34-38, 42-44, 46, 51, 54-55, 57, 60, 64-65, 72, 84, 94-99 (inclass). Finding (2011-2012) - Target: Met 92 out of 121 final exams (76% of total) were assessed using the Remark software. 66 were online and 26 were in a classroom setting. According to the software reports, 90% of combined online and in class scored 70% or greater on questions directly related to Property Insurance - 91% online and 88% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). The reports identified 2 questions with very low pass rates. Question #18 online (30% correctly answered) and Question 72 In class (only 19% correctly answered). While our target was met, the questions will be reviewed to determine if any changes need to be made to the exam or the course material. Related Action Plans (by Established cycle, then alpha): Review questions with low pass rate. Change SLO. Question #18 was reviewed by online Instructor Jennifer Ayres. Question 72 was reviewed by inclass Instructor, John Ingersoll. Established in Cycle: 2011-2012 Implementation Status: Finished Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Property Insurance Implementation Description: Jennifer Ayres reviewed online question #18 (expediting expenses) and determined it " is pulled verbatim from the notes in section four of the ACA OL class; therefore, I suggest we keep it in place". No changes made. John Ingersoll reviewed inclass question 72 and determined that "this is a basic question which is covered in the class material. No changes are needed". Change SLO for next cycle. Responsible Person/Group: Jennifer Ayres and John Ingersoll SLO 2: Workers Compensation Insurance Students will demonstrate an understanding of Workers Compensation Insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 3: Unit 6 Quiz Students will be given a quiz on Unit 6 to determine their mastery of the information presented on Workers Compensation Insurance. At the completion of the class, ordered software will grade and analyze a minimum of 30% of chapter 6 quizzes for the Fall Term classes. This chapter quiz evaluates the student's learning outcomes on Workers Compensation Insurance. The software will provide the department with a report of the quiz results on all the components of Workers Compensation Insurance. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of student's scores will result in a 70% or higher on Unit 6 quiz. Finding (2011-2012) - Target: Met 83 out of 121 Unit 6 quizzes (69% of total) were assessed using the Remark software. 69 were online and 14 were in a classroom setting. According to the software reports, 89% of combined online and in class scored 70% or greater on questions directly related to Workers Compensation Insurance - 99% online and 43% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this. We found that the quizzes were not administered consistently in each class and we had very few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams.

Related Action Plans (by Established cycle, then alpha): Train adjunct instructors. Change SLO for next cycle. Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 6 Quiz | Outcome/Objective: Workers Compensation Insurance Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Projected Completion Date: 07/2012 Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager, to communicate with the adjunct instructors. M 4: Final Exam Students will answer specific questions related to the topic of Workers Compensation Insurance on the final exam. At the completion of the Fall Term classes, purchased software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Workers Compensation Insurance. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of student's final exam scores will result in a 70% or higher on the questions related to Workers Compensation Insurance. Standardized test of subject matter knowledge question 29 (online) and questions 39, 41, 45, 61 and 62 (inclass). Finding (2011-2012) - Target: Met 92 out of 121 final exams (76% of total) were assessed using the Remark software. 66 were online and 26 were in a classroom setting. According to the software reports, 75% of combined online and in class scored 70% or greater on questions directly related to Workers Compensation Insurance - 65% online and 100% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). The lower online results could be attributed to the fact that the online final exam has only 1 question related to this topic (while the inclass exam has 5 questions) which makes it a pass/fail scenario. Related Action Plans (by Established cycle, then alpha): Review questions with low pass rate. Change SLO. Online Instructor will review the online final exam to determine if any changes need to be made regarding the topic of Workers Compensation. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Workers Compensation Insurance Implementation Description: A request for a review of this exam has been sent to the online Instructor who will go through Blackboard to review the final exam and determine if changes need to be made regarding the questions related to this topic. Projected Completion Date: 07/2012 Responsible Person/Group: Jennifer Ayres, Adjunct Instructor SLO 3: General Liability Students will demonstrate an understanding of General Liability. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 5: Unit 5 Quiz (Not assessed this cycle) Students will be given a quiz on Unit 5 to determine their mastery of the information presented regarding general liability. At the completion of the class, ordered software will grade and analyze a minimum of 30% of chapter 5 quizzes for the Spring Term classes. This chapter quiz evaluates the student's learning outcomes on General Liability. The software will provide the department with a report of the quiz results on all the components of General Liability. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to

determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on Unit 5 quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not assessed this cycle. SLO 4: Claims Adjusting Students will demonstrate an understanding of Claims Adjusting. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 7: Unit 12 Quiz (not assessed this cycle) Students will be given a quiz on Unit 12 to determine their mastery of the information presented on Claims Adjusting. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on Unit 12 quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not assessed this cycle.

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings? This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Financial Services. The specific course and the original final exam are reviewed and approved by the same State agency. Online courses have a different approval process and presentation can be quite different. In this course, even the books are different. Revisions can be made without a re-approval process. In addition, long term adjunct faculty (many with 20+ years' experience) who each teach one day of the four day classes, were resistive to adding end of day quizzes. In view of the excellent pre-exam review, the overall pass rate (90%) and the limited time to teach (hours are State mandated), they felt the quizzes were an imposition. These have been continuing education for those adjuncts. To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for either their second attempt at the final exam or in preparing for the State exam. 2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: · If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 2 questions (one online and one in the class) with very low pass rates. A review of both questions was conducted by our Faculty and it was determined that no changes are necessary.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Both in class and online showed differing findings for the same class/program. Surprisingly, the in class showed lower success more often. In class quizzes are given at the end of a 10 hour day (four ten hours days in a row) and prior to the students having a chance to solidify their understanding. For in class students, this takes place at night between classes and during the comprehensive class exam review. For marketing considerations (competition) it's not feasible to offer a different in class schedule. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

3. (ACAD) How have results been disseminated and discussed with advisory committee? Were all curriculum changes discussed with the Program Advisory Committee before submission to the Curriculum Committee?

Results discussed and disseminated with Advisory Committee on 5/15/2012. Their input will be used for future assesments. Curriculum is dictated by governing state agency.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs) along with input and approval by the Advisory Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program? Only two methods of delivery and sampling done on both.

Detailed Assessment Report 2011-2012 Insurance Customer Service Representative (PSAV) (POS 5727) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the Insurance Customer Representative Program is to assist students in their quest for a Resident Customer Representative License by providing qualifying pre-licensing education to potential Insurance Agents. This program is based on State-mandated curriculum and class hours and will earn the student a Registered Customer Service Representative (RCSR) designation and exemption from the state examination. By offering in class and online as well as in Spanish, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Property Insurance (Assess 2011-2012) Students will demonstrate an understanding of property insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 5: Unit 3 quiz Students will be given a quiz at the end of Unit Three to determine their mastery of the information presented on Property Insurance. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Property Insurance. The software will provide the department with a report of the quiz results on all the components of Property Insurance. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements.

Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Unit 3 quiz. Finding (2011-2012) - Target: Met 183 out of 256 Unit 3 quizzes (71% of total) were assessed using the Remark software. 164 were online and 19 were in a classroom setting. According to the software reports, 95% of combined online and in class scored 70% or greater on the Property Insurance unit quiz - 100% online and 53% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this. We found that the quizzes were not administered consistently in each class and we had very few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams. Related Action Plans (by Established cycle, then alpha): Train adjunct instructors. Change SLO for next cycle.

Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 3 quiz | Outcome/Objective: Property Insurance (Assess 2011-2012) Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Projected Completion Date: 07/2012 Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager, to communicate with the adjunct instructors. M 6: Final Exam Students will answer specific questions related to the topic of Property Insurance on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Property Insurance. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the questions related to Property Insurance. Standardized test of subject matter knowledge questions #12-18. Finding (2011-2012) - Target: Met 239 out of 256 final exams (95% of total) were assessed using the Remark software. 187 were online and 52 were in a classroom setting. According to the software reports, 92% of combined online and in class scored 70% or greater on questions directly related to Property Insurance - 94% online and 85% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). 2 of the online questions and 2 of the in class questions related to this topic had low pass rates. Online questions #14 (76%) and #17 (73%) and inclass questions #57 (69%) and #73 (65%). The instructors will review the exams and class material to determine if any changes are needed. Related Action Plans (by Established cycle, then alpha): Review questions with low pass rate. Change SLO. Online questions #14 (76%) and #17 (73%) and inclass questions #57 (69%) and #73 (65%). The instructors will review the exams and class material to determine if any changes are needed.

Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Property Insurance (Assess 2011-2012) Implementation Description: The online and inclass instructors will review the questions shown above and determine if changes need to be made to the course material or the exams. The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 05/2012 Responsible Person/Group: Jennifer Ayres, online Instructor; John Ingersoll, inclass instructor; Effectiveness Process Facilitator (Kathy Gladu) SLO 2: Workers Compensation insurance (Assess 2011-2012) Students will demonstrate an understanding of Workers Compensation insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 7: Unit 6 Quiz Students will be given a quiz at the end of Unit 6 to determine their mastery of the information presented on Workers Compensation Insurance.

Students will be given a quiz at the end of Unit Six to determine their mastery of the information presented on Workers Compensation. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Workers Compensation. The software will provide the department with a report of the quiz results on all the components of Workers Compensation. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Unit 6 quiz. Finding (2011-2012) - Target: Met 198 out of 256 Unit 6 quizzes (77% of total) were assessed using the Remark software. 163 were online and 35 were in a classroom setting. According to the software reports, 93% of combined online and in class scored 70% or greater on questions directly related to Workers Compensation Insurance - 100% online and 63% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this. We found that the quizzes were not administered consistently in each class and we had very few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams. Related Action Plans (by Established cycle, then alpha): Train adjunct instructors. Change SLO for next cycle. Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 6 Quiz | Outcome/Objective: Workers Compensation insurance (Assess 2011-2012) Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager, to communicate with the adjunct instructors. M 8: Final Exam Students will answer specific questions related to the topic of Workers Compensation on the final exam. At the completion of the Fall Term class, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Workers Compensation. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the questions related to Workers Compensation. Standardized test of subject matter knowledge question #29. . Finding (2011-2012) - Target: Met 239 out of 256 final exams (95% of total) were assessed using the Remark software. 187 were online and 52 were in a classroom setting. According to the software reports, 100% of combined online and in class scored 100% on questions directly related to Property Insurance - 100% online and 100% in class. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified either online or inclass on any of the questions related to this topic. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective):

Measure: Final Exam | Outcome/Objective: Workers Compensation insurance (Assess 2011-2012) Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 05/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) SLO 3: General Liability Students will demonstrate an understanding of general liability. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 9: Unit 5 quiz (Not assessed this cycle. ) Students will be given a quiz at the end of Unit Five to determine their mastery of the information presented on General Liability. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on General Liability. The software will provide the department with a report of the quiz results on all the components of General Liability. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Unit 5 quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not assessed this cycle. SLO 4: Crime Students will demonstrate an understanding of Crime. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 10: Units 7 quiz (Not assessed this cycle. ) Students will be given a quiz at the end of Unit Seven to determine their mastery of the information presented on Crime. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Crime. The software will provide the department with a report of the quiz results on all the components of Crime. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Units 7, 8 and 9 quizzes. Finding (2011-2012) - Target: Not Reported This Cycle Not assessed this cycle. SLO 5: Property and Liability concepts Students will demonstrate an understanding of property and liability insurance concepts. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 11: Unit 1 quiz (not assessed this cycle) Students will be given a quiz at the end of Unit One to determine their mastery of the information presented on

Property and Liability Insurance Concepts. At the completion of the class, software will grade and analyze a minimum of 30% of Unit One quizzes. This quiz evaluates the student's learning outcomes on Property and Liability Insurance Concepts. The software will provide the department with a report of the quiz results on all the components of Property and Liability Insurance Concepts. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Unit 12 quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not assessed this cycle.

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings? This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Financial Services. The specific course and the original final exam are reviewed and approved by the same State agency. Online courses have a different approval process and presentation can be quite different. In this course, even the books are different. Revisions can be made without a re-approval process. In addition, long term adjunct faculty (many with 20+ years' experience) who each teach one day of the four day classes, were resistive to adding end of day quizzes. In view of the excellent pre-exam review, the overall pass rate (90%) and the limited time to teach (hours are State mandated), they felt the quizzes were an imposition. These have been continuing education for those adjuncts. To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for either their second attempt at the final exam or in preparing for the State exam. 2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: · If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 4 questions (two online and two in the class) with lower pass rates. A review of the questions was conducted by our Faculty and it was determined that one of the questions #14 online required updating and the instructor made the necessary changes. No changes were deemed necessary for the other 3 questions.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Both in class and online showed differing findings for the same class/program. Surprisingly, the in class showed lower success more often. In class quizzes are given at the end of a 10 hour day (four ten hours days in a row) and prior to the students having a chance to solidify their understanding. For in class students, this takes place at night between classes and during the comprehensive class exam review. For marketing considerations (competition) it's not feasible to offer a different in class schedule. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

3. (ACAD) How have results been disseminated and discussed with advisory committee? Were all curriculum changes discussed with the Program Advisory Committee before submission to the Curriculum Committee? Results discussed and disseminated with Advisory Committee on 5/15/2012. Their input will be used for future assesments. Curriculum is dictated by governing state agency.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs) along with input and approval by the Advisory

Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program? Only two methods of delivery and sampling done on both.

Detailed Assessment Report 2011-2012 Insurance General Lines Agent (PSAV) (POS 5723) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the General Lines Agent Program is to assist students in their quest for an Insurance Sales License by providing the required qualifying pre-licensing education to potential Insurance agents. This program is based on State-mandated curriculum and class hours which prepare students for the State Licensing Exam. By offering online, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Property Insurance Students will demonstrate an understanding of property Insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 1: Unit 6 quiz Students will be given a quiz at the end of Unit Six to determine their mastery of the information presented on Property Insurance. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Property Insurance. The software will provide the department with a report of the quiz results on all the components of Property Insurance. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain a 70% or higher on the Unit 6 quiz to determine their mastery of Property Insurance. Finding (2011-2012) - Target: Met 31 out of 39 Unit 6 quizzes (79% of total) were assessed using the Remark software. All 31 were online as this Program is only delivered online. According to the software reports, 97% scored 90% or greater on questions directly related to Property Insurance. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for this online quiz. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 6 quiz | Outcome/Objective: Property Insurance Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 2: Final Exam Students will answer specific questions related to the topic of Property Insurance on the final exam. At the

Students will answer specific questions related to the topic of Property Insurance on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Property Insurance. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Standardized test of subject matter knowledge Target: 70% of students will achieve a 70% or higher on the questions related to Property Insurance. Standardized test of subject matter knowledge questions #23-34. Finding (2011-2012) - Target: Met 23 out of 39 final exams (62% of total)were assessed using the Remark software. All 23 were online as this Program is only delivered online. According to the software reports, 87% scored 75% or greater on the questions directly related to Property Insurance. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). 3 questions (#23, 29 and 32) related to this topic were identified as having correct responses of less than 80%. It was determined that each of the questions the "Material is verbatim within notes" and no updates or changes to the questions or material are needed. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Property Insurance Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) SLO 2: Workers Compensation Students will demonstrate an understanding of Workers Compensation. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 3: Unit 9 quiz Students will be given a quiz at the end of Unit Nine to determine their mastery of the information presented on Workers Compensation. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Workers Compensation. The software will provide the department with a report of the quiz results on all the components of Workers Compensation. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of quiz scores will be 70% or higher on the Unit Nine Quiz Finding (2011-2012) - Target: Met 28 out of 39 Unit 9 quizzes (72% of total) were assessed using the Remark software. All 28 were online as this Program is only delivered online. According to the software reports, 100% scored 90% or greater on questions directly related to Workers Compensation. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No trends or weakesses in the quiz were identified. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective):

Measure: Unit 9 quiz | Outcome/Objective: Workers Compensation Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 4: Final Exam Students will answer specific questions related to the topic of Workers Compensation on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Workers Compensation. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students exams will achieve a 70% or higher on the questions related to Workers Compensation. Standardized test of subject matter knowledge questions #59-65. Finding (2011-2012) - Target: Met 23 out of 39 final exams (59% of total) were assessed using the Remark software. All 23 were online as this Program is only delivered online. According to the software reports, 96% scored 71% or greater on questions directly related to Workers Compensation. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). Question #65 related to this topic had a correct response percentage lower than 80%. After further review, it was determined that "material is verbatim within notes" and no changes or updates are needed. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Workers Compensation Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) SLO 3: General Liability Students will demonstrate an understanding of General Liability. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 5: Unit 7 quiz (not assessed this cycle.) Students will be given a quiz at the end of Unit Seven to determine their mastery of the information presented on General Liability. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on General Liability. The software will provide the department with a report of the quiz results on all the components of General Liability. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of quiz scores will be 70% or higher on the Unit Seven Quiz Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. M 6: Final Exam(not assessed this cycle.)

Students will answer specific questions related to the topic of General Liability on the final exam. At the completion of the class, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of General Liability. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Standardized test of subject matter knowledge Target: 70% of students will achieve a 70% or higher on the questions related to General Liability. Standardized test of subject matter knowledge questions #35-46. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. SLO 4: Crime Students will demonstrate an understanding of Crime insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 7: Unit 10 quiz (not assessed this cycle.) Students will be given a quiz at the end of Unit Ten to determine their mastery of the information presented on Crime. At the completion of the class, software will grade and analyze a minimum of 30% of these chapter quizzes. This quiz evaluates the student's learning outcomes on Crime. The software will provide the department with a report of the quiz results on all the components of Crime. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the Unit Ten quiz.

Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. M 8: Final Exam (not assessed this cycle) Students will answer specific questions related to the topic of Crime on the final exam. At the completion of the class, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Crime. The software will provide the department with a detailed report of the combined results. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the questions related to Crime. Standardized test of subject matter knowledge questions #66-69. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle.

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings? This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Financial Services. The specific course and the original final exam are reviewed and approved by the same State agency. Revisions can be made without a re-approval process. Our approach has been to do specific item analysis on selected topics required and tested for by the State. This topic item analysis tells us the student degree of preparation for that topic for the state exam.

To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for the State exam. 2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: · If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 4 questions with pass rates lower than 80%. A review of all 4 questions was conducted by our Faculty and it was determined that no changes are necessary.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Only online delivery is available for this course. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

3. (ACAD) How have results been disseminated and discussed with advisory committee? Were all curriculum changes discussed with the Program Advisory Committee before submission to the Curriculum Committee? Results discussed and disseminated with Advisory Committee on 5/15/2012. Their input will be used for future assesments. Curriculum is dictated by governing state agency.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs) along with input and approval by the Advisory Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program? Only the one delivery system is available.

Detailed Assessment Report 2011-2012 Life Insurance Marketing (PSAV) (POS 5726) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the Life Insurance Marketing Program is to assist students in their quest for an Insurance Sales License by providing the required qualifying pre-licensing education to potential Insurance agents. This program is based on Statemandated curriculum and class hours which prepare students for the State Licensing Exam. By offering online, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Principles and uses Students will demonstrate an understanding of the principles and uses of life and health insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 1: Chapters 1,2,4 and 14 quizzes Students will be given a quiz at the end of chapters 1, 2, 4 and 14 to determine their mastery of the information presented on the principles and uses of life and health insurance.

At the completion of the class, ordered software will grade and analyze a minimum of 30% of chapter quizzes for the Fall Term classes. These chapter quizzes evaluate the student's learning outcomes on the principles and uses of life and health insurance. The software will provide the department with a report of the quiz results on all the components of the principles and uses of life and health insurance. The Program Manager, office staff, and a faculty member will review the reports at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain 70% or higher on the Chapter quizzes. Finding (2011-2012) - Target: Met 75 out of 178 of chapter quizzes 1, 2, 4 and 14 (42% of total)were assessed using the Remark software. 59 were online and 16 inclass. According to the software reports, 84% of combined online and inclass quizzes scored 70% or greater on the questions directly related to Principles and Uses of Life and Health Insurance. 100% for online and 40% inclass. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this. We found that the quizzes were not administered consistently in each class and we had very few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams. Related Action Plans (by Established cycle, then alpha): Train adjunct instructors. Change SLO for next cycle. Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective): Measure: Chapter 3 quiz | Outcome/Objective: Legal Concepts of the Insurance Contract Measure: Chapters 1,2,4 and 14 quizzes | Outcome/Objective: Principles and uses Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Projected Completion Date: 07/2012 Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager, to communicate with the adjunct instructors. M 2: Final Exam Students will answer specific questions related to the topic of Principles and Uses of Life and Health Insurance on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Principles and Uses of Life and Health Insurance. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of Final Exam scores will be 70% or higher on the questions related to Principles and Uses of Life and Health Insurance. Standardized test of subject matter knowledge questions #4, 12, 14, 18, 27, 29, 36,37,38,40. Finding (2011-2012) - Target: Met 130 out of 178 final exams (73% of total) were assessed using the Remark software. 52 were online and 78 in class. According to the software reports, 96% of combined online and inclass exams scored 70% or greater on the questions directly related to Principles and Uses of Life and Health Insurance. 94% for online and 97% in class. These results were reviewed by the Program Manager (Bill Barfield) and the Effectiveness Process Facilitator (Kathy Gladu). No weaknesses or trends were identified. Online final exam questions are pulled randomly from a Pool in Respondus which makes assessing these results extremely time consuming. It is suggested that the questions be changed to a static set rather than random. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle.

Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Principles and uses Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 05/2012 SLO 2: Legal Concepts of the Insurance Contract Students will demonstrate and understanding of the Legal Concepts of the Insurance Contract. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 3: Chapter 3 quiz Students will be given a quiz at the end of Chapter 3 to determine their mastery of the information presented on Legal Concepts of the Insurance Contract. At the completion of class, software will grade and analyze a minimum of 30% of chapter quizzes for the Fall Term classes. The chapter 3 quiz evaluates the student's learning outcomes on the Legal Concepts of the Insurance contract. The software will provide the department with a report of the quiz results on all the components of this topic. The Program Manager, office staff, and a faculty member will review the reports at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain 70% or higher on Chapter 3 quiz. Finding (2011-2012) - Target: Met 95 out of 178 of chapter 3 quizzes (53% of total) were assessed using the Remark software. 54 were online and 41 inclass. According to the software reports, 83% of combined online and inclass quizzes scored 70% or greater on the questions directly related to Legal Concepts of the Insurance Contract. 100% for online and 61% inclass. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for the online quizzes. However, it was determined that the gap between the online and inclass results could be attributed to several factors: 1) Administering graded quizzes is a new process in this 40 hour class. 2) The class is taught by 4 different longtime Adjunct instructors which vary class by class. 3) The graded quizzes do not impact the student's final grades and the students are aware of this as evidenced by remarks made on the student's Evaluations of the class. We found that the quizzes were not administered consistently in each class and we had few to assess. In addition, these quiz results do not appear to reflect the mastery of the topic as evidenced by the high results identified in the topic question scores on the Final Exams. Related Action Plans (by Established cycle, then alpha): Train adjunct instructors. Change SLO for next cycle. Continue to train and encourage instructors on the implementation of quizzes in the classroom setting. Since the target was met for this Measure, the SLO will be changed for the next cycle. Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective): Measure: Chapter 3 quiz | Outcome/Objective: Legal Concepts of the Insurance Contract Measure: Chapters 1,2,4 and 14 quizzes | Outcome/Objective: Principles and uses Implementation Description: Rewrite Instructor Instructions to include the administration of quizzes in the classroom. Program Manager will communicate with each Instructor to ensure they understand their responsibility. Projected Completion Date: 07/2012 Responsible Person/Group: Office staff to rewrite the Instructor instructions. Bill Barfield, Program Manager, to communicate with the adjunct instructors. M 4: Final Exam Students will answer specific questions related to the topic of Legal Concepts of the Insurance Contract on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Legal Concepts of the Insurance Contract. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements.

Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of Final Exam scores will be 70% or higher on the questions related to Legal Concepts of the Insurance Contract. Standardized test of subject matter knowledge questions #1, 6, 8, 13, 17. Finding (2011-2012) - Target: Met 130 out of 178 final exams (73% of total) were assessed using the Remark software. 52 were online and 78 in class. According to the software reports, 86% of combined online and inclass exams scored 70% or greater on the questions directly related to Legal Concepts of the Insurance Contract. 96% for online and 79% in class. These results were reviewed by the Program Manager (Bill Barfield) and the Effectiveness Process Facilitator (Kathy Gladu). Online question #4 resulted in a 57% pass rate and the In Class question # 8 resulted in a 68% pass rate. These questions were reviewed by the Program Manger and he determined: "4 uses correct answer but term is historic and not commonly used in everyday conversation. However it is a State tested principle of insurance. 8 uses a term (group) that typically isn't used for individual insurance risks. However it is a State tested description of an insurance type. We'll request more emphasis in class on these subjects." Online final exam questions are pulled randomly from a Pool in Respondus which makes assessing these results extremely time consuming. It is suggested that the questions be changed to a static set rather than random. Related Action Plans (by Established cycle, then alpha): Request more emphasis in class on subjects with low pass rate. Change SLO. Question 4 on the inclass final exam uses correct answer but term is historic and not commonly used in everyday conversation. However it is a State tested principle of insurance. Question 8 on the inclass final exam uses a term (group) that typically isn't used for individual insurance risks. However it is a State tested description of an insurance type We'll request more emphasis in class on these subjects. Established in Cycle: 2011-2012 Implementation Status: Planned Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Legal Concepts of the Insurance Contract Implementation Description: A request will be made by the Program Manager to the responsible Instructors to emphasize the material related to Questions 4 and 8 on the inclass final exam. This should be done prior to the next inclass session. Projected Completion Date: 05/2012 Responsible Person/Group: Program Manager, Bill Barfield. SLO 3: Life Insurance Policies Students will demonstrate an understanding of Life Insurance Polices - Provisions, Options and Riders. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 5: Chapters 5 & 6 quizzes (not assessed this cycle.) Students will be given a quiz at the end of Chapters 5 & 6 to determine their mastery of the information presented on Life Insurance Policies. At the completion of class, software will grade and analyze a minimum of 30% of chapter quizzes for the Fall Term classes. These chapter quizzes evaluates the student's learning outcomes on Life Insurance Policies. The software will provide the department with a reort of the quiz results on all the components of this topic. The Program Manager, office staff, and a faculty member will review the reports at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements.. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain 70% or higher on the quizzes for chapters 5 and 6.. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. M 6: Final Exam (not assessed this cycle.) Students will answer specific questions related to the topic of Life Insurance Policies on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Life Insurance Policies - Provisions, Options and Riders. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be

identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of Final Exam scores will be 70% or higher on the questions related to Life Insurance Policies. Standardized test of subject matter knowledge questions 2,3,7,9,10,20,23,24,31 and 35. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cylce.

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings? This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Financial Services. The specific course and the original final exam are reviewed and approved by the same State agency. Online courses have a different approval process and presentation can be quite different. Revisions can be made without a re-approval process. Our approach has been to do specific item analysis on selected topics required and tested for by the State. This topic item analysis tells us the student degree of preparation for that topic for the state exam. In addition, long term adjunct faculty (many with 20+ years' experience) who each teach one day of the four day classes, were resistive to adding end of day quizzes. In view of the excellent pre-exam review, the overall pass rate (90%) and the limited time to teach (hours are State mandated), they felt the quizzes were an imposition. These have been continuing education for those adjuncts. To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for either their second attempt at the final exam or in preparing for the State exam. 2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 2 questions (one online and one in the class) with low pass rates. A review of both questions was conducted by our Program Manager and it was determined that no change is necessary for the OL question. However, the Program Manager will request that our Faculty put more emphasis on the concept in class that relates to question number 8 on the final exam.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Both in class and online showed differing findings for the same class/program. Surprisingly, the in class showed lower success more often. In class quizzes are given at the end of a 10 hour day (four ten hours days in a row) and prior to the students having a chance to solidify their understanding. For in class students, this takes place at night between classes and during the comprehensive class exam review. For marketing considerations (competition) it's not feasible to offer a different in class schedule. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

3. (ACAD) How have results been disseminated and discussed with advisory committee? Were all curriculum changes discussed with the Program Advisory Committee before submission to the Curriculum Committee? Results discussed and disseminated with Advisory Committee on 5/15/2012. Their input will be used for future assesments. Curriculum is dictated by governing state agency.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs) along with input and approval by the Advisory Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program?

Only two methods of delivery and sampling done on both.

Detailed Assessment Report 2011-2012 Personal Lines Insurance (PSAV) (POS 5737) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the Personal Lines Insurance Program is to assist students in their quest for an Insurance Sales License by providing the required qualifying pre-licensing education to potential Insurance agents. This program is based on State-mandated curriculum and class hours which prepare students for the State Licensing Exam. By offering online, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Automobile insurance Students will demonstrate an understanding of automobile insurance. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 1: Unit 3 quiz Students will be given an online quiz at the end of Unit Three to determine their mastery of the information presented on Personal Automobile Insurance and Misc. Automobile Rules. At the completion of the Fall Term classes, ordered software will grade and analyze a minimum of 30% of Unit Three quizzes. This quiz evaluates the student's learning outcomes on Automobile Insurance and Misc. Automobile Rules. The software will provide the department with a report of the quiz results on all the components of Personal Automobile Insurance and Misc. Automobile Rules. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements.

Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will obtain 70% or higher on the Unit 3 quiz. Finding (2011-2012) - Target: Met 22 out of 29 Unit 3 quizzes (76% of total)were assessed using the Remark software. All 22 were online as this Program is only delivered online. According to the software reports, 100% scored 90% or greater on questions directly related to Automobile Insurance and Miscellaneous Automobile Rules. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends were identified for this online quiz. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 3 quiz | Outcome/Objective: Automobile insurance Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 2: Final Exam Students will answer specific questions related to the topic of Automobile Insurance and Misc. Automobile Rules on the final exam. At the completion of the Fall Term classes, purchased software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topics. These specific questions evaluate the student's learning outcomes related to all components of Automobile Insurance

and Misc. Automobile Rules. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the questions related to Automobile Insurance and Misc. Automobile Rules. Standardized test of subject matter knowledge questions: Automobile Insurance: #9-29 Misc. Automobile Rules: #30-41 Finding (2011-2012) - Target: Met 21 out of 29 final exams (72% of total) were assessed using the Remark software. All 21 were online as this Program is only delivered online. According to the software reports, 100% scored 71% or greater on questions directly related to Automobile Insurance and 95% scored 75% or greater on questions related to Miscellaneous Automobile Rules. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres) and identified three questions that required more indepth analysis due to a lower pass rate. Jennifer reviewed further and determined that: Question #27 requires updating. Question #28 "Material is verbatim in notes" and no changes are needed. Question #37 "This is a "thinking question", but the notes and textbook provide the material necessary to answer. Related Action Plans (by Established cycle, then alpha): Update final exam. Change SLO. Question #27 on the online final exam will be updated by the online instructor. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Automobile insurance Implementation Description: Instructor will update question 27 on the final exam through Blackboard in all classes. The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Responsible Person/Group: Jennifer Ayres will update the final exam. Kathy Gladu will add the new SLO to Weave and update the Remark material. SLO 2: Property Insurance, Homeowner's Students will demonstrate an understanding of Property Insurance and Homeowner's. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 3: Unit 4 quiz Students will be given an online quiz at the end of Unit Four to determine their mastery of the information presented on Property Insurance and Homeowners. At the completion of the Fall Term classes, ordered software will grade and analyze a minimum of 30% of Unit Four quizzes. This quiz evaluates the student's learning outcomes on Property Insurance and Homeowners. The software will provide the department with a report of the quiz results on all the components of Property Insurance and Homeowners. The Program Manager, office staff, and a faculty member will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of quiz scores will be 70% or higher on the Unit 4 quiz. Finding (2011-2012) - Target: Met 22 out of 29 Unit 4 quizzes (76% of total) were assessed using the Remark software. All 22 were online as this Program is only delivered online. According to the software reports, 100% scored 90% or greater on questions directly related to Property Insurance, Homeowners. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres). No weaknesses or trends related to the questions on this quiz were identified. Related Action Plans (by Established cycle, then alpha):

Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Unit 4 quiz | Outcome/Objective: Property Insurance, Homeowner's Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 4: Final Exam Students will answer specific questions related to the topic of Property Insurance and Homeowners on the final exam. At the completion of the Fall Term classes, purchased software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Property Insurance and Homeowners. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 70% of students will achieve a 70% or higher on the final exam questions related to Property Insurance and Homeowners. Standardized test of subject matter knowledge questions:

Property Insurance: #42-50 Homeowners: #51-63 Finding (2011-2012) - Target: Met 21 out of 29 final exams (72% of total) were assessed using the Remark software. All 21 were online as this Program is only delivered online. According to the software reports, 95% scored 78% or greater on questions directly related to Property Insurance and 86% scored 77% or greater on questions directly related to Homeowners. These results were reviewed by the Program Manager (Bill Barfield), Effectiveness Process Facilitator (Kathy Gladu) and Faculty (Jennifer Ayres) and 3 questions were identified that required more indepth analysis due to a lower pass rate. Jennifer reviewed them further and determined: Question #51 "Material is verbatim in notes" and no changes are needed. Question #54 "This is a "thinking question", but the notes and textbook provide the material necessary to answer. Question #59 "This is a "thinking question", but the notes and textbook provide the material necessary to answer. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Property Insurance, Homeowner's Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu)

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings?

This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Financial Services. The specific course and the original final exam are reviewed and approved by the same State agency. Revisions can be made without a re-approval process. Our approach has been to do specific item analysis on selected topics required and tested for by the State. This topic item analysis tells us the student degree of preparation for that topic for the state exam. To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for either their second attempt at the final exam or in preparing for the State exam. 2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 6 questions with very low pass rates. A review of the questions was conducted by our Faculty and it was determined that 1 question required updating which was completed by the Faculty. No changes are necessary for the other 5 questions.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Only online delivery is available for this course. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

3. (ACAD) How have results been disseminated and discussed with advisory committee? Were all curriculum changes discussed with the Program Advisory Committee before submission to the Curriculum Committee? Results discussed and disseminated with Advisory Committee on 5/15/2012. Their input will be used for future assesments. Curriculum is dictated by governing state agency.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs along with input and approval by the Advisory Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program? Only the one delivery system is available.

Detailed Assessment Report 2011-2012 Real Estate Sales Agent (PSAV) (POS 5733) (DE) As of: 2/07/2013 03:22 PM EST

(Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.)

Mission / Purpose The mission of the Real Estate Sales Agent Program is to assist students in their quest for a Real Estate Sales License by providing the required qualifying pre-licensing education to potential Real Estate Sales agents. This program is based on State-mandated curriculum and class hours which prepare students for the State Licensing Exam. By offering in class and online, it optimizes educational opportunities for both local and state-wide students.

Student Learning Outcomes/Objectives, along with Any Associations and Related Measures, Targets, Findings, and Action Plans SLO 1: Real Estate Financing Students will demonstrate an understanding of basic real estate financing. Will assess 2011-2012 cycle. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations

1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 1: Chapter 12 quiz Students will be given a quiz at the end of chapter 12 which contains specific questions to evaluate their mastery of the information presented on basic Real Estate financing. At the completion of the class, software will grade and analyze a minimum of 30% of the chapter 12 quizzes for the Fall Term classes. The chapter 12 quiz evaluates the student's learning outcomes on the components of basic real estate financing. The software will provide the department with a report of the quiz results of all the components of real estate financing. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will obtain a 90% or higher on the chapter 12 quiz. Finding (2011-2012) - Target: Met 20 out of 20 final exams (100% of total) were assessed using the Remark software. 20 were online and 0 were in class (not enough students to run the Fall Term lecture class). According to the software reports, 75% scored 90% or greater on questions directly related to Real Estate Financing. These results were reviewed by the Program Manager (Bill Barfield), and Effectiveness Process Facilitator (Kathy Gladu). No weaknesses or trends were identified in this quiz. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Chapter 12 quiz | Outcome/Objective: Real Estate Financing Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 2: Final Exam Students will answer specific questions related to basic real estate financing on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the Final Exam related to the topic. These specific questions evaluate the student's learning outcomes related to all the components of Real Estate financing. The software will provide the department with a detailed report of the final exam results. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Standardized test of subject matter knowledge Target: 50% of students will obtain a 70% or higher on the questions related to this topic on the final exam. Standardized test of subject matter knowledge questions 26, 28, 33, 36, 71, 78 and 93. Finding (2011-2012) - Target: Met 16 out of 18 final exams (89% of total)were assessed using the Remark software. 16 were online and 0 were in class (not enough students to run the Fall Term lecture class). According to the software reports, 62% scored 71% or greater on questions directly related to Real Estate Financing. These results were reviewed by the Program Manager (Bill Barfield) and the Effectiveness Process Facilitator (Kathy Gladu) .An item analysis identified 3 questions having a low pass rate. Question 38 - 63% correct relates to LTV ratio; Question 59 50% correct relates to terminology; Question 92 - 63% correct relates to "HER" calculations. These findings were discussed with the Program Manager and it was determined that all the information necessary to answer these questions are contained in the textbook. These are typically difficult questions in this course since they include calculations and no changes will be made to the Exam. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Real Estate Financing Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be

assessed in the new cycle. Projected Completion Date: 07/2012 Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) SLO 2: Titles, Deeds and Ownership Restrictions Students will demonstrate an understanding of Titles, Deeds and Ownership Restrictions. Will assess 2011-2012 cycle. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 3: Chapter 9 quiz Students will be given a quiz at the end of chapter 9 to evaluate their mastery of the information presented regarding Titles, Deeds and Ownership Restrictions. At the completion of the class, software will grade and analyze a minimum of 30% of the chapter 9 quizzes for the Fall Term classes. These chapter quizzes evaluate the student's learning outcomes of Titles, Deeds and Ownership Restrictions. The software will provide the department with a report of the quiz results of all the components of these topics. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will obtain a 90% or higher on the chapter quizzes. Finding (2011-2012) - Target: Met 20 out of 20 chapter 9 quizzes (100% of total) were assessed using the Remark software. 20 were online and 0 in class (not enough students enrolled in Fall Term classes to run). According to the software reports, 80% of online classes scored 90% or greater on the questions directly related to Titles, Deeds and Ownership Restrictions. These results were reviewed by the Program Manager (Bill Barfield) and Effectiveness Process Facilitator (Kathy Gladu). No weaknesses or trends were identified on this quiz. Related Action Plans (by Established cycle, then alpha): Change SLO. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Chapter 9 quiz | Outcome/Objective: Titles, Deeds and Ownership Restrictions Implementation Description: The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Responsible Person/Group: Effectiveness Process Facilitator (Kathy Gladu) M 4: Final Exam Students will answer specific questions related to the topic of Titles, Deeds and Ownership Restrictions on the final exam. At the completion of the Fall Term classes, purchased software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Titles, Deeds and Ownership Restrictions. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will achieve 70% or higher on questions related to Titles, Deeds and Ownership Restrictions. Standardized test of subject matter knowledge questions 10, 25, 56, 58, 82, 90, and 92. Finding (2011-2012) - Target: Met 16 out of 16 final exams (100% of total) were assessed using the Remark software. 16 were online and 0 were in class (not enough students to run the Fall Term lecture class). According to the software reports, 81% scored 70% or greater on questions directly related to Titles, Deeds and Ownership Restrictions. These results were reviewed by the Program Manager (Bill Barfield) and the Effectiveness Process Facilitator (Kathy Gladu). An item analysis identified 2 questions having a low pass rate. Question 4 - 69% correct relates to Deeds; Question 66 - 56% correct relates to terminology (Mortgagor vs Mortgagee). These findings were discussed with the Program Manager and it was determined that Question 4's correct answer contained a typo which could have made it confusing to the student and should be corrected. Question 66 - no changes needed as these terms are defined in the class material. Related Action Plans (by Established cycle, then alpha): Update Final Exam. Change SLO.

The instructor will update the final exam. Since target has been met, we will change our focus to another SLO. Established in Cycle: 2011-2012 Implementation Status: In-Progress Priority: High Relationships (Measure | Outcome/Objective): Measure: Final Exam | Outcome/Objective: Titles, Deeds and Ownership Restrictions Implementation Description: The instructor will go into Blackboard to correct the typo found in the answer to question 4 in the final exam. The 2012/2013 cycle SLO will be changed in Weave for this Program. The Remark answer keys will be updated to identify the questions related to the topics that will be assessed in the new cycle. Projected Completion Date: 05/2012 Responsible Person/Group: Pat Watkins, Adjunct Instructor Effectiveness Process Facilitator (Kathy Gladu) SLO 3: Property rights Students will demonstrate an understanding of property rights. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 5: Chapters 8 quiz (Not assessed this cycle. Will assess Fall 2012). Students will be given a quiz at the end of chapter 8 to determine their mastery of the information presented on property rights. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will obtain a 90% or higher on the chapter quizzes. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. M 6: Final Exam Not assessed this cycle. Will assess Fall 2012). Students will answer specific questions related to the topic of Property rights on the final exam. At the completion of the Fall Term classes, software will grade and analyze a minimum of 30% of student's responses to the specific questions on the final exam related to the topic. These specific questions evaluate the student's learning outcomes related to all components of Property rights. The software will provide the department with detailed reports of the combined results. The Program Manager, office staff, and faculty will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee members will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will achieve 70% or higher on questions related to property rights. Standardized test of subject matter knowledge questions 8, 24, 45, 57, 63, 73, and 79. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. SLO 4: Computations and Title Closing Students will demonstrate an understanding of Computations and Title Closing. Not assessed this cycle. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 7: Chapters 14 quiz (Not assessed this cycle. Will assess Fall 2012). Students will be given a quiz at the end of chapter 14 to determine their mastery of the information presented on computations and title closing. At the completion of the class, software will grade and analyze a minimum of 30% of the chapter 14 quizzes for the Fall Term classes. This chapter quizz evaluates the student's learning outcomes of Computations and Title Closing. The software will provide the department with a report of the quiz results of all the components of these topics. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target:

50% of students will obtain a 90% or higher on the chapter quizzes. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. SLO 5: Value of real property Students will demonstrate an understanding on the process of estimating the value of real property. Not being assessed this cycle. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 8: Chapter 15 quiz (Not assessed this cycle. Will assess Fall 2013) Students will be given a quiz at the end of chapter 15 to determine their mastery of the information presented on estimating the value of real property. At the completion of the class, software will grade and analyze a minimum of 30% of the chapter 15 quizzes for the Fall Term classes. These chapter quizzes evaluate the student's learning outcomes of Estimating Real Property Value. The software will provide the department with a report of the quiz results of all the components of these topics. The Program Manager, office staff, and a faculty member will review the report at the end of the Term to determine if weaknesses or trends can be identified in any area of this topic. Faculty and Advisory Committee will also be consulted on possible improvements. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will obtain a 90% or higher on the chapter quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle. SLO 6: Investment analysis Students will demonstrate an understanding of investment analysis in Real Estate. Not being assessed this cycle. Relevant Associations: Standard Associations Distance Education - (DE) 1 For programs in which a student can obtain at least 50% or more of degree requirements via distance education, sampling must be representative and findings and analysis discussed. College Goals Associations 1.3 Collegewide Goal Three (as of Aug 2011): Optimize access to College programs and services. Related Measures M 9: Chapter 17 quiz (Not assessed this cycle. Will assess Fall 2013) Students will be given a quiz at the end of chapter 17 to determine their mastery of the information presented on investment analysis in Real Estate. Source of Evidence: Faculty pre-test / post-test of knowledge mastery Target: 50% of students will obtain a 90% or higher on the chapter quiz. Finding (2011-2012) - Target: Not Reported This Cycle Not reported this cycle.

Analysis Questions and Analysis Answers 1. (ALL) Describe up to three of the most significant/important improvements in your program or unit. What primary changes are you making to improve student learning (in academic programs and educational support units) or improve achievement of unit outcomes (for non-academic programs and educational support units) as a result of the findings? This program is a single course PSAV Pre-Licensing Program that has curriculum and duration dictated by the Florida Department of Business Regulation. The specific course and the original final exam are reviewed and approved by the same State agency. Online courses have a different approval process and presentation can be quite different. Revisions can be made without a re-approval process. Our approach has been to do specific item analysis on selected topics required and tested for by the State. This topic item analysis tells us the student degree of preparation for that topic for the state exam. To assist in the collection of data for assessment purposes, our department implemented the use of the Remark OMR software. This software has enabled us to make improvements to our Programs in 2 ways. 1. We are able to provide the individual student with a report that provides them with not only their overall score, but also their score for each topic tested, the units that relate to that topic and the percentage of the state exam questions on each topic as indicated in the State's published Examination Content Outline. This report can be used to assist the student in preparing for either their second attempt at the final exam or in preparing for the State exam.

2. The software produces reports that enable us to do an item analysis of the final exam questions. The analysis looks at the topic question grouping and if appropriate, the specific questions. This analysis can tell us: · If the instructional need isn't being met; If the question is poorly designed; It's lacking in both areas; or, It's an area or type that historically presents difficulty to students. For this particular cycle, our analysis identified 5 questions with low pass rates. A review of the questions was conducted by our Faculty and it was determined that 1 required a correction but the others are typically difficult questions in this course since they include calculations and no changes will be made to the Exam.

2. (ACAD and EDUC SUPPORT ONLY) How do your outcome assessment findings differ by modality (“face-toface,” hybrid, and distance education program delivery) and by location (if program is offered on more than one campus or center)? Please discuss the assessment data results and action plan college-wide and per campus, center, distance education, and military education, if applicable. Only online delivery was assessed as the inclass did not have enough students to make in the Fall Term. Action plan is to continue to review and discuss both question improvement and instructional improvement with the adjunct faculty.

4. (ALL) Who was involved in the development of the plan/report? Kathy Gladu Effectiveness Process Facilitator, Bill Barfield Instructional Program Manager, and Jennifer Ayres, Adjunct faculty (there are no full time faculty in these programs) along with input and approval by the Advisory Committee.

5. (ACAD and EDUC SUPPORT) In assessment plan development and implementation, what process did you use to ensure sampling of all campuses, centers, high school dual enrollment, distance education, and military education, if applicable to your program? Only the one delivery system was assessed this cycle.

Suggest Documents