Copyright © 2007 by ABET, Inc.
Preparing an Effective Self-Study
Stu Zweben CAC Chair 1
Copyright © 2007 by ABET, Inc.
Preparing an Effective Self-Study Goal To facilitate preparation of the selfstudy by programs preparing for ABET evaluation visits in 2008-09 or later Expected outcome Attendees will understand the types of responses expected by teams to various types of questions in the self-study 2
Copyright © 2007 by ABET, Inc.
Organization Self-Study Basics and Context – Zweben Frequently Asked Questions – Hodek Preparing an Effective Report of Your
Assessment Process – Rogers
Completing the Institutional Summary –
Robinson
Completing Program Specific Questions -
Wolfe
3
Copyright © 2007 by ABET, Inc.
Self-Study Basics and Context Presents your program to the evaluation
team
Informs team of elements of the program
as they relate to the criteria
Affords team its FIRST IMPRESSION of the
extent to which the program meets the criteria
Gives an impression of the institution’s
preparation for the upcoming visit
4
Copyright © 2007 by ABET, Inc.
Audience for the Self-Study Team Chair Overall team manager Typically responsible for institutional issues Responsible for overall report and presentation to commission detailing findings Experienced evaluator, but maybe not in your specific discipline Program Evaluator(s) Concerned with program-specific details Will coordinate findings with other PEVs on team and with team chair to seek consistent and appropriate interpretation relative to the criteria Expertise in specific discipline May or may not have a lot of ABET experience 5
Copyright © 2007 by ABET, Inc.
Tips to Connect With Your Audience Make it easy for the reader to find information
required
Good Table of Contents To-the-point responses Specific pointers to documents or other sections as
appropriate
Clearly explain institution or program specific
jargon
OK to use disciplinary jargon Footnote if not sure what response is expected,
to explain your interpretation
6
Copyright © 2007 by ABET, Inc.
How is the Self-Study Organized? In concert with the criteria Students Program Educational Objectives Program Outcomes Continuous Improvement Curriculum Faculty Facilities Support Program Criteria (as applicable)
7
Copyright © 2007 by ABET, Inc.
Types of Responses Narrative explanations Tables and figures Appendices (vitae, course
descriptions, institutional summary)
8
Copyright © 2007 by ABET, Inc.
Differences Across Commissions No difference in institutional tables
required in the appendix
Other differences reduced through
harmonization of criteria categories
Further harmonization in process to
word common criteria identically when possible
9
Copyright © 2007 by ABET, Inc.
Preparation Process Tips Self-study due July 1 of year of visit Self-study template posted on ABET website in
July of year prior to visit
Appoint leader of document preparation early in
fall prior to year of visit
Assign tasks to key persons at program, college
and institutional level as appropriate
Synthesize materials into coherent whole Leave time for review before due date ABET HQ staff will help as questions arise 10
Copyright © 2007 by ABET, Inc.
Frequently Asked Questions
Ralph Hodek ASAC Chair 11
Copyright © 2007 by ABET, Inc.
Frequently Asked Questions By the Self-Study Writer By the Program Chair By the Dean By the Chief Operating Officer
12
Copyright © 2007 by ABET, Inc.
How Do I Start? Get a copy of the Questionnaire.
ASAC CAC EAC TAC
Self-Study
It is a guided tour of what you will
want to put into the report. It can be used as a check list. It is not intended to be limiting.
13
Copyright © 2007 by ABET, Inc.
Am I Required to Follow the Format of the Self-Study Questionnaire? No, but it is a very rational outline. The reader expects to see your self-
study report developed more-or-less in this manner. Be sure to include all items that appear in the Self-Study Questionnaire Table of Contents. 14
Copyright © 2007 by ABET, Inc.
What About Tables and Figures? Don’t change the format without a good
reason.
Feel free to add additional tables and/or
figures to make your self-study more understandable to the primary reader, your program evaluator.
The objectives are content (numbers,
facts, and trends) and clear communication.
15
Copyright © 2007 by ABET, Inc.
What is the Time Period For My Self-Study? The self-study should reflect the academic
year in which it is produced and submitted.
Assessment results and analyses probably
will go back several years.
Upcoming changes to the program should
be mentioned, particularly if they will be effective by the time of the visit.
16
Copyright © 2007 by ABET, Inc.
We Made Major Changes in the Program Recently. What Do We Do (No New Data)? Great! You identified through your
program of continuous improvement that change was needed to achieve objectives and/or outcomes.
Describe what led to the changes and
when the impact of the changes will be determined. 17
Copyright © 2007 by ABET, Inc.
We Thought We Were Ready for Accreditation or Reaccreditation, But the Self-Study Questionnaire Demonstrates That We Need Another Year of Preparation. What Do We Do? 18
Copyright © 2007 by ABET, Inc.
Seek the advice of Maryanne Weiss at ABET headquarters right away. Do not wait until the last minute. Several alternative solutions may be available to your problem.
19
Copyright © 2007 by ABET, Inc.
What Are the Visitors Really Looking For? A demonstration that your program meets
the criteria. Continuous improvement is an ongoing process. The visiting team is looking over the program’s shoulder at that ongoing process to determine whether that process is being applied continuously, and not just before the self-study report must be prepared. 20
Copyright © 2007 by ABET, Inc.
Preparing an Effective Report of Your Assessment Process
Gloria Rogers ABET’s Associate Executive Director of Professional Services 21
Copyright © 2007 by ABET, Inc.
For Guidance Only The institution may employ any means it chooses to represent its assessment and evaluation processes to the visiting team. Consequently, the references to specific processes in the following are for guidance only. The information may be presented in any manner the institution chooses. 22
Copyright © 2007 by ABET, Inc.
Components of an Assessment Process Program Educational Objectives and
Learning Outcomes
Similar but not the same What are some of the differences? Degree of specificity Role of constituents Types of measurements possible Cycles of data collection 23
Copyright © 2007 by ABET, Inc.
24
Copyright © 2007 by ABET, Inc.
25
Copyright © 2007 by ABET, Inc.
Reporting your results What (are you assessing) Who When What (are your results) What (did you do with the
results) What (difference did it make) 26
Copyright © 2007 by ABET, Inc.
Reporting Results Data or Information?
27
Copyright © 2007 by ABET, Inc.
Reporting Results: Data or Information?
28
Copyright © 2007 by ABET, Inc.
Samples of 51, 56 and 58 students (49% of 2001, 52% of 2003 and 51% of 2005 cohort) were assessed. This represents 2 of 4 sections of ME412 which is the second semester of a twotwo-semester team experience.)
29
Copyright © 2007 by ABET, Inc.
Table
Teaming: Samples of 51, 56 and 58 students (49% of 2001, 52% of 2003 and 51% of 2005 cohort) were assessed. This represents 2 of 4 sections of ME412 which is the second semester of a two-semester team experience.) Outcome
Performance Criteria
Research and Gather Information
Teaming
Ethics Life Long Learning
2001
2003 2005
61%
72%
84%
Fulfill team roles
50%
65%
72%
Share work
58%
62%
75%
Listens
70%
89%
91%
Know the code of ethics
45%
64%
77%
Analyze ethical issues
32%
56%
74%
Conduct independent research
64%
68%
65%
Identify opportunities for continued education in the field
57%
67%
86%
Indicate interest in continuing education
65%
76%
87%
30
Learning Outcome: Students can work effectively in teams Performance Criteria
Strategies
Assessment Method(s)
Source of Assessment
Time of data collection
Assessment Coordinator
Evaluation of Results
1.
Research and gather information
ME 234, ME256, ME113, EM213, ME213, ME235, ME333, ME412
Peer Evaluations, Faculty Evaluations
ME412
Spring- odd years (2003, 2005)
2003 – Brackin Curriculum 2005 - Richards Committee
2.
Fulfill team roles
ME113, EM213, ME213, ME235, ME333, ME412
Peer Evaluations, Faculty Evaluations
ME412
Spring- odd years (2003, 2005)
2003 – Brackin Curriculum 2005 - Richards Committee
3.
Shares work equally
ME113, EM213, ME213, ME235, ME333, ME412
Peer Evaluations, Faculty Evaluations
ME412
Spring- odd years (2003, 2005)
2003 – Brackin Curriculum 2005 - Richards Committee
4.
Listens to teammates
ME113, EM213, ME213, ME235, ME333, ME412
Peer Evaluations, Faculty Evaluations
ME412
Spring- odd years (2003, 2005)
2003 – Brackin Curriculum 2005 - Richards Committee
Department
Department
Department
Department
Results 2003: A sample of 56 students (52% of 2005 cohort) were assessed. This represents 2 of 4 sections of ME412 which is the second semester of a two-semester team experience.) The percent of the sample that demonstrated each criterion were as follows: Criterion 1 - 72%; Criterion 2 - 65%; Criterion 3 - 62%; Criterion 4 - 89%. Actions 2004-05: Based on the analysis of the results, the department asked faculty to provide the teaming evaluation rubrics to students with the course assignments where the students were provided opportunities to demonstrate their teaming skills as defined by the criteria. A sub-committee of the department Curriculum Committee met to review the performance criteria. It was decided not to make any changes at this time. Faculty decided that they would review their assignments to be sure that students were given adequate opportunities to demonstrate the performance identified for teaming. Faculty also agreed to make students performance on the criteria a part of their grade for the activity. The Teaching/Learning Center will also provide a seminar for faculty on how to integrate effective teaming into the classroom. Second-Cycle Results: A sample of 59 students (51% of cohort) were assessed. This represents 2 of 4 sections of ME 412 which is the second semester of a two –semester team experience. Based on changes made, the following improvements were seen: Criterion 1 – +12% (84%); Criterion 2 - +7% (72%); Criterion 3 - +13% (75%); Criterion 4 - +2% (91%). Actions 2006-07: Although progress was made on all criteria, the Curriculum Committee recommended that the Department take another look at all the performance criteria related to teaming. The Teaching/Learning Center was asked to provide the Department some feedback on the criteria and also provide other examples of teaming performance criteria. This will be one of the issues that 31 will be discussed at the Department retreat for possible revisions for the 2007 academic year.
Copyright © 2007 by ABET, Inc.
Common Mistakes Too many data, not enough
information
Reporting numbers or percentages
without putting them into context
How many students/graduates in cohort How many students/graduates provided
data
Not describing how the data are
evaluated Using very complex charts describing your assessment processes
32
Yr 1
Yr 4
Yr 3
Yr 2
Repeat process to check for improvement
Copyright © 2007 by ABET, Inc.
Common Mistakes Discussing all outcomes/objectives at
once instead of one at a time. Using the terms “objectives” and “outcomes” interchangeably. Referencing the outcomes/objectives by numbers or letters that refer back to a chart. Don’t require the reader to go back in the self-study for the reference.
35
Copyright © 2007 by ABET, Inc.
Program Educational Objectives 1. 2.
3. 4.
Supporting Program Outcomes a, b, c, e, k, j d, g , l e, f, I, j, l h, I, j
36
Program Educational Objectives
1. Be effective in engineering design and the practical application of engineering theory
2. Exhibit teamwork and effective communication skills
3. Be characterized by effective leadership skills and high standards of ethics
4. Expand their knowledge and capabilities
Copyright © 2007 by ABET, Inc.
Supporting Program Outcomes a) ability to apply knowledge of math & science b) ability to design and conduct experiments/ analyze data c) ability to design a system, component, or process to meet needs with realistic constraints e) ability to identify, formulate, and solve engineering problems k) ability to use the techniques, skills, and modern engineering tools needed for engineering practice j) knowledge of contemporary issues d) ability to function on multidisciplinary teams g) ability to communicate effectively l) a willingness to assume leadership roles and responsibilities e) ability to identify, formulate, and solve engineering problems f) understanding of professional and ethical responsibility i) Recognition of and ability to engage in lifelong learning j) knowledge of contemporary issues l) a willingness to assume leadership roles and responsibilities h) broad education to understand effect of engineering solutions in a global, economic, environmental, and societal context i) Recognition of and ability to engage in lifelong learning j) knowledge of contemporary issues 37
Copyright © 2007 by ABET, Inc.
Summary Keep the report focused. Have someone read your report
that knows nothing about your program. If they don’t understand it, chances are neither will the visiting team. There is elegance in simplicity.
38
Copyright © 2007 by ABET, Inc.
SelfSelf-Study Questionnaire
Institutional Summary Appendix
Mike Robinson TAC Chair 39
Copyright © 2007 by ABET, Inc.
For Guidance Only The institution may employ any means it chooses to represent its assessment and evaluation processes to the visiting team. Consequently, the references to specific processes in the following are for guidance only. The information may be presented in any manner the institution chooses. 40
Copyright © 2007 by ABET, Inc.
Institutional Information The Institution
Educational Unit
Type of Control
Credit Unit
History of Institution Instructional Modes Student Body
Grade-Point Average
Accreditation
Academic
Personnel & Policies
Supporting Units Faculty Workload
41
Copyright © 2007 by ABET, Inc.
Table 1. Programs Offered by the Educational Unit
Modes Offered2
Program Title1
D a y
Mechanical Engineering Technology
X
Mechanical Engineering Technology
X
Electronics Engineering Technology
X
Coop
X
Off Cam pus
Altern ate Mode
Nomin al Years to Compl ete
Administ rative Head
4
M. E. Tec
5
M. E. Tec
2
S. M.
Conductor
Administrati ve Unit or Units (e.g. Dept.) Exercising Budgetary Control
Submitted for Evaluation3 Now Accre dited.
School of
X
School of
X
Technolog y
Technolog y
School of
Technolog y
Not Now Accre dited
Offered, Not Submitted for Evaluation4 Now Accre dited
Not Now Accre dited
X
42
Copyright © 2007 by ABET, Inc.
Table 2. Degrees Awarded and Transcript Designations by Educational Unit Modes Offered2
Program Title1
Da
Mechanical Engineering Technology
X
Mechanical Engineering Technology
X
Electronics Engineering Technology
X
Coy op
Off Campu s
Alternativ e
Mode
X
Name of Degree Awarded3
Designation on Transcript4
Bachelor of Science
Bachelor of Science
Bachelor of Science
Bachelor of Science
Associate of Science
Associate of Science
43
Copyright © 2007 by ABET, Inc.
Table 3a. Support Expenditures School of Technology 07 – 08 Current year (estimate)
06 - 07
Fiscal Year
08 – 09 Next Year (budgeted)
Expenditure Category Operations staff)4
(not
including $414, 679
Travel5
$66425
$400,647
$466,432
$69299
$58800
Equipment6 (a) Institutional Funds
$110,560
$40600
$202160
(b) Grants and Gifts7
$912218
$280674
$484975
Graduate Teaching Assistants
0
0
0
Part-time Assistance8 (other than teaching)
0
0
0
Faculty Salaries
$1,705,000
$1,905,000
$2,100,000 44
Copyright © 2007 by ABET, Inc.
Table 3b. Support Expenditures
Mechanical Engineering Technology Program (A.S. & B.S.) 07 – 08 Current year (estimate)
06 - 07
Fiscal Year
08 – 09 Next year (budgeted)
Expenditure Category Operations staff)4
(not
including $171,141
Travel5
$30881
$130,720
$141,800
$34162
$40701
Equipment6 (a) Institutional Funds
$3234
$27200
$10,526
(b) Grants and Gifts7
$720,050
$162,017
$406,320
Graduate Teaching Assistants
0
0
0
Part-time Assistance8 (other than teaching)
0
0
0
Faculty Salaries
$340,000
$381,000
$420,000 45
Copyright © 2007 by ABET, Inc.
Table 4a. Personnel and Students School of Technology (Fall 07) Headcount FT
PT
FTE
Ratio to Faculty
Administrative4
5
0
5.0
Faculty (tenure-track)
24
3
25.0
Other Faculty (excluding Student Assistants)
0
0
0
Student Teaching Assistants
0
0
0
0
Student Research Assistants
0
0
0
0
Technicians/Specialists
2
1
2.5
0.1
Office/Clerical Employees
3
0
3.0
0.12
Others5
0
0
0
0
Undergraduate Student enrollment6
600
150
700
28.0
Graduate Student enrollment
0
0
0
0 46
Copyright © 2007 by ABET, Inc.
Table 4b. Personnel and Students Mechanical ET Program (Fall 07) Headcount FT
PT
Ratio to Faculty
FTE
Administrative4
1
0
1.0
Faculty (tenure-track)
4
2
5.0
Other Faculty (excluding Student Assistants)
0
0
0
Student Teaching Assistants
0
0
0
0
Student Research Assistants
0
0
0
0
Technicians/Specialists
1
0
1.0
0.2
Office/Clerical Employees
1
0
1.0
0.2
Others5
0
0
0
0
100
0
100
20.0
0
0
0
0
Undergraduate Student
enrollment6
Graduate Student enrollment
47
Copyright © 2007 by ABET, Inc.
Table 5a. Program Enrollment and Degree Data School of Technology Tota
CURRENT (07-08)
06 - 07
05 - 06
Enrollment Year
Und Gra
Academi c Year
1st
2nd
3rd
4th
5th
60 0
FT
18 0
140
120
100
60
600
15 0
PT
70
30
20
10
20
150
l d
Tota
l Grad
Degrees Conferred Bachelo r
Master
Doctor
0
75
0
0
12 0
0
15
0
0
20
FT
0
0
0
PT
0
0
0
FT
0
0
0
PT
0
0
0
Ot
48
h
Copyright © 2007 by ABET, Inc.
Table 5b. Program Enrollment and Degree Data Mechanical ET Program Tota l Grad
5th
Tota l Und Gra d
Enrollment Year
CURRENT (07-08)
Academi c Year
1st
2nd
10 0
FT
35
30
16
14
5
600
0
PT
0
0
0
0
0
0
06 - 07
05 - 06
3rd
4th
Degrees Conferred Bachelo r
Master
Doctor
Ot h
0
21
0
0
30
0
0
0
0
0
FT
0
0
0
PT
0
0
0
FT
0
0
0
PT
0
0
0 49
Copyright © 2007 by ABET, Inc.
Table 6a. Faculty Salary Data School of Technology (07(07-08)
Professor
Associate Professor
Assistant Professor
Instructor
Number
5
10
5
5
High
$100,000
$90,000
$80,000
$70,000
Mean
$90,000
$80,000
$70,000
$60,000
Low
$80,000
$70,000
$60,000
$50,000
50
Copyright © 2007 by ABET, Inc.
Table 6b. Faculty Salary Data Mechanical ET Program (07(07-08) Professor
Associate Professor
Assistant Professor
Instructor
Number
1
2
1
1
High
$92,000
$90,000
$72,000
$57,000
Mean
$92,000
$80,000
$72,000
$57,000
Low
$92,000
$70,000
$72,000
$57,000
51
Copyright © 2007 by ABET, Inc.
SelfSelf-Study Questionnaire
Reporting Curriculum and Faculty Information
Mary Leigh Wolfe EAC Chair 52
Copyright © 2007 by ABET, Inc.
Criterion 5. Curriculum Curriculum Table (Table 5-1) Course and Section Size Summary
(Table 5-2)
Course Syllabi Course Material Display Prerequisite flow chart (all except
applied science)
53
Copyright © 2007 by ABET, Inc.
Curriculum Table Category (Credit Hours)
Math & Basic Sciences Year; Semest er or Quarter
Course (Department, Number, Title)
Professiona l Program Topics or Program Core or Program Advanced or Engineerin g Topics
General Education
Other
3 1.5
1.5 54
Copyright © 2007 by ABET, Inc.
“One Year” of Study Criteria
One Year of Study =
Computing (CAC)
30 semester credits (45 quarter credits)
Engineering (EAC)
32 semester credit hours OR 25% of total credits for degree, whichever is less 55
Copyright © 2007 by ABET, Inc.
Notes About Curriculum Table Options, concentrations, pathways Current vs. new curriculum Add footnotes if helpful
56
Copyright © 2007 by ABET, Inc.
Syllabi - Purpose Support classification of courses as
shown in curriculum table Show scope of courses – breadth/depth of topics Indicate rigor of courses Indicate textbooks or other supporting documents
57
Copyright © 2007 by ABET, Inc.
Syllabi - Format Follow specified outline (specifies
minimum required information) Common format for all courses No more than two pages for each course (engineering and technology) 2 or 3 pages for each course (computing) 58
Copyright © 2007 by ABET, Inc.
Link Between Curriculum and Outcomes Explain how the curriculum
addresses the program outcomes. Include a table showing how each course contributes to the program outcomes. (computing)
Syllabi include course relationship to
outcomes
59
Copyright © 2007 by ABET, Inc.
Criterion 6. Faculty Summary of faculty activities and
responsibilities
Faculty workload summary (table) Faculty analysis (table) Faculty resumes
60
Copyright © 2007 by ABET, Inc.
Criterion 6. Faculty Summary and Description of Composition (including size), credentials, experience, and workload of program faculty Teaching, research, and other scholarly activity and performance Service activity and performance Course and program development and delivery Competencies Professional development activities and funding Process and timeline for ongoing assessment of program faculty (applied science programs only) 61
Copyright © 2007 by ABET, Inc.
Faculty Analysis Table Name
Years of Experience
Rank
Govt./Industry Practice
Type of Academic
This Institution
Total Faculty
Appointment TT, T, Professional NTT Registration/ Certification FT or PT Level of Activity Highest Degree and (high, med, low, Field none) Institution from which Professional Society Highest Degree Research Consulting/Summer Work Earned & Year in Industry
62
Copyright © 2007 by ABET, Inc.
Faculty Vitae/ Résumés - Purpose Support summary in faculty
analysis table Show education, experience, recent and current activities, currency in the field Help program evaluator identify who to interview 63
Copyright © 2007 by ABET, Inc.
Faculty Vitae/ Résumés - Format Common format for all faculty -
include requested information 2-page limit (engineering and
technology) Suggested 2-page limit (applied science) 3-page limit (computing) 64
Copyright © 2007 by ABET, Inc.
Thank You for Your Attention! Time for Q&A
65
