EDF 6471 Survey Design and Analysis in Educational Research Fall

EDF 6471— Survey Design and Analysis in Educational Research Fall - 2005 Time: Tuesdays 12:50 - 1:40 pm Wednesdays 12:50 - 2:45 pm Room: NRN 219 Profe...
7 downloads 0 Views 76KB Size
EDF 6471— Survey Design and Analysis in Educational Research Fall - 2005 Time: Tuesdays 12:50 - 1:40 pm Wednesdays 12:50 - 2:45 pm Room: NRN 219 Professor: Dr. Walter Leite Office: Norman Hall 1420 Phone: 352-392-0723 extension 240 E-mail: [email protected]

Office Hours: Tuesdays 2:00 – 3:30 pm Wednesdays 3:00 – 4:30 pm Or by appointment

Required Course Material •

Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method. New York, NY: John Wiley & Sons, Inc.



Bers, T. H., & Seybert, J. A. (1999) Effective Reporting. Association for Institutional Research. (The order form for this book will be provided by the instructor)



Course package available at Target Copy.

Course Overview This course will prepare students with the skills to be critical users of survey data, undertake their own surveys, and analyze secondary survey data. This course assumes a basic understanding of simple statistics: estimates of population ratios, means, standard deviations, variances, and standard errors of these parameter estimates. This course does not assume previous experience with psychometric properties of questionnaire items. This course will require the use of spreadsheet programs, such as Excel, to undertake some simple statistical analyses. Students who do not know how to use spreadsheets are strongly encouraged to learn as soon as possible.

Course Assessment Projects: There will be three projects, each designed to give students a chance to apply and practice the concepts learned in class; you will be writing survey questions, drawing samples and calculating sampling variance, and designing a survey administration plan. Feel free to consult each other when working on the projects, but work should be done individually. Assignments can be handwritten but must be well organized and easy to understand. Also, you should always keep a photocopy or electronic copy of your work for your own protection. The dates the projects are due are indicated in the table attached, and should be submitted on time for full earned credit. Late work will be accepted for full earned credit IF AND ONLY IF arrangements are made with me PRIOR TO THE DUE DATE. Otherwise, 5% of the points possible will be deducted for each weekday the assignment is late.

Quizzes: On every other Wednesday (tentatively, 9/7, 9/21, 10/5, 10/19, 11/2, 11/16) a short quiz will be administered. Each quiz will cover material from the lessons since the last quiz (unless otherwise specified) and should not take more than 15 or 20 minutes. Missed quizzes may not be made up unless arrangements have been made prior to class. You will be able to drop your lowest quiz grade. Final Project: There will be a comprehensive take-home project assigned on the 15th week of classes; it will be due on the last day of class. The project will consist of a proposal for a survey, in response to three alternatives presented in class. Your survey proposal should be professional in content. You will use the training developed over the semester to prepare the proposal. Students are on their honor to prepare the proposal completely independently; students found doing otherwise will be subject to the maximum university penalties.

Course Grades This course is not graded on a curve (norm-referenced). Final grades will then be assigned based on the scale below: Assessment Project #1 Project #2 Project #3 Total quiz points converted to a percentage Attendance and participation in class work Total final project points converted to a percentage

weight 20% 15% 20% 15% 5% 25%

Overall course percent 90.0% - 100% 85% - 89.9% 80.0% - 84.9% 75% - 79.9% 70.0% - 74.9% 65% - 69.9%

grade A B+ B C+ C D+

Unless a computational error has been made, grades will not be changed after the end of the semester. No Extra Credit Your course grades are based only on the above information. There will be no extra-credit opportunities.

Attendance You are expected to attend each class and to be on time. If you have more than one absence, your final grade will be reduced. If you need to be absent for a special reason, you must talk to your instructor in advance in order to avoid grade reduction. If you are not able to attend a class because of illness, you need to give your instructor a medical note. Academic dishonesty For University’s honesty policy regarding cheating and use of copyrighted materials, see: http://www.dso.ufl.edu/judicial/procedures/honestybrochure.php Disability Policy Students requesting classroom accommodation must first register with the Dean of Students Office. The Dean of Students Office will provide documentation to the student who must then provide this documentation to the instructor when requesting accommodation.

Calendar of Topics and Readings Following are the readings that students are expected to be doing, whether or not the material is explicitly addressed in class. Because I do not know how long topics will take to cover, at this time I am providing a tentative calendar. Topics may differ from this schedule, but I will keep you informed of changes during class. Topic

Readings

Week 1 - 8/24 Introduction History of surveys

(1) Rossi, P. H., Wright, J. D., & Anderson, A. B. (1983). Sample surveys: history, current practice, and future prospects. In: Rossi, P.H., Wright, J.D., & Anderson, A.B. (Eds.) Handbook of Survey Research. San Diego, CA: Academic Press. (2) Dillman, D. A. (2002). Navigating the rapids of change: Some observations on survey methodology on the early twenty-first century, Public Opinion Quarterly (Vol. 66, pp. 473-494): American Association for Public Opinion Research.

Week 2 – 8/30, 8/31 Preliminary survey planning Types of surveys Focus groups

(1) Czaja, R. & Blair, J. (2005). Chapter 2: Stages of a survey. In: Designing surveys: A guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press. (2) Fowler Jr., F. J. (2002). Chapter 4: Methods of data collection. In: Survey Research Methods. Thousand Oaks, CA: Sage Publications. (3) American Statistical Association. (1997). What are Focus Groups? ASA.

Week 3 – 9/6, 9/7 Question writing Quiz 1(9/7)

(1) Dillman, D. A. (2000). Chapter 2: Writing Questions. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons, Inc.

Week 4 – 9/13, 9/14 Questionnaire construction

(1) Dillman, D. A. (2000). Chapter 3: Constructing the Questionnaire. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons, Inc.

Week 5 – 9/20, 9/21 Pretesting Quiz 2 (9/21)

Assignment of Project 1 (9/21)

(1) Czaja, R. & Blair, J. (2005). Chapter 6: Questionnaire design: Testing the questions. In: Designing surveys: A guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press. (2) American Statistical Association. (1997). How to conduct pretesting. (part of the ASA Series: What is a survey?) (3) Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Short course presented at the annual meeting of the American Statistical Research Association.

Week 6 – 9/27, 9/28 Population selection Sampling procedures

(1) American Statistical Association. (1997). What is a Margin of Error? (part of the ASA Series: What is a survey?) (2) Czaja, R. & Blair, J. (2005). Chapter 7: Designing the sample. In: Designing surveys: A guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press.

Week 7 – 10/4, 10/5 Sampling procedures

Handouts to be given in class

Quiz 3 (10/5)

Week 8 – 10/11, 10/12 Survey administration and implementation

(1) Dillman, D. A. (2000). Chapter 4: Survey Implementation. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons, Inc.

Project 1 Due (10/12) Assignment of Project 2 (10/12)

Week 9 – 10/18, 10/19 Survey administration and implementation

Quiz 4 (10/19)

(1) Weinberg, E. (1983). Data collection: planning and management. In Rossi, P.H., Wright, J.D., & Anderson, A.B. (Eds.) Handbook of Survey Research. San Diego, CA: Academic Press.

Week 10 – 10/25, 10/26 Internet/web-based surveys; Mixed mode surveys Project 2 Due (10/26) Assignment of Project 3 (10/26)

(1) Couper, Mick P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494. (2) Dillman, D. A. (2000). Chapter 11: Internet and interactive voice response surveys. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons, Inc.

Week 11 – 11/1, 11/2 Internet/web-based surveys; Mixed mode surveys

Quiz 5 (11/2)

(1) Dillman, D. A. (2000). Chapter 6: Mixed-mode surveys. In: Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons, Inc. (2) Schaefer, David R., & Dillman, Don A. (1998). Development of a standard e-mail methodology: Results of an experiment. Public Opinion Quarterly, 62, 378-397.

Week 12 – 11/8, 11/9 Sources of error/nonresponse/missing data Project 3 Due (11/9)

(1) (American Statistical Association. (1998). Judging the quality of a survey. (part of the ASA Series: What is a survey?) (2) Czaja, R. & Blair, J. (2005). Chapter 9: Reducing sources of error in data collection. In: Designing surveys: A guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press.

Weeks 13 – 11/15, 11/16 Survey data analysis Presentation of survey cases for final project and assignment of final project (11/15)

(1) Fowler Jr., F. J. (2002). Chapter 8: Preparing survey data for analysis. In: Survey Research Methods. Thousand Oaks, CA: Sage Publications.

Quiz 6 (11/16)

Week 14 – 11/22, 11/23 Presentation of survey results and numerical display

(1) Bers, T. H., & Seybert, J. A. (1999) Effective Reporting. Association for Institutional Research.

Week 15 – 11/29, 11/30 Survey research ethics National databases

(1) American Statistical Association. (1998). Surveys and Privacy. (2) American Association of Public Opinion Research (2005). Code of professional ethics and practices. (3) American Association of Public Opinion Research (2002). Best practices for survey and public opinion research. (4) Handout about national databases

Week 16 – 12/6, 12/7 Issues related to developing questionnaires to measure latent variables Final projects due (11/7)

(1) Handout about measurement of latent variables

Suggest Documents