Reference Services Evaluation Process

Reference Services Evaluation Process Overview Ten years ago the UCSD Biomedical Library adopted a Single Service Desk (SSD) model as a way to improve...
Author: Cassandra Woods
36 downloads 1 Views 270KB Size
Reference Services Evaluation Process Overview Ten years ago the UCSD Biomedical Library adopted a Single Service Desk (SSD) model as a way to improve customer service. It has become increasingly difficult to staff the ”reference side” of the desk as demands on librarian schedules have forced many to cut back on contributions to in-person reference services. Changes in technology and user expectations have dramatically shifted the way academic medical libraries provide reference services. Many librarians spend more time answering e-mail reference questions, participating in chat reference services, and providing more information literacy instruction to groups (online and in-person). Deep budget cuts across the UCSD Library system have required serious reflection about services provided. The University Librarian has recently asked librarians to consider what they will stop doing to be able to sustain core services. In addition, library-wide priorities have led to librarian schedules becoming fuller, making them less available to staff the SSD. Currently, there are more desk hours in one week need to be covered than there is staff to cover it. Numerous discussions at Reference staff meetings about how to handle the situation have led to more questions and concerns than answers. It is clear that more data is needed to make a decision about the future of reference services.

Evaluation Process This plan suggests how to collect data to make decisions about providing reference services at the Biomedical Library. It is organized in four stages as described in the figure on page 2. Before any plan is implemented, this should be discussed by staff at a Reference meeting. Suggestions and changes should be noted in the minutes for consideration. A task force should be created to coordinate and implement the evaluation. The task force should have the following three members, but not exceed five people: • • •

Interim Head of Instruction & Outreach Services One librarian SSD representative One non-librarian SSD representative

Once the task force is created, they should consider incorporating changes or recommendations from the Reference staff meeting before proceeding.

EDTEC 685: D. Turnbow

1

Evaluating Reference Services at the UC San Diego Biomedical Library

Stages of the Evaluation Process

II. Staff Retreat (1 month) I. Review Extant Data (1 month) • Task force collects data described for Stage I in the Information Table (p. 3). • Data is summarized and distributed to all UCSD Biomedical Library staff members for review

EDTEC 685: D. Turnbow

• The task force organizes and leads the annual retreat with staff to introduce the project and discuss data. Examples: • lead a discussion about new statistics that should be collected • get feedback about other reference models identified in the literature • actively listen to concerns from staff members about the evaluation and/or implementing a new reference model )

III. Collect & Analyze Data (2-3 months) • The task force will do interviews and surveys as described for Stage III in the Information Table (pp. 4-7). • Results will be analyzed and key findings will be organized into themes in preparation for the final report.

IV. Final Report (2-3 months) • The task force will write the final report with recommendations that include a new reference services model(s). • If possible, this report will include a plan for implementation.

2

Evaluating Reference Services at the UC San Diego Biomedical Library

Information Table Stage 1

1

Purpose Extant Data

Literature Review

EDTEC 685: D. Turnbow

Sources

Provide background information about the current service model.

Reference desk statistics

Provide information about national trends in reference services and uncover solutions that other librarians have implemented to deal with similar challenges.

Article databases (LISA, Library Literature)

SSD competencies

Publications from: • American Libraries Association (ALA) • Association of College & Research Libraries (ACRL) • Reference & User Services Association (RUSA) • Medical Library Association (MLA)

Sample Questions

Methods

• What are the most/least common types of questions received at the desk? Via e-mail or chat? • Have the types of reference transactions changed? If so, how? • Is there overlap in the competencies for librarians and non-librarian staff? If yes, how much? Which competencies are different?

Review statistics logs and competencies document

• Do the Biomedical Library reference statistics reflect national trends? • What changes have other academic, medical libraries made to deal with similar challenges? How are the UCSD Biomedical Libraries users’ needs and expectations similar and different? • Who staffs the reference desk at other academic and medical libraries? • For libraries that made the transition from librarians staffing the desk to non-librarian staff only, what challenges were encountered? How did they resolve them? • What kinds of training do libraries provide nonlibrarian staff for triage, if any? • Do other libraries view reference services as a part of an overall instructional program or separate? • How do libraries evaluate the effectiveness of reference services?

Read literature from article databases, blogs, web sites, etc.

3

Evaluating Reference Services at the UC San Diego Biomedical Library

Stage

Purpose

Sources

Sample Questions

Methods

3

Library Director

Provide an overall vision for the Biomedical Library that is consistent with the UCSD Libraries strategic plan.

Melissa Just, UCSD Biomedical Library Director

• Do you have a vision for the Biomedical Library SSD? What would you like to see it look like in one year? In five years? • Do you think the SSD model is working right now? Why/why not? • Do you think it is important to consider alternative staffing models for the SSD at this point in time? Why/why not? • Do you think it is necessary for librarians to be on the SSD to answer in-person questions? Explain. • Do you know other libraries with similar staffing challenges? What have they done to resolve this problem? Do you think the solutions would work here? Why /why not? • What are your concerns, if any, about having only non-librarian staff on the SSD? • What components would an ideal solution for staffing the SSD include? What would it look like?

Interview (by a task force member)

3

Librarians

Provide service on the reference side of the desk and can speak to competing priorities.

Librarians that staff the SSD at the Biomedical Library (n=8)

• Provide a list of typical librarian responsibilities (including staffing the SSD) and ask participants to rate the importance of them in relation to one another. • Do you think the SSD should continue to be staffed by a librarian for all of the current hours (9-5, M-F)? Why/why not? • Currently, there are at least two people on the SSD. Do you think it is necessary to keep the desk staffed with at least two people (librarian or non-librarian) going forward? If yes, why? If not, what hours do you think could be staffed with only one person?

Anonymous online survey

EDTEC 685: D. Turnbow

4

Evaluating Reference Services at the UC San Diego Biomedical Library

Stage

Purpose

Sources

Sample Questions

Methods

• What are your concerns, if any, about having only non-librarian staff on the SSD? • Provide a list of the SSD competencies. Which tasks/questions must be done/answered by a librarian (instead of non-librarian staff)? How often do these tasks/types of questions occur at the reference desk? • Would you be comfortable with non-librarian staff being trained to triage questions that you think should be answered by a librarian? What are your concerns about using a triage model for reference questions? • Do you know of successful staffing models that have been implemented at other universities? If so, please describe them here. 3

Non-librarian Provide service on the staff circulation side of the desk. They would be most affected by librarians not staffing the desk because all service transactions would go to them and they would be expected to answer reference-type questions.

Non-librarian employees (the work at least 75%) that staff the circulation side of the SSD (n=9)

• Do you think the SSD should continue to be staffed by a librarian for all of the current hours (9-5, M-F)? Why/why not? • Currently, there are at least two people on the SSD. Do you think it is necessary to keep the desk staffed with at least two people (librarian or non-librarian) going forward? If yes, why? If not, what hours do you think could be staffed with only one person? • What are your concerns, if any, about having only non-librarian staff on the SSD? • Would you be willing to be trained to know when to triage a question to a librarian? Why or why not?

Anonymous online survey

3

Non-librarian This group has been staff working trained to answer

Non-librarian employees that staff the reference

• Do you think the SSD should continue to be staffed by a librarian for all of the current hours (9-5 M-F)?

Anonymous online survey

EDTEC 685: D. Turnbow

5

Evaluating Reference Services at the UC San Diego Biomedical Library

Stage

Purpose on the reference side of the SSD

reference questions. They would be able to provide perspective about the types of questions that should be answered by librarians and feedback about training needs.

Sources

Sample Questions

side of the SSD (n=3) •



• • • •

3

UCSD Biomedical Library primary users

Learn about the needs and expectations of the Biomedical Library’s primary audience.

Faculty from the School of Medicine and Pharmacy and the Division of Biological Sciences Graduate students from School of Medicine and Pharmacy and the Division of Biological Sciences Undergraduate students

EDTEC 685: D. Turnbow

Methods

Why/why not? Currently, there are at least two people on the SSD. Do you think it is necessary to keep the desk staffed with at least two people (librarian or non-librarian) going forward? If yes, why? If not, what hours do you think could be staffed with only one person? Provide a list of the SSD competencies. Which tasks/questions must be done/answered by a librarian (instead of non-librarian staff)? How often do these tasks/types of questions occur at the reference desk? Do you feel comfortable answer most questions at the reference side of the SSD? Do you feel comfortable calling a librarian for help when you need it? If not, why? What are your concerns, if any, about having only non-librarian staff on the SSD? Is there training that you wish you would have received prior to staffing the reference side of the SSD? Describe.

• Do you come to the Biomedical Library? If yes, how often? If user does come to the library, provide a list of spaces and services offered and ask them how often they use them. • Define research question. Have you ever asked a question like this at the library in-person? Via e-mail? Via chat? How would you like to get questions like this answered? • Provide feasible ways to provide answers to user questions (knowledge base, short videos, etc.). Ask

Anonymous online survey

6

Evaluating Reference Services at the UC San Diego Biomedical Library

Stage

Purpose

Sources from the School of Medicine and Pharmacy and the Division of Biological Sciences

EDTEC 685: D. Turnbow

Sample Questions

Methods

users to rate how likely they would be to use them. • How long are you willing to wait for an answer to a question? • Can you think of a for- or non-profit company that uses different methods for answering your questions? If yes, which company? Describe what they do and why you like it.

7

Evaluating Reference Services at the UC San Diego Biomedical Library

References Bracke, M. S., Chinnaswamy, S., & Kline, E. (2008). Evolution of reference: A new service model for science and engineering libraries. Issues in Science & Technology Librarianship. Carlson, S. (2007). Are reference desks dying out? librarians struggle to redefine-and in some cases eliminate-the venerable institution. The Reference Librarian, 48(2), 25-30. Gerlich, B. K., & Whatley, E. (2009). Using the READ scale for staffing strategies: The Georgia College and State University experience. Library Leadership & Management, 23(1), 26-30. Kuruppu, P. U. (2007). Evaluation of reference services--A review. The Journal of Academic Librarianship, 33(3), 368-381. Miller, J. (2008). Quick and easy reference evaluation: Gathering users' and providers' perspectives. Reference & User Services Quarterly, 47(3), 218-222. Murphy, B., Peterson, R. A., Vines, H., Von Isenburg, M., Berney, E., James, R., Rodriguez, M., & Thibodeau, P. (2008). Revolution at the library service desk. Medical Reference Services Quarterly, 27(4), 379-393. Neville, T. M., & Henry, D. B. (2009). Reference classification--is it time to make some changes? Reference & User Services Quarterly, 48(4), 372-383. RUSA Board of Directors. (2008). ALA definitions of reference. Retrieved 11/10/2009, 2009, from http://www.ala.org/ala/mgrps/divs/rusa/resources/guidelines/definitionsreference.cfm Sonntag, G., & Palsson, F. (2007). No longer the sacred cow – no longer a desk: Transforming reference service to meet 21st century user needs. Library Philosophy and Practice, (February), November 10, 2009. Wang, X. (2010). Performance analysis for public and nonprofit organizations. Sudbury, Mass.: Jones and Bartlett Publishers. Watstein, S. B., & Bell, S. J. (2008). Is there a future for the reference desk? A point-counterpoint discussion. The Reference Librarian, 49(1), 1-20.

EDTEC 685: D. Turnbow

8