. The Balanced Scorecard as a Strategy and Assessment Tool

Jim Self Strategic Assessment Services University of Virginia Library

Texas Library Association April 26, 2013 Fort Worth

Balanced Scorecard • Originally a commercial management tool

• Now used by mission-driven nonprofits • Reflects organization's goals and strategy • Written and unwritten

• Uses specific, measurable goals • From different perspectives • Integrates diverse data into a single system

• Integrates strategy and measurement

The “Balance” of the Scorecard • Four areas of focus • The user perspective • The finance perspective • The internal process perspective • The learning and growth perspective

Scorecard perspectives • User perspective: "how well is the library meeting the

needs and expectations of its customers?” • Finance perspective: "how well are the library's finances managed?" • Internal process perspective: "how well do processes function to deliver library services?" • Future or learning perspective: "how well is the library positioned to ensure future success?"

Balanced Scorecard Illustrated User perspective

Process perspective

Finance perspective

Future/ Learning perspective Jim Self,

The Balanced Scorecard • Reflects the organization’s strategy • Clarifies and communicates the strategy

• Measures the health and success of the organization

Rationale for the U.Va. BSC: Getting Control of the Data • Focus

• Balance • Assessment • Intelligibility

The Culture of the U.Va. Library • Customer satisfaction • Growth and development • Innovation and flexibility

• Collecting and using data • Efficiency and effectiveness • Staff participation and discussion

BSC history at the U.Va. Library • Implemented in 2001 • Metrics tallied FY02 through FY09 • Abbreviated scorecard for FY10 • Focus on assessment, not strategy • Studied and refined in FY10-11 • Strategy map devised in FY12

Developing the U.Va. Scorecard Define goals in each perspective 2. Select indicators for each goal 3. Construct metrics 4. Collect data and review results 1.

Defining the Goals -- Reflecting Values • What is important? • What is the institution trying to accomplish?

• What is the library trying to accomplish?

Choosing Metrics --Being Practical • Use existing measures when possible

• Use sampling • Collect data centrally • Minimize work by front line

Focus on Performance Metrics • Clearly defined measurements with specific targets

indicating success, partial success, or failure. • At the end of the year we knew if we met our target for

each metric.

What Sort of Measures? • Customer survey ratings • Staff survey ratings • Timeliness and cost of service

• Usability testing of web resources • Success in fundraising • Comparisons with peers

15

Importance of Targets • Measure quantitatively • Set challenging, but achievable targets • Use two sets of targets: • Complete success • Partial success • Address problems that are revealed

16

October 2007

Metric U.1.A: Overall rating in student and faculty surveys • Target1: An average score of at least 4.25 (out of 5.00)

from each of the major constituencies. • Target2: A score of at least 4.00.

FY10 Result: Target2 • Graduate students 4.17 • Undergraduates 4.15 • Faculty 4.26

17

Metric F.1.B. Library spending compared to University expenditures • Target1: : The University Library will account for at least

2.50% of the University’s academic division expenditures. • Target2: : The Library will account for at least 2.25% of expenditures. • Result FY10: Target2. • 2.40% ($29.0M of $1207M)

Balanced Scorecard UVa Fiscal Year 2009

Target1

Target2 Not Met

Success of the BSC at UVa • Wide participation among staff • Appreciation of importance of assessment

• Consistency of goals through the years • Higher performance of targeted activities • Attention to areas needing improvement

• Demonstration of Library value to institution • Articulation of particular values and goals • Visibility and leadership among libraries

• Development of the strategy map

Shortfalls and Challenges at U.Va. • Limited connection with planning process • Limited impact on digital programs

• Too many metrics • Skepticism among some staff • Administrative transitions

Enabling research, teaching, and learning through services, collections, tools, and spaces for the faculty and students of today and tomorrow. Customers

C1. Improve the student experience

Finance

C2. Facilitate scholarship

C3. Enhance the reputation of the University

Internal Operations

F1. Increase the financial base

F2. Provide resources and services with a high ratio of value to cost

F3. Raise funds for high priority initiatives

Learning & Growth

I1. Ensure preservation of the scholarly record (C2/C3)

I3. Maximize spaces for research and scholarship (C1/C2)

L1. Restructure and develop new models for leadership

L3. Develop workforce to gain needed skills

I2. Improve ease of access to resources (C1/C2)

I4. Support new models of research and scholarship (C2/C3)

L2. Develop effective processes to carry innovation into production

L4. Align Library priorities with mission and goals of UVa schools & departments

FY2011/2012

August 29, 2011

The BSC elsewhere • Considerable discussion and interest among libraries • Limited implementation until 2009/10 • ARL Scorecard Project • 4 participants initially • 10 more in the 2011/12 cohort • Currently looking for 2013/14 participants

Goals of the ARL project • Encourage a culture of assessment • Link strategy, planning, and assessment • Learn from others

• Revisit and revise our own scorecard • Benefit from knowledgeable consultants • Associates of Kaplan and Norton, the BSC inventors

What did the first ARL cohort learn? • BSC is • A performance management system • Built on a clear organizational strategy • A way to align resources and drive change • BSC is not • Merely a data container or assessment tool • An isolated project managed by a few people

ARL Project as of September 2012 : Progress in Creating Measures Status Have not started

0

Started but all measures still “under development”

4

Some measures fairly developed, but most undeveloped

1

Most measures fairly developed, a few undeveloped

2

All measures fairly developed, but awaiting approval

0

All measures fairly developed and starting to collect data

1

All measures fairly developed and starting to collect data on all of them

0

TOTAL RESPONDENTS

8

Keys to Scorecard Success • A good planning process already in place • Leadership engagement • Organizational commitment

• Association of Research Libraries • Invitation to participate in Cohort 3 • February 27, 2013

Overview of recent BSC experiences • Focus on strategic developments • Many strategy maps

• Not so much measurement

A few stories • VIVA consortium and ICOLC • Notre Dame • McMaster • Texas

VIVA and ICOLC • To explore suitability of BSC • To encourage cross-consortial collaboration • To develop a toolkit for metrics • Eight participating consortia • Three in-person sessions, plus calls

• All produced strategy maps, • with the customer on top • Metrics are works in progress

VIVA’S STRATEGY MAP VIVA's mission is to provide, in an equitable, cooperative and cost-effective manner, enhanced access to library and information resources for the Commonwealth of Virginia's nonprofit academic libraries serving the higher education community. CUSTOMER Deliver Value Equitably Maximize Access Address Expectations

FINANCIAL Maximize Resources Demonstrate Value Fund for the Present & the Future

PROCESS Strengthen Communication Nurture Collaboration and Engagement Ensure Efficient & Effective Operations

LEARNING & GROWTH Build Staff Capabilities Explore Wider Vision

Optimize VIVA Culture

Notre Dame • ARL Cohort 2 • To develop wide and ongoing assessment • Influenced by the U.Va. Scorecard History • Closely tied to University goals • Focus on creating a sustainable culture of

continuous improvement and service excellence

U.T. Austin • “The University of Texas Libraries is committed to

embedding a planning and continuous improvement process into the fabric of the organization in order to provide strategic direction, optimise our staff and resources and enable orginisational accountability.”

• Meredith Taylor, Northumbria International Conference, 2011

U.T. Austin • Not part of the ARL initiative • Used a consultant and many library staff • Implemented the BSC in a relatively short time • 13 objectives, 21 initiatives, 34 measures • What they learned:

• Track only the most important measures • Use diplomacy when devising measures • It takes a lot of staff time

McMaster University • ARL Cohort 1 • Living the scorecard! • Putting the score in scorecard! • Has measures in place • Complete public disclosure of results

• Putting the score in scorecard • Ongoing reporting and updating • Requires bravery • …and a willing to talk and to adapt.

Questions to ask… before you implement. • Do you have ongoing administrative support?

• Are you willing to give sufficient time and attention? • Are you willing to state goals, metrics, targets clearly? • Are you willing to acknowledge some failures? • Are you expecting perfection? • Can you keep at it?

Questions and Comments

• Jim Self

• [email protected] • http://assessment.library.virginia.edu/ • http://www.arl.org/stats/balanced-scorecard/index.shtml