Considerations for Conducting Pulse or Spot Surveys
INTRODUCTION The word, “pulse,” is used to describe the regular or rhythmical beating of the heart. A pulse can be a single beat or throb and measures the health of the heart. Having a pulse is a good thing: not having a pulse is disastrous. In organizations, the pulse can also be used to describe the health or well-being of the organization’s culture, climate and employees. Surveys have been used for many years in organizations to drive business decisions. Whether given to employees, customers or suppliers, surveys have been used to collect information that helps organizations take action and hopefully, make improvements to drive better business outcomes. Surveys can be focused in nature, designed to collect information about a single area or topic. Or they can be general, collecting information on a variety of topics. One type of survey that is becoming increasingly more popular with the wide use of web-based technology is the Pulse or Spot survey. A pulse survey can be used to measure the health of employees within an organization or work group. Usually a pulse survey is focused, geared toward collecting data on a limited number of questions or gathering data from a limited number of employees. A pulse survey can produce a single statistic or measure from which to track trends. The pulse survey can also provide a cost effective way to continually monitor the health of the organization over time by reducing the cost of an organizational-wide research project or large scale survey. The focus of this article is to help organizational leaders decide if and when a pulse survey is appropriate for their organization. This is not to say that pulse surveys should replace large scale survey projects yet used in conjunction with organization-wide surveys to continually monitor the health of the organization.
Types of Pulse Surveys To determine whether or not a pulse survey is right for your organization, consider the following options. Keep in mind that a pulse survey is not intended to replace an ongoing survey process, such as an annual or biennial employee survey.
Snapshot A “snapshot” pulse survey provides a glimpse or picture in time of the opinions, thoughts or ideas of a given employee population. Using demographic or historical lenses, managers can gain insight into trends, differences or similarities among and between groups. A “snapshot” pulse survey typically provides a single, quick measure to aid in decision making processes.
Performance Indicators Many organizations set target goals for performance indicators. One organization, for example, uses an index score, a subset of questions, to measure improvement in employee engagement. This index score and improvement over time may impact executive compensation and bonus decisions. A “performance indicator” pulse is administered in between the full employee survey administration cycles to track movement on the key performance indicator. Leaders can then make knowledgeable changes to their action plans that will positively impact the final score. Another client organization tracks performance through the use of a quality improvement metric similar to Six Sigma. A “performance indicator” pulse is administered as part of their evaluation process so that employee survey results are accurate and up-to-date. The “performance indicator” pulse provides a single metric for the employee survey on the work group’s score card.
Follow up Survey One of the most common uses for a pulse survey is a follow up to a larger, annual or biennial survey process. Many organizations conduct employee surveys on a regular basis such as once a year or once every two years. The “follow up” pulse survey can provide additional information to specific actions taken as a result of the annual survey and can help organizational leaders determine if progress is being made on employees’ action plans. The “follow up” pulse surveys serve as an interim check and allow leaders to change or redirect action planning efforts to better align with business outcomes. Instead of waiting a year or two to see if improvements have occurred, the “follow up” pulse survey provides a more immediate answer and allows for changes to be made along the way. Often questions asked in a “follow up” pulse survey are about the process of action planning – Did you participate in a feedback session? Did you participate in an employee action team? Questions can also be focused on improvement areas the organization has been addressing (i.e. training, team building, leadership development, etc.).
Audience or Topic Specific “Audience or topic specific” pulse surveys focus on specific questions or topics and/or specific audiences where information is needed. For example, the Marketing department may wish to survey customers who are recipients of a new
2
product to determine their overall satisfaction. The Quality department may wish to survey suppliers to determine their overall understanding of a new process that was recently implemented. In either case, both the audience and the content of the survey focus on a specific area of interest.
Sample Survey A “sample” pulse survey could include a sampling of questions from a larger survey which is administered to all employees or customers or it can be a full survey administered to a sample of employees or customers. A “sample” pulse survey is designed to gather information on a select set of items or with a select set of people. One client organization conducts an employee survey of all employees every two years. During the “off year” a sample of questions, usually those determined to be key drivers of engagement, are administered to all employees. This shorter survey is administered via the internet and is an early indicator of the movement in next year’s survey results. Like a “follow up” survey, the “sample” pulse survey can provide an early indication of action planning efforts.
Frequency or Timing We check our own pulse on a regular basis. Every time you go to the doctor, even for routine visits, your blood pressure, pulse and heart rate are measured to make sure all systems are functioning properly. Changes in your pulse rate may indicate medical issues that need be addressed or issues that could lead to future problems. How often a “pulse” survey should be conducted can best be answered by understanding what you hope to accomplish from the information gathered. The frequency and timing are often driven by the urgency in which information is needed.
After a Company-wide Survey Most company-wide employee surveys are conducted every year or every other year. Once the results are rolled out and action plans put in place, managers may want to know if their action planning efforts have made a difference before the next survey administration period. Pulse surveys for the purpose of following up after a company-wide survey can be administered six to twelve months after action plans have been implemented. This allows ample time to measure the impact of changes, improvements and new policies.
In “Off” Year As explained above, many company-wide employee surveys are conducted on a periodic basis rather than every year. During the “off” year, many organizations conduct a pulse survey to track improvement efforts and to gain an early indication of whether or not survey results will improve during the next employee survey cycle. The “off” year pulse tells you whether your pulse is beating erratically or strong. All of the standard or core survey questions can be administered to a sample of employees or a sample of the core survey questions can be administered to all employees in an “off” year pulse survey. Both methods provide a mid-cycle indication of the movement of survey results.
After Action Plans Have Been Implemented Organizational survey programs typically follow a similar cycle – a survey is developed, data are collected, results analyzed, reports produced and distributed, interpretation of data takes place and action plans are implemented. Organizations usually work through the entire process. Even with well-intentioned action planning processes, organizations fail to implement changes for a variety of reasons. One of the reasons cited (J. Waclawski and A. Church) is a lack of formal mechanisms to track performance and improvement. Rarely do organizations answer the question, “How well are we doing?” until the next survey administration rolls around. Pulse surveys conducted after action plans have been implemented measure the effectiveness of action planning efforts. Based on data collected, managers can keep moving ahead with plans or make adjustments to support improvements efforts.
Types of Questions to Ask The types of questions included in a pulse survey should support your reason for conducting one in the first place. Ask yourself, “What do I want to know?”
Follow-up or Clarification Questions Follow up or clarification questions can provide additional information that was not included in the organizational-wide survey or was uncovered as a result of feedback sessions and focus groups. A client organization includes questions about the action planning process in their pulse survey as a way to measure the level of involvement by employees and work groups across the globe. Since employee involvement is a key component of their quality improvement process, organizational leaders want to make sure all employees are involved in follow up action planning activities.
3
Another client organization administered a pulse survey after focus groups uncovered challenges with communications plans. Follow up questions were developed to directly address the communications issues so that appropriate steps could be implemented to remedy the problems identified during the focus groups.
Key Drivers Questions Key driver analyses using statistical techniques are often conducted after a large, organizational-wide survey to determine the focus for action plans. Key driver questions generally account for the largest percentage of the variance in overall scores and help managers understand the impact of improvement efforts. For example, if scores on key driver questions improve, then the overall score should improve the next time the survey is administered (if all other things remain equal). A key driver analyses can pinpoint specific focus areas and help shape action plans. A pulse survey that contains key driver questions can be an early indicator of improvement (or decline). Importantly, key driver pulse survey results provide an indicator of the movement in survey scores yet not the exact amount of increase or decline. Since the pulse survey is limited to a few key questions, an organization cannot determine what the actual amount of increase or decline will be until the next full survey is administered.
Bottom 5 or Targeted Questions Pulse survey questions can focus on the bottom five scoring questions from a large, organizational-wide survey or can focus on specific target areas identified as action items from further analysis of results. One client organization conducted a pulse survey of their five lowest scoring questions, which were the focus areas for company-wide action planning. The pulse survey was administered to all employees on a quarterly basis for over a year until the next full survey was administered. Overall scores significantly increased over time moving the selected items from the bottom of the list to a more favorable position. This supports the adage that what you focus on and measure usually improves.
Open-ended Questions Up until this point we have described pulse surveys from a quantitative point of view. However, pulse surveys can also contain open-ended questions to gather additional feedback. Whether you need clarification about a particular issue, need feedback from customers or employees or want to explore areas that were not included in the organizationalwide survey, a pulse survey that includes open-ended questions can provide additional information to managers and organizational leaders. One client organization provides a list of topics or dimensions to focus employees’ responses. Employees select a topic or dimension for every comment they enter. The comments are grouped and reported by topic or “theme”. This gives managers the ability to read and analyze comments from a thematic perspective.
Results and Reporting High-level Results Pulse survey results can be reported in many different ways depending on the type of survey you’ve selected. A pulse survey that has been administered to a sample of employees to assess the impact of company-wide action plans may require only a few high level reports to understand the impact of the action plans. A pulse survey administered to a stratified sample of employees across major regions or locations may require reports for every region or location so that local managers get the information they need to make appropriate business decisions. Typically, fewer report cuts are created for pulse surveys due to the limited number of questions asked and potentially limited number of people included in the sample. Few Demographic “Cuts” Pulse surveys often include few if any demographic questions. Unless you are collecting demographic data directly from an HRIS file, you may find it difficult to ask numerous demographic questions on the survey. As a general rule of thumb in organizational research, you should not ask more demographic questions than you ask closed-ended or open-ended questions. Therefore, if you limit the number of questions in a pulse survey to five, ten or even 20, then you shouldn’t be asking more than a few demographic questions. If you do want to include some demographic data for reporting purposes, take a look at what demographic cuts will give you the most valuable information for what you are trying to achieve. For example, you administer a pulse survey to employees that includes questions about the effectiveness of a recently-implemented management development program; you may want to include a position demographic. Crossing the data by this demographic may help you understand differences between employees, managers and executives regarding this important initiative.
4
Trends in Pulse Surveys With the wide-spread use of internet-based technology, pulse surveys are becoming more common as a tool to collect data that drives business decisions. Pulse surveys help you see if your action plans are on target. Pulse surveys help you refocus and regroup as an organization once decisions have been implemented. Pulse surveys can be targeted to reach specific groups of people at the right time with the right questions, and pulse surveys provide a cost effective alternative when the need for information is critical but budgets are tight.
About the Author Sharon Parker is a Senior Consultant with TNS Employee Insights. She has worked in the area of Human Resources Development over the past twenty years specializing in the areas of employee selection, training, coaching and leadership development to improve customer service, sales performance and inventory management systems. Sharon is currently working with clients worldwide to develop and implement organizational surveys and leadership assessments to create improvement strategies that measure progress. She has coauthored two books on web-based survey development and action planning. Sharon holds a Masters Degree in Industrial/ Organizational Psychology from Elmhurst College and is an adjunct professor at the college.
Contact Information TNS Employee Insights 65 Oakwood Road Lake Zurich, IL 60047 (847)726-4040 www.tnsemployeeinsights.com
References & Recommended Reading • Shannon D. & Bradshaw, C. (2002). A comparison of response rate, response time, and costs of mail and electronic surveys. The Journal of Experimental Education, 70(2), 179-187. • Fox, R.J. & Crask, M.R., Jonghoon, K. (1988). Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly. Vol 52(4), 467-491. • Parker, S.F., Schroeder, M.J., & Fairfield-Sonn, J.W. (2002). Action Planning: How to Follow Up on Survey Results to Implement Improvement Strategies. Engagement Press. • Kraut, A.I. (1996). Organizational Surveys: Tools for Assessment and Change. Jossey-Bass Inc.
©2011 TNS Employee Insights TNS is a trademark of Taylor Nelson Sofres, a Kantar Group Company
TNS_2146-11AR-PulseSurveys
5
