Practice Management Network

A best practice guide to using real-time patient feedback

In partnership with

Supported by

Introduction

With an increasing focus on a ‘patient-led’ NHS, responding to the needs and wishes of patients is central to the work of every GP practice – whether you’re designing new services, changing existing ones or simply evaluating your own performance. Gathering real-time feedback when patients visit your practice allows you to listen and respond to their views quickly and effectively. About this guide This best practice guide to using real-time patient feedback is informed by a pilot study carried out across a diverse range of 22 GP practices in England in 2009–10. The study collected useful learning from practice managers and GPs on how to get the most out of real-time patient feedback. You can use this guide to help you consider: • the business case for using real-time patient feedback; • what you want to find out from your patients; and • the best way to carry out your feedback process. The guide doesn’t set out one definitive method for using real-time patient feedback, but it will help you think about the choices you need to make to: • choose a survey method that is suited to your practice; • gather useful information you can deliver improvement against; and • get genuine value from your investment in real-time feedback equipment. This best practice guide builds on the original step-by-step advice on patient feedback in section 8.3 of the Practice Management Network’s Improving access, responding to patients: a ‘how-to’ guide for GP practices and should be read alongside it.

http://www.practicemanagement.org.uk/introduction

Improving access, responding to patients

Real-time patient feedback – what it is and how to use it Real-time feedback technology is located in your GP practice (usually in the patient waiting area) and collects patients’ views at the point at which they are accessing your services – for example, while they are waiting to be seen by their doctor or nurse or after they have been seen. How to use real-time feedback You can use real-time feedback to ask patients about their: • experiences when visiting your practice; • understanding of the services you offer; and • opinions on changes you may have recently introduced or plan to make. You can act on this feedback to become more responsive to patients’ needs:

GP practices asked patients about a range of issues: • The Hathaway Medical Centre (Wiltshire PCT) asked patients how easy they found the automated telephone booking system to use when arranging their appointment. • The Hema Medical Centre (Derby City) asked patients how friendly and approachable they found reception staff during their visit to the practice.

‘With a good feedback system, you can increase your understanding of what patients think about your practice, understand areas of concern and take action to transform the experience for patients. You can make changes and use the system to monitor patient reaction, gradually improving the practice based on accurate feedback, not guesswork.’

• The Dipton Surgery (County Durham) asked patients how far in advance they would like to be able to book an appointment with a doctor or nurse.

Practice Management Network, Improving access, responding to patients: a ‘how-to’ guide for GP practices

• The Bingley Medical Practice (Bradford and Airedale) asked patients about their experiences of having consultations with their doctor by phone, and whether they would be prepared to try this again in future.

A ‘how-to’ guide for GP practices

• The Wellington Health Centre (Westminster) asked patients if they were aware that they could order repeat prescriptions online as well as in person.

BP 01

Real-time patient feedback – what it is and how to use it continued Building the business case It’s important to use real-time patient feedback as a tool to improve or monitor performance in your practice. Before you get started, you should have a clear idea of why you are using it, how much it will cost and what benefits you expect to see. Most of the GP practices taking part in the pilot study got value from using real-time patient feedback, and believed it had the potential to improve patient experience. What could the benefits be? Think about what you want to get out of using real-time patient feedback. It can help you: • improve how you engage with and respond to your patients; • find out more about an area of your service that you know is currently under-performing; • monitor the quality of services you are delivering to patients over time; • assess patient reactions to changes you have recently made or are planning on introducing; • understand more about how your patients currently use your services, and services they might like to use in future; and • raise awareness of changes at the practice.

BP 02

How much might it cost? Exact costs will vary depending on the type of survey method you choose and the level of support you will need. As a minimum, you will need to: • buy or lease your electronic survey equipment and be trained on how to use it; • work with your equipment supplier to agree the areas you will investigate and the focus of your questions; and • agree with your supplier how you will access and analyse your patient feedback – suppliers may offer a range of analysis tools. For an individual practice, using a single, basic feedback device, the cost of using realtime patient feedback is likely to start from approximately £1,500–£2,500 per year. Practices that need a number of devices or a higher level of support from their supplier to design questions and analyse patients’ responses will need to plan for higher costs. When you meet with potential suppliers, it’s a good idea to discuss your exact real-time feedback requirements and look at what these may cost. It is therefore recommended that GP practices interested in using real-time patient feedback work with other practices and their local GP consortium to develop their business case. Joint investment at consortium level could allow costs and equipment to be shared across several practices and significantly reduce costs.

Improving access, responding to patients

Will real-time patient feedback be suitable for my practice? The pilot study found that all 22 practices were able to manage the requirements of setting up and running real-time patient feedback. The study also identified a number of positive factors that could help staff and patients respond well to the introduction of a feedback project. It may be helpful to ask the following questions before you start: • Do you have the support of your GP partners? This can have a positive impact on how the project is regarded by patients and staff. • Do you have a dedicated lead for real-time patient feedback? Having a lead person who can manage feedback as part of their regular duties will help to assign clear responsibility for the work. • Do you have sufficient staff resource to ask patients for feedback? It’s important to consider whether your staff will have the time to ask patients regularly for feedback, especially if your practice gets very busy. This may influence the type of equipment you choose. • Do you have the support of your Patient Participation Group or patient volunteers? Patients can be a particularly useful ally in talking to other patients and generating interest in feeding back. • Do you have a regular footfall of patients to survey? Real-time feedback is most valuable if you can collect a high number of responses. If your practice is very small, or your list size very limited, you may wish to look at other ways of getting patients’ views, such as paper surveys or face-to-face interviews.

A ‘how-to’ guide for GP practices

The Brook Medical Centre (Northamptonshire) identified early on that clinician support for real-time feedback was important. The practice manager spent time at the start of the project securing GP support by explaining that feedback would be very useful in helping the practice to improve and that the results would be available to GPs for analysis. The Christmas Maltings and Clements Practice (Suffolk) appointed an internal project manager to manage the relationship with their technology supplier, monitor the level of feedback collected and organise extra ‘pushes’ for feedback when required, analyse data and communicate regularly with patients and staff on progress. The Dipton Surgery (County Durham) found it challenging to attract consistently high levels of feedback. Its smaller practice list size (2,500) meant that staff were obliged to ask the same small sample of patients for feedback each time they visited the practice – in some cases, these patients eventually felt they were being over-surveyed. The practice concluded that, in its case, face-to-face communication might be a more effective way for patients to raise areas of concern.

BP 03

Choosing your electronic survey method and supplier Real-time feedback electronic survey technology comes in various forms. The pilot study looked at three methods: • handheld devices (tablet PCs or PDAs) that can be handed out to patients directly; • stand-alone kiosks that patients can use on their own; and • desktop devices that can be placed on your practice’s reception desk or in other locations around the waiting area. Other feedback methods available include SMS text messaging services and online surveys. What type of technology will suit you best?

You should also ask the following questions to help you decide your choice of equipment:

It’s important to select a method you know will work well with your patients, staff and the layout of your practice. Section 8.3 of Improving access, responding to patients: a ‘how-to’ guide for GP practices reviews the advantages and disadvantages of handheld and kiosk devices. In addition, the pilot study found that:

How many questions do you want to ask patients? Some types of equipment limit the number of questions you can ask. While there is no firm rule on the ideal number of questions, think about the impact the length of your survey may have on your patients and practice:

• handheld devices are useful for engaging directly with patients and can be handed out for them to complete, but more staff time may be needed to manage this process and supervise patients’ responses; • kiosks are lower maintenance and require less staff time, but their location needs to be carefully considered or they may end up being used less often and by fewer patients; and • desktop devices are secure and can be overseen by reception staff, but they may be hard for patients to use when the reception area is busy. Patients may also be concerned about the privacy of their responses if the equipment is close to where staff are working.

BP 04

• Asking fewer than ten questions, for example, will help to focus your thinking, be easier for patients to complete, and attract a higher number of complete responses. However, this limits how much you can find out about your patients. • Asking more than ten questions will allow you to collect more information about patients and give you more data. However, this may discourage patients by taking longer to complete and will require more time for your practice to analyse. • Choosing a survey method that allows you to ask more questions may help you to collect information such as age, gender and ethnicity. This could help you to understand experiences for different groups who attend your practice.

Improving access, responding to patients

Choosing your electronic survey method and supplier continued Some technologies also offer ‘free text’ sections that allow patients to provide detailed written answers. Reading through these can provide a useful context to patients’ responses, but it will take longer for your practice to analyse these comments fully.

The Derby PCT Practice (Bolton) found that it was useful to share free text answers at team meetings. The longer written responses provided valuable feedback and insights to reception staff in particular, as well as suggestions on services that patients would like to see in future.

What level of support will you need with question setting? Your practice may already be experienced in running patient surveys, or you may need help from a supplier in identifying effective questions that provide useful data. Some suppliers may have pre-existing question sets you can choose from, or you may want to select tried-andtested questions from previous national surveys. How often will you want to refresh your questions? Depending on what you are using feedback for, you may want to use a method that allows you to change your questions easily and cheaply.

What level of support will you need with analysing patient feedback? You may want to select a supplier that can help you with this process. How much help will you need with publicising the feedback project to your patients? Are you already experienced in communicating with patients, or will you need guidance from your supplier? Do your patients have any additional needs? You may need: • translations of your questions to make sure non-English speakers can respond; • large-print versions of your questions and/or easily accessible technology to ensure older patients can respond; and • devices incorporating voice technology that can be used by those with sight loss, language or reading difficulties. If so, it’s a good idea to discuss these issues with your supplier to identify who is best placed to provide these additional requirements, and to involve your patients to ensure that you provide the most appropriate solution.

How do you want your feedback represented? There are a number of ways in which your feedback can be represented, including visual summaries, graphs and statistics. You may also want to investigate how you will access data once it has been collected – some suppliers allow you to review feedback online at any point, and others may provide you with updates at regular intervals.

A ‘how-to’ guide for GP practices

BP 05

Choosing your electronic survey method and supplier continued

The Derby PCT Practice (Bolton) found that involving their patients was useful in developing effective translations of questions. The practice discovered by talking to Gujarati patients that only a small number of patients spoke the form of the Gujarati language that was being used in their current translation, and in many cases were using the English language version instead. This type of insight can help you ensure that all your patients can respond to your survey.

Remember that real-time patient feedback on its own may not be sufficient to capture the views of all your patients. You should also continue to offer alternative channels for people who may not be able to use or be comfortable with responding via electronic methods, such as patients who are unable to read or write.

Choosing a supplier Many companies offer real-time patient feedback electronic survey equipment and support. Some are listed on the Practice Management Network website’s suppliers section (http://www.practicemanagement. org.uk/supplier-listing). Different suppliers will offer a range of services from simple installation to complex reporting and analysis, so it is important to discuss and agree your requirements in advance.

BP 06

Improving access, responding to patients

The six-step real-time patient feedback process

Step 1 – Planning Step 2 – Method Step 3 – Collection Step 4 – Triangulation Step 5 – Action Step 6 – Evaluation

A ‘how-to’ guide for GP practices

BP 07

Step 1 – Planning ✔ Involve the right people When you are thinking about what you want to measure with real-time patient feedback, make sure you involve a broad range of patients and staff. This will ensure your questions genuinely reflect the issues facing your practice and respond to people’s concerns. Make sure you consult: • Patient Participation Groups (PPGs) and/ or patient volunteers – what insights do they already have into your patients’ concerns? • Patients in general – are there recurring issues that keep coming up in verbal feedback or written complaints? • Reception and administrative staff – what issues are they already aware of that are important to patients?

‘You need to have buy-in, not only from GPs, but from everybody.’ GP practice, East Midlands

The Chesterfield Drive Surgery (Suffolk) involved a wide variety of staff by discussing patient feedback at regular multi-disciplinary team meetings. This enabled a practice-wide forum of doctors, nurses and administrative staff to contribute to developing the questions.

The Hema Medical Centre (Derby City) found that consulting their PPG helped to focus their thinking on what they should ask patients. It emerged that PPG members’ understanding about the length of appointments and how long they could be booked in advance was lower than expected. The practice therefore included questions on these issues in their survey.

• Clinical staff – what important areas of work are coming up, and what topics do GPs and nurses most want to find out about? • Previous national and local surveys – are there areas where you know you are already under-performing and want to improve?

BP 08

Improving access, responding to patients

Step 1 – Planning ✔ Focus on areas you can change

✔ Keep it as simple as possible

Prioritise the areas you are going to look at. Trying to cover too many areas at once may leave you with too broad a range of questions and a large amount of data to analyse. Concentrate on the following:

You are more likely to benefit from real-time patient feedback if you focus on simple changes that can be made in a short amount of time, often at low or no cost – for example:

• Areas you know you can change – asking patients about car parking or your premises will not be useful if you know you cannot make any changes to these. It’s also best to avoid asking about issues that may be too expensive to change. • Areas where change can be made within a specific time – an important use of realtime feedback is to help you measure the effect of changes you make. If you make a significant change to how your practice is run, make sure your staff and patients understand how long it may take for this change to be made. • Areas where you don’t already know the answer – it’s more worthwhile to investigate areas where your knowledge about patient opinion is low, rather than confirming issues you already understand. However, real-time feedback can also be valuable in challenging assumptions you may have about your patients.

• making changes to how your appointments are organised or managed; • making practical changes to your waiting or reception area; or • improving the information provided to patients. Of course, you may also want to understand the impact of bigger changes such as introducing a new patient telephone system for booking appointments. If you plan to do this, it is important to work with your supplier to agree a realistic period of time to allow for collecting feedback after the change is made, in order to: • allow patients time to react to the change; and • make sure the effect of any change you make is sustained.

The Lister House Surgery (Luton) found real-time feedback tested their existing theories about patients. Their feedback showed that most patients believed they could book appointments only 48 hours in advance, rather than one month in advance as the practice had been expecting.

A ‘how-to’ guide for GP practices

BP 09

Step 2 – Method

‘Come up with your questions and, without collecting any data, run these by patients to make sure there is no ambiguity in their meaning.’ GP practice, South West

✔ Pilot your questions

You can use this initial feedback to refine your questions if necessary.

It can be useful to pilot your questions with a small group of patients and staff before launching your survey. This will help you to find out if:

The Marple Cottage Surgery (Stockport) identified during their first cycle of real-time feedback that their patients were not clear exactly what phrases like ‘long-term problem’ and ‘chronic illness’ meant when these were referred to in survey questions. The practice concluded that it was important to ensure patients fully understood any medical or technical terms used in the survey.

• any of the questions you are asking are unclear or confusing; • any of the language you are using is not understood by patients; • you are asking your questions in a logical order; and • your survey can be completed in a reasonable time.

✔ Think about the choice of answers you will give patients Remember that with most electronic survey methods, you supply both questions and a range of answers to choose from. So as well as designing your questions carefully, think about the potential answers to each question. The choice of answers you offer to patients will affect what you can do with the information you get. For this reason, it’s a good idea to pilot the answers as well as the questions.

Remember, for some questions, it may be important to offer patients the choice to answer ‘None of these’ if they do not agree with any of the options they have been offered. Below are some types of answer you could offer:

Type of answer

Advantage

Disadvantage

Yes/No

Generates very simple data

You can’t probe for other possible responses or understand reasons why

Very good/Good/Fair/ Poor/Very poor

Allows you to quantify the extent of patients’ experience/understanding

Subjective, and can be harder for practices to interpret results

Set answers (e.g. 1 week/ 2 weeks/1 month)

Allows you to customise possible question responses

Does not allow patients to suggest alternative answers

Free text response

Provides rich data source, and allows patients to qualify their answers

Takes longer to analyse, and you may not be able to respond to this data

BP 10

Improving access, responding to patients

Step 2 – Method ✔ Seek final expert input before launching your survey Many practices in the pilot study found it useful to work with their supplier to ensure that their questions and answers were appropriate before beginning the survey. Getting expert input before you begin will help you to collect valuable information from the moment you launch your survey and avoid the need to amend your questions later, which may cost more.

A ‘how-to’ guide for GP practices

BP 11

Step 3 – Collection ✔ Think about how the technology you choose will work in your practice To be truly effective, any survey method you use will have to work well with your practice layout. It may help to think about the following: • Where you will put your survey equipment. Your equipment should ideally be positioned in an area with high patient footfall (e.g. your waiting area, entrances and exits), but which also offers patients privacy to complete their answers. Identifying the right location will help increase the amount of feedback you collect. • How the equipment will link with your existing facilities. Will it be easy to locate your equipment close to existing power points and, if necessary, your internet connection? How will you position it in relation to other devices, such as automatic touch-screens or electronic display screens? • How you will make your equipment secure. Can you offer confidentiality to patients while also positioning the equipment where staff can supervise its use? • Whether you will need more than one device. If you have a large practice with high patient footfall and multiple exits and entrances, you may need to think about whether you need and can afford to have more than one device.

BP 12

‘You need somebody on reception proactively asking patients for feedback on their way in and out.’ GP practice, West Midlands

The Kildonan House Group Surgery (Bolton) found that placing the equipment on their reception desk was not an ideal solution for their practice – the desk and staff often became busy, which could discourage patients from giving feedback. The practice concluded that it is helpful to test out in advance where you might locate your equipment, to make sure patients are not obstructed and feel able to feed back comfortably and confidentially.

✔ Think about when you will ask your patients for feedback The time at which patients are approached for feedback can have a big impact on the number of responses you collect. In the pilot study: • Practices that asked patients for feedback before they were seen by a doctor or nurse often found it easier to get a response. Patients may be more open to being approached at this stage and often have time to give feedback while they are waiting to be seen. • Practices that asked patients for feedback after their appointment often reported that it was comparatively hard to attract a response. Patients may be in a rush to leave or preoccupied with their appointment, or it may be harder for receptionists to catch patients on their way out of the practice.

Improving access, responding to patients

Step 3 – Collection ✔ Think about when you will ask your patients for feedback continued The Marple Cottage Surgery (Stockport) discovered that surveying patients after their consultation can affect how easy it is to gather feedback. Because of their building’s multiple exits, staff had to position posters carefully in consultation rooms, make sure that receptionists asked patients to feed back before they left and ask doctors and nurses to remind patients to complete the survey on their way out.

On balance, it may prove easier to ask patients for their feedback as they are waiting to be seen, particularly if you are asking them about their experience of making the appointment – for example, how easy was it to get through on the telephone and were they given a time that was convenient to them? If you are interested in patients’ entire experience, including the appointment itself, asking them to feed back after their consultation may be necessary.

✔ Think about who can help you to collect feedback Working with staff In most cases, reception staff will be responsible for inviting patients to feed back. It is important to involve reception staff early on to ensure they: • are clear about why you are using real-time patient feedback; • are comfortable asking patients for their feedback; and A ‘how-to’ guide for GP practices

• can answer any questions patients may have, such as how long the survey will take to complete or what their answers will be used for. You should also identify any additional training needs for reception staff before you launch your survey. Involving reception staff helps to give them ownership of real-time patient feedback, and can help sustain their enthusiasm and commitment. This in turn can have an impact on patients’ willingness to feed back. The Wellington Medical Practice (Telford and Wrekin) delivered a presentation to its reception staff at the start of its patient feedback project, and built discussion about it into regular reception staff team meetings. The practice subsequently found that staff were clear about what the feedback was for, proactive in collecting responses, and able to see the value in real-time feedback as a means of delivering what patients wanted.

Working with partners GPs and nurses can help by reminding patients to provide feedback at the end of their consultations. You may also have dedicated patient-facing staff who can help. The St Paul’s Way Medical Centre (Tower Hamlets) have a full-time patient liaison officer who works ‘front of desk’. The practice found it was extremely useful to have someone who could explain how the feedback would be used, answer patients’ questions and, if necessary, accompany patients to the feedback kiosk to ‘walk them through’ the feedback experience.

BP 13

Step 3 – Collection ✔ Think about who can help you to collect feedback continued Working with patients Patient Participation Groups (PPGs) and patient volunteers can be extremely helpful in: • generating initial patient interest in real-time patient feedback; • sustaining patient enthusiasm and attracting high levels of feedback; and • working alongside or in place of practice staff to speak to patients directly and ask them to provide feedback. If you already have a PPG, it may be a good idea to ask them to act as real-time patient feedback ‘champions’ on your behalf. If you do not have a PPG, you may want to think about other ways in which patients can work with your staff on the feedback process. The Windhill Green Medical Centre (Bradford and Airedale) asked for volunteers from their Patient Forum to lead on gathering patient feedback. The practice felt its patients would respond more openly to volunteers approaching them. Four volunteers worked in the practice waiting area in regular morning and afternoon shifts. They were positively received by patients, and reported that they got real personal value from taking part.

See www.growingppgs.com for more information on setting up a PPG.

✔ Think about how you will communicate with patients Practices that communicate directly with their patients about patient feedback are more likely to attract a high, consistent number of responses. The pilot study found that patients can be reluctant to take part if they are unsure about what real-time patient feedback is, unclear how their responses will be used for or unfamiliar with the technology. Communicating with patients could involve: • approaching patients in your waiting area to explain the project; • reassuring patients about any misgivings they may have – for example, making it clear that all feedback given is anonymous; • actively inviting patients to feed back, by directing them to the survey equipment and if necessary walking them through their first response; and/or • publicising your patient feedback system via posters or information screens in your practice, on your practice website, or via your telephone system, prescription system or newsletter. Remember that posters alone may not be enough to catch your patients’ attention – several practices in the pilot study found that: • notice boards can already be very crowded with other information and messages; and • patients may not take the initiative to give feedback unless they are prompted. It is best to use a combination of communication methods to attract your patients’ interest, and be prepared to experiment to get the best response.

BP 14

Improving access, responding to patients

Step 3 – Collection ✔ Think about how you will communicate with patients continued The Wellington Health Centre (Westminster) used a range of channels to communicate with patients about real-time feedback, including posters in waiting areas, information on the practice television screen and prescriptions, and updates in the practice newsletter and website. They also used verbal reminders by receptionists and doctors. The practice found that around half of patients who responded sought out the real-time feedback device on their own, rather than being actively approached to feed back.

✔ Think about the amount of feedback you want to see Real-time feedback is most useful if you manage to collect a substantial number of responses. This will provide you with more robust and meaningful data from which to work. While there is no single definition of a ‘good’ number of responses, it is important to think in advance about what level of feedback you would like to see. This could be informed by the following: • Looking at your patient list size. Based on the total number of patients your practice serves, what do you think would be a good proportion to survey each month? • Looking at the number of appointments you offer. Of these, how many patients would you like to see feeding back each week?

✔ Remind patients to keep feeding back Unlike traditional ‘one-off’ surveys that patients may be sent in the post or asked to complete on the street, real-time patient feedback allows you to gather continuous feedback from patients each time they visit your practice. This can help you to:

• Looking at your practice profile. Are there particular groups of patients you are interested in surveying, or which you have found more difficult to survey in the past? Your technology supplier may be able to advise you on appropriate response levels to aim for, based on your practice profile.

• monitor your practice’s performance over time; and

Setting informal targets for number of responses may help you manage your feedback more effectively, by:

• see whether changes you make have a positive influence on patients’ experience, awareness or opinion.

• helping to motivate staff and maintain their enthusiasm over time;

If you are using real-time feedback in this way, it is important to clarify to patients that you would like them to give feedback every time they visit your practice. The pilot study found that patients did not always realise that they could or should feed back on every visit. A ‘how-to’ guide for GP practices

• helping you to identify when a ‘push’ on feedback levels may be needed; • identifying suitable intervals at which you can analyse your feedback; and • giving you confidence that your decisions are based on robust data. BP 15

Step 4 – Triangulation

‘If patients can see the results of what they’re doing, it does make a difference and encourages them to keep feeding back.’ GP practice, Yorkshire and the Humber

By now you will have agreed your questions, communicated with your patients and started collecting feedback.

✔ Take a baseline reading and see what this tells you After you have collected what you or your supplier considers to be a good initial level of feedback, it is important that you measure your baseline position by reviewing the feedback so far. Your baseline reading will establish what patients thought or felt before you introduced any change – without measuring this, you will be unable to establish what effect your changes have had. This reading could be taken after you have been running real-time feedback for a while (one month, for example), or after you have collected a high enough number of responses. The most important thing is that you have enough information to make informed decisions. Your baseline reading will tell you: • what your patients think or understand about your services, and how this compares with your original assumptions; • areas where it may be appropriate to introduce change, based on what patients have told you; and • areas where your performance may already be strong – or better than you expected. If all your results are more positive than you expected, it may be appropriate to revise your questions – are there other areas you could look at?

BP 16

The Chesterfield Drive Surgery (Suffolk) found that patients were more satisfied than expected with the quality and clarity of the chronic disease review invitation letters the practice was sending out: nearly 85% of patients were happy with them. As a result, the practice modified their questions to look at other areas where their performance might not be as strong.

✔ Review your findings in the wider context When analysing your findings, it is also useful to consider what was going on in your practice at the time the responses were being gathered. This may offer insight into any unexpected results. The pilot study identified examples where local events could have influenced practices’ results. For instance: • staff absences in one practice meant some patients were unable to get appointments with the doctor of their choice, and could have faced longer waiting times; and • in another practice, adverse weather meant patient services were significantly reduced, which may have accounted for a lower number of patient responses and more mixed results. The purpose of triangulating your results is not to find excuses for your findings, but to make sure feedback is considered alongside other sources of information to ensure that your analysis is robust.

Improving access, responding to patients

Step 4 – Triangulation ✔ Review your findings in the wider context continued This will help you identify whether the findings you have collected so far can be acted upon, or whether you need to gather more feedback first in order to understand whether certain factors may have affected your results.

This is often referred to as a ‘you said, we did’ feedback cycle, in which patients can see a clear link between their responses and your subsequent actions. To achieve this, make sure that you publicise your initial findings clearly, via: • posters or electronic displays within your waiting area; • verbal updates to your Patient Participation Group or patient volunteers; and

✔ Get input from your supplier Even if you are already experienced in studying patient survey data or your results present clear findings you have no trouble interpreting, it may be valuable to involve your supplier during the analysis stage. Many of the practices taking part in the pilot study found suppliers offered extra expertise in reviewing the findings, and offered a different perspective on their own analysis of the results.

✔ Share your initial findings An important principle of real-time patient feedback is that patients can see what impact their feedback is having on your practice. This is so that patients:

• updates in your practice newsletter or on your website. Remember also to update your staff on your findings to date. Making your real-time feedback process as transparent as possible will help maintain the interest of your patients and the commitment of your staff. The Christmas Maltings and Clements Practice (Suffolk) engaged patients by creating a regular display of their findings, and subsequent actions, in their patient waiting area. This helped create good patient awareness of the impact that their feedback was having, and contributed in part to the practice achieving over 2,000 patient responses during the pilot.

• feel their feedback is valued and worthwhile; • see how their opinion can positively influence their local practice; and • remain interested, and are encouraged to keep feeding back.

A ‘how-to’ guide for GP practices

BP 17

Step 5 – Action

✔ Decide what changes you are going to make It may be helpful at this point to review the project and remind yourself what your original goals were in carrying out a patient feedback exercise. Your response to the findings you have collected so far will depend on what you are using patient feedback to achieve: • If you are using it to conduct market research or to find out patient attitudes to a change that you have already introduced, this first analysis stage may represent the end of your real-time feedback ‘cycle’. • If you already know the change you are planning to make, you can treat this initial data as your baseline or starting position before you introduce the change. • If you are using feedback to monitor your performance over time, and your results are not as strong as you would like, you may wish to continue to collect results for a longer period, in order to understand wider trends. • If you are using feedback to monitor your performance over time, and your results are positive, you may simply wish to keep monitoring your feedback in order to ensure that you sustain these results and can take prompt action if there is a noticeable decline.

‘It’s been able to give our practice some concrete evidence that we need to make some changes.’ GP practice, North West

If you are planning to introduce change based on what your feedback has told you: • make sure you prioritise the changes you are going to make – changing too many things too soon may become unmanageable; • keep your changes as simple as possible – these may be easier to implement in a shorter time, and may show quicker results; and • think about what improvements you want to see – having made your change, what response do you want to see from your patients, and can you set informal targets around this? Once you have decided on your changes, make sure your GPs, nurses and administrative staff understand what you intend to do. You may want to hold an action planning meeting with your staff to support this. The Park Medical Practice (Derby) found that patients wanted pre-bookable appointments at a range of times during the morning and afternoon – not just in the evenings as they first expected. The practice was able to offer a more flexible range of pre-bookable appointments at no extra cost, and hopes to see improvement in future patient satisfaction surveys as a result.

• As recommended, if you are using real-time feedback to achieve performance improvement you may now want to study your data in order to decide what changes you are going to make to your practice. You can then see what impact these changes have on future patient feedback.

BP 18

Improving access, responding to patients

Step 5 – Action

✔ Decide what changes you are going to make continued The Lawley Medical Practice (Telford and Wrekin) introduced simple changes to their reception layout, including a bell on the desk so that patients were able to attract the attention of practice staff. This straightforward change had a positive impact: the proportion of patients reporting that they were seen straight away by staff when they arrived increased by nearly a quarter after the change was made.

The Hema Medical Centre (Derby City) found from feedback that the most popular way of communicating with patients was via a newsletter, and they have since introduced a quarterly publication.

✔ Keep your patients informed about what you are doing Once you have decided what you are going to change, it may be tempting to postpone updating patients until you have implemented that change. However, it is important to keep communicating with patients, in order to maintain their interest and commitment to giving feedback. For example: • thanking patients for their feedback so far; • outlining what their feedback has helped to identify; and • explaining what action you are taking in response to this feedback. Showing patients that their feedback will help to improve their experience will help you collect more responses. Think about mechanisms that could help you to do this. These could include: • putting publicity materials and/or leaflets in your waiting area; • updating your Patient Participation Group on the progress being made; and • giving updates in your practice newsletter or on your website.

A ‘how-to’ guide for GP practices

BP 19

Step 6 – Evaluation ✔ Measure again to see what impact you have had If you have introduced a change since taking your baseline position, you need to identify when it is appropriate to look again at what impact this change has had on patient feedback. A positive improvement to look for might be increased patient satisfaction with, or awareness of, the services you are offering. It may be useful to work with your supplier to identify what constitutes the good level of improvement that your practice should be aiming for. The Cape Cornwall Surgery (Cornwall) saw a 20% increase in patients’ awareness that they could book a doctor’s appointment up to four weeks in advance, following extra communication about this service.

‘It’s helped us to look at how we work, and has helped to focus people’s thinking around how they can change and what needs to change.’ GP practice, East Midlands

The Marple Cottage Surgery (Stockport) found that patients’ awareness that they could speak with a receptionist privately if they wanted to talk about a confidential issue doubled following increased communication about this.

Once you have identified the impact of the change you have introduced, you may then decide to: • continue to measure feedback to see if this performance can be sustained or improved; or • turn your attention to measuring performance in other areas if you are satisfied with the level of improvement so far. If you have seen positive improvement since making your change, make sure that you share and celebrate this with patients and staff. You may also want to communicate your success to your local GP consortium.

The Sunnyside Medical Centre (Portsmouth) saw a 15% increase in patients’ awareness that they could book an appointment with a nurse up to four weeks in advance, following a publicity campaign in the practice’s waiting area and on its website.

The Derby PCT Practice (Bolton) saw patients’ perception that is ‘very easy’ to get through on the phone nearly double during the pilot period.

BP 20

Improving access, responding to patients

Step 6 – Evaluation

✔ Look for other positive evidence of the change you have made While your survey results tell you whether your change has had an impact on patients’ opinions, it is important to look for other evidence of the impact on your practice. For example: The Lister House Surgery (Derby City) reported a decline in ‘did not attend’ rates by approximately 20% as a result of providing better publicity on patient appointments.

The Wellington Health Centre (Westminster) saw increased take-up of online prescription services following extra communication about this option.

The Portsdown Group Practice (Portsmouth) saw increased take-up of its extended-hours service following clearer explanation about its opening hours.

Think in advance about other ways in which you can track the business impact your changes are having. Other ideas include the following: • Online tracking. If you are trying to increase use of your online services, can you measure this through increased website hits or take-up of online services? • Accessibility. If you are trying to make it easier for patients to get through to your practice, can you measure this by monitoring waiting times on the telephone service or the number of abandoned calls?

A ‘how-to’ guide for GP practices

Your supplier can help you scope any additional measures before beginning your survey.

✔ Refine and improve your process As outlined in section 8.3 (page 142) of the Practice Management Network’s Improving access, responding to patients: a ‘how-to’ guide for GP practices, real-time patient feedback represents a cycle of responsive general practice. Once you have measured, implemented change and measured again, it is important to consider your options for what to do next. You could: • continue with the same questions, aiming to improve or sustain your performance; • refresh your questions and focus on new areas or changes in your practice; or • discontinue real-time patient feedback if your practice feels that your original aims have been achieved. If you decide to continue the project, make sure that you conduct a short review at this stage to identify what worked well and what could have been done better. Use these findings to refine your process to make your feedback process even more effective next time. You may wish to speak to patients and staff to identify what their experience has been, and how this learning can be built into your next cycle of feedback.

BP 21

Acknowledgements

Many thanks to the following GP practices, which participated in the real-time patient feedback pilot between October 2009 and March 2010. They offered invaluable insights and learning from their experiences of using real-time patient feedback: • Bingley Medical Practice, Bradford and Airedale • Brook Medical Centre, Northamptonshire • Cape Cornwall Surgery, Cornwall • Chesterfield Drive Surgery, Suffolk • Christmas Maltings and Clements Practice, Suffolk • Derby PCT Practice, Bolton • Dipton Surgery, County Durham • Easton Family Practice, Bristol • Hathaway Medical Centre, Wiltshire • Hema Medical Centre, Derby City • Kildonan House Group Surgery, Bolton • Lawley Medical Practice, Telford and Wrekin • Lister House Surgery, Luton • Lister House Surgery, Derby City • Marple Cottage Surgery, Stockport • Park Medical Practice, Derby • Portsdown Group Practice, Portsmouth • St Paul’s Way Medical Centre, Tower Hamlets • Sunnyside Medical Centre, Portsmouth • Wellington Health Centre, Westminster • Wellington Medical Practice, Telford and Wrekin • Windhill Green Medical Centre, Bradford and Airedale Thanks should also go to the three technology suppliers that were involved in the pilot study and provided valued input to the end evaluation: • Customer Research Technology and Picker Institute Europe • Dr Foster Intelligence • Picker Institute Europe and Fr3dom Health Solutions

BP 22

Improving access, responding to patients