For Customer Experience Professionals January 24, 2012

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation by Vidya L. Drego with Ron Rogowski and Allison Stone

Executi v e S ummary Exploratory research uncovers customer needs that have yet to be met, exposes opportunities for differentiation, provides a holistic view of a problem space, and helps firms keep pace with the quickly changing technology landscape. PayPal, in collaboration with its user experience research partner AnswerLab, uses a class of techniques called innovation games to hone its vision for new products. The research teams’ techniques have dramatically improved product definition and requirements documentation while driving stakeholder understanding and engagement in customer experience. Exploratory research helps firms create differentiated experiences Innovative firms constantly look to differentiate themselves in the marketplace with new products and services.1 Increasingly, firms expect the customer experience to drive that differentiation (see Figure 1).2 Exploratory customer research techniques such as ethnography and contextual inquiry help with that effort because they (see Figure 2):

· Identify customer needs that have yet to be met. Because much of demand is latent, customers

can’t explicitly identify how companies should satisfy their needs. When they do have suggestions, their thinking about what is possible is often restricted by what they’ve experienced in the past. Exploratory research methods are based on observing people in a variety of situations and allow trained researchers to identify customer challenges and potential opportunity areas without customers having to articulate their needs.

· Expose opportunities for competitive differentiation. Companies that simply seek parity with

their competitors often base their products and services on what their competitors offer, not on what their customers actually need to achieve their real goals.3 Techniques that focus on how customers solve a problem, not simply on how they use an interface, provide insight into where current solutions in the marketplace do not sufficiently support customer goals.

· Provide a holistic view of a problem space. Most companies assume that when customers interact with their touchpoints, they are focused completely on the task at hand. The reality is that most customers are dealing with many distractions and often use multiple touchpoints to address their needs. Because ethnographic research methods are based on observing customers in their natural environments, they can help uncover how the users’ context affects their experiences. For example,

Headquarters Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA, 02140 USA Tel: +1 617.613.6000 • Fax: +1 617.613.5000 • www.forrester.com

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

2

For Customer Experience Professionals

a bank customer checking a balance may get distracted by a ringing phone or face other challenges like lack of privacy when using a smartphone — needs that must be reflected in how the experience is designed to ensure that the task is both useful and usable across environments.

· Highlight emerging uses for new or rapidly changing technologies. It’s difficult to predict

how customers will adopt new technologies and device features without knowing their context for use. Exploratory research techniques help find gaps that new technologies might fill. For example, when smartphone apps were newly introduced, most firms didn’t yet know how they would be adopted. In the absence of usage data, firms interested in designing the right solution for customers used exploratory research to study common tasks, identified ones that might be useful from a mobile device, and even observed consumers using prototypes to evaluate and improve their usability.

Figure 1 Firms Want To Differentiate On Their Customer Experience “How would you describe your executive team’s goal for customer experience?” To differentiate ourselves from all firms across any industry

13%

To differentiate ourselves from competitors in our industry

63%

To maintain parity with leaders in our industry

11%

To stay in the mainstream in our industry

12%

To keep from falling too far behind leaders in our industry 1%

76% want to differentiate with customer experience.

To stay slightly behind the mainstream in our industry 1% Base: 103 customer experience professionals at firms whose executives have customer experience goals (percentages do not total 100 because of rounding) Source: February 17, 2011, “The State Of Customer Experience, 2011” Forrester report 59110

January 24, 2012

Source: Forrester Research, Inc.

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

3

For Customer Experience Professionals

Figure 2 Exploratory Research Helps Uncover Opportunities To Differentiate Exploratory research Definition Broad, open-ended qualitative insights typically used to develop a concept; uncovers what people do

Evolutionary research More focused and directed than exploratory research; balances qualitative and quantitative insights to validate and flesh out a concept; explores what people do (or say they do) with existing tools in a specific domain and what they say about the experience

Research Ethnography, interviews, Interviews, participatory techniques observation, contextual inquiry design exercises Typical Journey maps, personas, outputs mood boards

Personas, scenarios, paper prototypes, wire frames

Evaluative research Most focused of the three approaches; uses qualitative and quantitative methods to test a concept in use; tests what people do with a specific interface and how they react to it Analytics, heuristic evaluations, usability tests (remote or in person), emotional feedback techniques Usage dashboards, reports, wire frames

Source: December 20, 2010, “How To Craft Your Customer Research Plan” Forrester report 59110

Source: Forrester Research, Inc.

PayPal’s Innovation Games Encourage Design Thinking Across The Enterprise PayPal operates in an increasingly crowded competitive landscape with a vision to “create a better shopping experience from start to finish, no matter what’s being bought, how it’s being paid for, or where it’s being sold.”4 We spoke with Kuldeep Kelkar, the head of global research for PayPal, and Liang Zhang, user experience researcher from AnswerLab, to understand how the two teams, led by Beverly Freeman of PayPal, employ ethnographic research to help PayPal bring to life its vision. The teams use innovation games, a technique where research participants engage in a set of directed games to help PayPal:5

· Hone its vision with designers, product managers, and stakeholders. Last year, PayPal

worked with AnswerLab to retrofit its usability lab to look like the inside of a coffee shop to test a new mobile payment process (see Figure 3). Researchers tested different scenarios, varying some part of the payment process, like allowing users to pay in advance via their mobile phone for an item or asking subjects to imagine that they were running late and trying out the new payment experience. This allowed them not only to understand whether someone would say that she would use some functionality but also to model how she might use it. For example, one user forgot that he had paid in advance and stood patiently in line. When he was reminded that he could skip ahead since he had already paid, he felt badly about jumping ahead of others in line and recommended creating a separate line for people who had paid ahead so that customers wouldn’t have to make the perceived social faux pas of cutting ahead in line.

January 24, 2012

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

4

For Customer Experience Professionals

· Expose mental models that govern real-world experiences. AnswerLab then conducted an

exercise where it asked research subjects to organize the contents of their wallets and purses, grouping like items and talking about their existing organization scheme. The research team was interested in how subjects’ organization schemes might change if the information on their cards was stored on their phone, so they went through these sorting exercises with both physical artifacts and digital accounts and profiles. This helped reveal the existing mental models that governed how users thought about their wallets and allowed researchers to explore how this might change as the concept of a wallet moves into the digital realm.

· Co-create useful solutions to today’s pain points. Finally, the AnswerLab team had

participants keep a shopping diary over a period of two weeks so that researchers could understand different types of shopping events (routine versus non-routine) and participants could note down key details of these events. It then had all of the participants bring in their diaries and work in small groups to design an assistant (either a person or a tool) that would help them with the various subtasks related to shopping. It found that research subjects were under a lot of stress even on routine shopping trips. Several groups wanted an assistant to help them be mindful of their diet while compiling a shopping list; remember to bring their coupons, loyalty cards, and shopping bags to the store; pick out the freshest produce; bag the groceries; carry the items home; organize them in the pantry; and even remind them to use the food before it expired. This helped the PayPal team understand all of the factors and considerations that go into shopping beyond just making a payment.

January 24, 2012

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

5

For Customer Experience Professionals

Figure 3 PayPal Retrofitted Its Usability Lab To Resemble A Coffee Shop

PayPal’s research team turned its usability lab into a makeshift coffee shop and asked research candidates to complete common tasks.

Source: PayPal and AnswerLab 59110

January 24, 2012

Source: Forrester Research, Inc.

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

6

For Customer Experience Professionals

PayPal’s Exploratory Exercises Encourage Collaboration For each innovation game AnswerLab organized, the PayPal research team invited a diverse group of business stakeholders including executives, product managers, and marketers to the lab to observe. After each session, observers participated in a 60- to 90-minute debriefing session with the research team. The room in which they debriefed was optimized for the observers; it was covered with whiteboards and empty walls to help encourage brainstorming and capture ideas (see Figure 4). As a result of interacting immediately after the research session, as opposed to waiting for the PowerPoint report to be released:

· Product managers wrote more holistic requirements. After observing the exploratory

research and engaging in post-research ideation sessions, product managers better understood how interaction points beyond the mobile device affected users — insights that they reflected in much more detailed requirements documentation. Their requirements now take into account the entire customer journey and refer to other touchpoints like emails, text messages, or other alerts that the customer might receive outside of the mobile app.

· Executives understood how much the experience of a new idea matters. As a result of their

participation in the research, leaders from product management to engineering now understand how subtle changes in the user’s context can lead to dramatically different experiences. This insight is driving continued funding of customer research to ensure that PayPal designs and develops solutions that are not only innovative but also useful and usable for its customers. Teams in the budgeting process have already earmarked funds for the research team to conduct more sessions about the products it’s developing.

January 24, 2012

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

7

For Customer Experience Professionals

Figure 4 Stakeholders Observe The Research And Participate In Brainstorming And Ideation

Stakeholders view streaming video of research tasks in a room equipped with whiteboards to discuss and share what they see.

Source: PayPal and AnswerLab 59110

January 24, 2012

Source: Forrester Research, Inc.

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

8

For Customer Experience Professionals

R ecomme n da t io n s

drive innovation With exploratory exercises Not all firms need to design elaborate innovation games to achieve results on par with PayPal’s. In reality, innovation games can be fairly simple to set up — the purpose is to get subjects out of the mindset of merely providing feedback and into the mindset of the task. Regardless of the specific technique used, diverse roles need to consider the experience from the customer’s point of view and use that knowledge to make customer experience improvements. To achieve their own innovation successes, customer experience professionals must:

· Define what the organization hopes to get from the research. PayPal partnered with user experience research firm AnswerLab to help it design exploratory research studies that shed light on a new concept. This helped it get beyond what customers said that they would do to what they actually did in as close to real-world situations as possible and allowed stakeholders to observe these behaviors firsthand. Firms should begin with their organizations’ objectives first in order to define the scale of the research needed and the questions that need to be answered. For example, if your firm is trying to design a new product, it’s worth understanding how satisfied people are with the solutions available to them today.

· Select the techniques that can best answer the questions. Exploratory research techniques come in a variety of flavors, and not every technique provides the insight needed to shed light on every question. When considering a research technique like contextual interviews, observation, ride-alongs, or diary studies, customer experience pros should not only consider the technique but also the context of the user, the depth of feedback needed, and the sophistication of the concept they’re testing.6 The circumstances around how and when a technique is used, and how successfully it was used, are as important as the technique itself.

· Consider techniques that encourage contextual feedback. Innovation games are uniquely valuable because, unlike techniques that directly ask users for their ideas or feedback, games distract them from designing the product and allow them to focus on providing rich contextual reaction to a proposed design. In addition, these games keep users engaged and interested in the research process, relieve the pressure from having to come up with good ideas, and generate a wealth of artifacts. The technique works extremely well when soliciting new ideas or reaction to new concepts that have yet to be developed.

· Immerse stakeholders in the research process. Customer experience professionals should invite stakeholders from other parts of the organization to participate in the research and, more importantly, the debrief process. Why? Because observing customers in context has a powerful effect. It creates meaning for the work beyond just meeting internal deadlines or monetary incentives. It also allows stakeholders to understand a customer problem holistically and encourages them to take initiative to help solve that problem.7 January 24, 2012

© 2012, Forrester Research, Inc. Reproduction Prohibited

How PayPal Uses Exploratory Research To Drive A Culture Of Customer-Focused Innovation

9

For Customer Experience Professionals

Endnotes 1

The very nature of competition has changed in the age of the customer. Amid this challenging new world order, some companies have found ways to compete successfully based on innovative customer experience strategies that guide the delivery of differentiated value propositions. Forrester uncovered nine such strategies that align with Michael Porter’s three generic company strategies. They illustrate some of the many ways firms orchestrate activities to amplify their competitive positions as cost leaders, differentiators, or segmentors. See the August 22, 2011, “Innovative Customer Experience Strategies” report.

2

To assess the state of customer experience in 2011, Forrester surveyed 118 customer experience professionals around the globe. More than two-thirds said that their firms aim to differentiate based on customer experience. But most companies are ill-prepared to compete on that basis. See the February 17, 2011, “The State Of Customer Experience, 2011” report.

3

Companies hoping to engage their customers in digital channels should adopt what Forrester calls Emotional Experience Design (EED). The first principle of the framework asserts that firms need to address their customers’ real goals by continually uncovering latent needs, helping people make good decisions, answering questions before users ask them, and blending experiences across relevant channels. Successful firms build disruptive experiences, provide complementary tools, and completely reframe existing interaction models based on insights gained through exploratory testing techniques. To lay a strong foundation for emotional engagement, customer experience professionals should foster a culture of empathy, use exploratory research techniques as a springboard for creating concepts, and use a combination of tools to document user sentiment. See the January 18, 2011, “Mastering Emotional Experience Design: Address Customers’ Real Goals” report.

4

Read more about PayPal’s vision on its website. Source: PayPal (https://www.paypal-media.com/payments/ vision).

5

Innovation games were introduced by Luke Hohmann in his book, Innovation Games: Creating Breakthrough Products Through Collaborative Play. Source: Luke Hohmann, Innovation Games: Creating Breakthrough Products Through Collaborative Play, Addison-Wesley Professional, 2006.

6

At Forrester’s 2011 Customer Experience Forum, panelists from Bentley Design and Usability Center, frog design, and Wells Fargo shared their experiences using exploratory research techniques. While these firms have long used qualitative ethnographic methods to understand how customers actually behave, what motivates and drives them, as well as what opportunities exist to better meet their needs, these firms have realized additional benefits from their exploratory research. See the August 22, 2011, “What Three Companies Learned From Conducting Ethnographic Research” report.

7

In the book Drive, author Daniel Pink argues that the secret to high performance and satisfaction is the deeply human need to direct our own lives, to learn and create new things, and to do better by ourselves and our world. Helping stakeholders create new (but relevant) products and solutions to solve a customer pain point can be one way to boost both performance and satisfaction at work. Source: Daniel Pink, Drive, Penguin Audio, 2010.

Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology. Forrester works with professionals in 19 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 28 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit www.forrester.com. © 2012 Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, RoleView, Technographics, TechRankings, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective owners. Reproduction or sharing of this content in any form without prior written permission is strictly prohibited. To purchase reprints of this document, please email [email protected]. For additional reproduction and usage information, see Forrester’s Citation Policy located at www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. 59110