Mapping Marketing Decision Space: An Exploration Of Emerging Cognitive Mapping Tools For Analysis Of Marketing Scenarios And Decision-Making Hugh M. Pattinson, University of Technology, Sydney Abstract The paper reports on the author’s experiences with a range of applications for analysing and mapping unstructured data from two research projects connected to exploring marketing decision-making and future marketing scenarios. A mix of researcher judgement and semiautomatic applications is reviewed with discussion on that application or rejection in two research projects. The paper concludes that researcher judgement and mapping still dominate analysis of unstructured data, but new and emerging applications are at last providing valuable additional insights and inputs to effectively augment initial analysis. New semantic web and Java-based analytical applications are likely to greatly enrich researcher judgement analysis but the hope and vision of the founder of the World-Wide Web is that a new and emerging semantic web will indeed enable advanced analysis of unstructured information at a level much closer to that of human storytelling and memory application. Keywords: applications, unstructured information, researcher judgement, qualitative research, cognitive mapping, marketing decision-making, storytelling, memory

Introduction and Overview The purpose of this paper is to explore and discuss a range of new and emerging cognitive mapping and qualitative analytical tools, based on experience drawn from recent research projects undertaken by the author. The first research project focused on analysing and cognitively mapping marketing decision-making associated with research, development and delivery of new Internet-based applications (Pattinson, 2005), while the second project focused on analysis of interviews and secondary data associated with development of future scenarios for the Television Industry in Australia out to the year 2015 (Lloyd-Jones, Chandler and Pattinson, 2005). For both research projects, our main objective was to effectively analyse unstructured data and to produce clear visualisations, or useful cognitive maps of key themes and associated concepts – and then to produce strong interpretative research findings out of them.

Literature Review Understanding deeper insights and nuances associated decision-making through mapping of strategic thought was articulated by Huff (1990) and Weick (1990), and noted as an effective form of sensemaking by Weick (1995 and 2001). However, up to the mid-1990’s, actual analysis of decision-making accounts and stories, and development of maps and visualisations was mainly a manual set of processes. The researcher used their own judgement (judgement analysis) to identify key themes, issues and items – and then to manually attempt to develop associations of concepts. The researcher would then attempt to manually map selected concepts with linkages. Over the last ten years, several software applications have been released that can be used to analyse and map accounts of decision-making and key themes and issues associated with ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

23

them. Examples include NVivo, TACT, MindManager, NetMap and Leximancer Copernic Summarizer, and Grokker. These applications will be discussed in the context of evaluation and use in two research projects The first research project focused on analysing and cognitively mapping marketing decisionmaking associated with research, development and delivery of new Internet-based applications. The project was undertaken from 2001 to 2003. A substantial history of cognitive mapping of marketing strategies, activities, issues and decision-making decisions going back to the 1960’s exists, using Marketing Decision Systems Analysis (DSA). Early DSA mapping focused on computer flowcharting (systems flowcharting) of typical marketing decisions (see Cyert, Simon and Trow 1956; Morgenroth 1964; Hulbert, Farley and Howard 1972; Howard, Hulbert and Farley 1975 and revisited by Hulbert in Hulbert 2003; Mintzberg, Rasinghani and Theoret 1976). DSA mapping was enhanced and updated specifically for business-to-business (B2B) marketing contexts through the period from the mid-1970’s to 2000 (Capon and Hulbert 1975; Woodside and Samuel 1981; Vyas and Woodside 1984; Woodside 1994; Woodside and Wilson 2000). The project extended DSA through adding specific cognitive mapping based on reanalysing previous accounts of marketing decision-making through multiple rounds of interviewing (Langley et al., 1995). These rounds were supported by new forms of cognitive mapping, derived from knowledge on mapping strategic knowledge and strategic thought which in turn can be viewed as contributing to a cognitive theory of the firm. Cognitive mapping was used in the second research project to identify and link key themes, concepts and issues required for the development of scenarios for Television in Australia out to 2015, based on interpretation of accounts from media industry participants and observers. The pilot study undertaken in the first half of 2005 explored a range of applications and methodologies for mapping key themes and issues for input to scenario development. A fullscale research project using multiple-scenario planning methodologies (see Lindgren and Bandhold 2003) and selected cognitive mapping applications is planned for 2006-07.

Methodology For the first research project, an extended Case Study approach (Burawoy et al. 1991) was developed using six US-based software development houses systematically selected from over 50 candidates. The cognitive mapping component of the extended DSA package was based on Huff’s (1990) work on developing cognitive maps, which provided schemas for associating concepts. Nath and Newell developed specific cognitive styles that were adopted and modified for the research project (see Huff 1990, Huff, Huff and Barr 2000, Huff and Jenkins 2002, Nath and Newell 1998). Our preference for the project was to unearth deep insights from small number of detailed interviews. We selected a combination of tools that produced keywords, enabled coding of key words and phrases and, which in association with a schema for association of concepts, enabling developing of cognitive mapping of key themes, concepts and linkages between them.

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

24

NVivo was chosen as the central application for analysis and mapping (QSR International 2002). NVivo enables storage of documents and a system of coding key words, phrases or sentences for use in further analysis (NVivo has the Decision Explorer application incorporated as a module for effective and dynamic mapping. NVivo was the best available application at the time of analysis for the research project (2002-03). However, NVivo works on judgement analysis and coding i.e. the determination of key themes, concepts and mapping them with links of association is purely under the control of the researcher. Another challenge with NVivo is that it does not deliver ranked word or theme lists, not even based on word-frequencies. This issue highlighted a number of times to the software supplier but to date has not been incorporated into updated versions of NVivo. The TACT application produces straight ranked word-lists by frequency and is a simple firstround check on a document or aggregated documents (University of Toronto 1993). It does not offer any more analytical capability than that, but was be used as a pre-processor for NVivo in the project. Toward the end of the research project, Copernic Summarizer emerged as a new generation application that attempts to produce key concepts using semantic web technology which can connect concepts based on “meanings” rather than just text-words (Copernic 2005). It can produce key concepts as words, phrases or small sentences giving a more sophisticated list of key concepts. Copernic Summarizer also produces “S-Nets” which are stories created using key concepts within a minimum passage-size i.e. a story that summaries the key concepts in a document within a minimum size derived extract. Copernic Summarizer was applied to aggregated cross-case analysis to highlight key summary themes and findings which were further developed into a range of theoretic propositions. Copernic Summarizer is applicable for small sets of document but can be applied with diminishing returns to larger sets of aggregated documents. Enhancements and new applications using semantic web technology are likely to take over from existing straight text analysis tools over the next five years. Lessons on document analysis and cognitive mapping were drawn from the research project on exploring marketing decision-making associated with research, development and delivery of Internet-based software houses using storytelling methods into the second research project. For the second research project, 24 Interviews were undertaken with Television industry executives, analysts and observers. Secondary research inputs included over 90 Industry-related documents included a selection of previous industry scenario and future visions reports, company reports, and industry organisation and regulator reports. Although not planned at the commencement of the project, there was a realisation during the data collection phase of the project that development of “Key Theme and Concept Maps” would be a useful new methodological input for scenario development, plus a valuable way to present findings to media industry executives and observers at industry forums. Several software applications were reviewed for the purpose of producing effective maps. Some were suitable for developing and mapping themes using purely researcher judgement, while others provided mostly automatic generation of themes lists and maps. There were ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

25

significant time constraints for the pilot research project, which favoured developing quick “first-pass” maps. Most members of the research project team were familiar with the mindmap approach (Buzan, 1996). According to Wikipedia (2005): “A mind map or mindmap is a generic term used to describe a pictorial representation of a semantic network or cognitive map. The form of the map can involve colour or monochrome images, words, and lines, and can be arranged intuitively according to the spatial arrangement of concepts in the mind, or be organized into groupings, branches, or areas.” MindMap applications allow the researcher to quickly set up trees or islands or clusters of themes with associated concepts as branches off a tree or as entities on an island. There are several mindmap applications available, but the MindManager application (MindJet Inc. 2005) was selected, mainly of researcher familiarity with the application. Mindmaps were developed from researcher review and assessment of the 24 interview transcripts. There was still a requirement for at least a partially automated application to analyse the over 90 secondary documents associated with the project. Discussion on evaluation of these applications can be found in the Findings and Discussion Section below, but ultimately Leximancer was selected and used to analyse both interviews transcripts and secondary documents. A summary report and single scenario report was produced to complete the second research project.

Findings and Discussion Several outputs were produced for the first research project. The NVivo application complemented by TACT was effective for analysing and mapping accounts of decisionmaking. Cognitive maps as part of a comprehensive extended DSA package assisted in gaining deeper insights on decision-making, especially when reviewed and revised directly with the original decision-makers. Over 35 case-specific and cross-case findings were highlighted and 20 research propositions were developed out of the project. Nevertheless, despite the successful application of NVivo for cognitive mapping, there is recognition that newer Java-based applications, combined with semantic matching and new generation textual capabilities are likely to take over from existing applications. The Copernic Summarizer application was used toward the end of the project and future versions are likely to make it more attractive for both content analysis and mapping. The second project highlighted evaluation of various applications for identification and mapping of key themes and concepts. NVivo was evaluated for the project as a partially automated application in that it can select key words and phrases driven by the researcher and enable the researcher to manually develop maps for visualising key themes and manually associated concepts. Although there was considerable research experience and capability to use NVivo in the TV project, the application is best suited for small numbers of documents and is very labour-intensive to use and it was rejected for this project. As noted earlier the MindManager application was chosen to quickly map key themes and concepts drawn from the interview transcripts. However, NVivo could also have been used to ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

26

develop an initial map or ontology of key themes and concepts using just the Modelling tool without coding the interview transcripts. The equivalent standalone Decision Explorer application could also be used to produce “first-pass” maps. TACT, Copernic Summarizer, Grokker, NetMap and Leximancer were evaluated for analysis and mapping of both the interview transcripts and the 90 secondary documents associated with the project, TACT was quickly dismissed as while it produced simple word-lists, the application possessed no mapping capability. Copernic Summarizer was run on a small subset of documents and some produced interesting theme lists and a basic S-Net. The S-Net through was not in the form of a map – rather a story was produced that contained the most significant number of key concepts in the smallest story passage size. Copernic Summarizer was not regarded as efficient for analysis of large numbers of documents in its current form and capability, and was rejected for the project. However, as noted for the first project, Copernic Summarizer has great development potential. Extensions to link mapping tools to its S-Net capabilities and to manage large numbers of Web-based documents could see Copernic Summarizer emerge as a content analysis and mapping application of choice over the next three to next five years. Grokker is a new Java-based application that can trawl through WebPages and produce effective visualisations of key concepts within WebPages (Groxis, 2005). It can easily aggregate concepts from WebPages but has limited supporting capability in terms of listing and further analysing key concepts. Grokker was rejected for the project. Grokker though represents a new generation of Java-based applications that offer new approaches to analysis and mapping, either as a standalone option or in concert with several other emerging application technologies. NetMap was chosen early in the project as a preferred application to identify patterns of emerging key themes and concepts. NetMap is a well-established application that uses a range of algorithms and UNIX text action commands to select and extract and map matching concepts (NetMap Analytics, 2005). Depending on how data is set up and if the datasets are very large, NetMap can produce emerging concepts and map them. NetMap can be very powerful in unearthing emergent patterns from a wide range of datasets. During the data collection phase of the project, it was discovered that NetMap required preprocessed data inputs from other applications, plus a starting taxonomy (or ontology) of initial concepts before further NetMap processing can be undertaken. NetMap was the oldest application reviewed but can work with most of the other newer pre-processing applications reviewed for the project. Leximancer is a relatively new application that attempts automatically to identify key themes and to position or associate concepts with themes using Bayesian and a range of algorithms (Leximancer, 2005). Leximancer can also factor in customised Thesaurus sets to enhance theme and concept analysis. The application is integrated with Concept Ranking, Summarization, and Mapping capabilities all in the one package. Leximancer also is an effective pre-processing application for NetMap. On first view, Leximancer was attractive because it appears to produce maps associated with themes and concepts thus saving time for the researcher. However, there is a substantial risk ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

27

that the researcher may accept the association mapping at face value and not undertake further interpretative analysis and judgement – and therefore not really understand what is behind the maps produced by the application. Closer examination of Leximancer maps highlighted a powerful package where data can be dynamically manipulated from a basic set of key themes with no concepts, through to attempted matching of thousands of concepts to key themes – and ultimately to simple manipulation of on-screen toggles. Leximancer can produce a large number of maps, but it was difficult to produce one-shot summary maps. Leximancer’s maps provide different interpretations depending on the depth and number of concepts to be highlighted within the maps. In fact the Leximancer maps at various depths were judged as not to be clear enough to present to TV industry executives. The MindManager was preferred as the “front-end” presentation application for a more practical industry oriented audience. Leximancer was chosen to analyse both interview transcripts, and then the collective secondary data documents. An extra run analysed both sets of documents. Comparative tables highlighted themes and concept lists from the MindManager and the Leximancer applications. While there were intentions to input extra themes and concepts from the Leximancer application to set up an enhanced MindManager Map, time constraints held back this option for the pilot project. Both MindManager and Leximancer were recommended as the main applications for analysis and mapping for the proposed full-scale TV Project. Through the two research projects presented in this paper, a selection of the most advanced text analysis and textual engineering available were analysed for their applicability to produce a powerful set of key concepts and to turn them into effective cognitive maps. Applications evaluated in the two projects discussed in this paper are presented in Table 1. The Table highlights the key features as noted in evaluations for the projects discussed in the paper. No one application could produce everything that was required. Two or three applications together may constitute an effective form of triangulation for a research project. For the two projects reviewed, mapping based on judgement analysis produced the clearest maps for presentation, that is, MindManager (with NVivo also producing very good maps but at a significant disadvantage in terms of preparation). The partially automatic applications generally produced effective concept lists but none produced clear and effective visualisations. Emerging applications based on semantic and Java-based technologies are now emerging, that are closer to producing storytelling analysis more like how themes and concepts are committed to memory and then revisited and applied to different contexts are now emerging. However, a prerequisite for those applications to really bloom will be the development a more advanced state of computer mediation. That is indeed the main goal of the original developer of the World-Wide Web who today is focused on developing a semantic Web environment that will facilitate much more advanced management of unstructured information (BernersLee, 1999; W3C Worldwide Consortium 2004). Such an emerging environment might not be able to replicate human thinking, storytelling and mapping perfectly – but it may be sufficient to produce substantial leaps in understanding and acting on unstructured information to will make today’s applications look as ancient as software applications in other contexts were twenty years ago.

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

28

Table 1: Applications Reviewed For Analysis of Unstructured Data Application

Developer

TACT

University of Toronto

NetMap

NetMap Analytics

Features/ Judgemental Or Automatic? • • • • •

NVivo

MindManager

QSR International

MindJet Inc.

• • • • • • • •

Analysis and Ranking of Words Automatic on Text Uses a range of algorithms and UNIX text action commands to select and extract matching concepts and to map them Highlights emergent data patterns Judgemental For Set-Up Then Automatic for Pattern Matching and Mapping Storage of Documents into Project Sets Ranking of Search-Driven Words Coding of Words, Phrases, Sentences as “Nodes” Coding Across Several Documents Mapping of Coded Nodes (Model Explorer component is Decision Explorer from Banksia Software Judgemental For Coding and Map Development Manual Mapping of Themes and Concepts Judgemental

Comments •

Old Text Analysis Application from Late 1980’s/Early 1990’s

•

Developed in Late 1980’s/Early 1990’s as a powerful pattern matching application across large numbers of documents. Used in criminal matching searches and fraud detection. Requires substantial work to set up initial ontologies using “pre-processor” applications. Developed out of Nud*ist (used for analysing larger numbers of Documents). Powerful Analytical after Coding of Themes and Concepts completed (Very Labour Intensive Process)

• • • •

• •

Leximancer

University of Queensland

• • • •

Copernic Summarizer

Copernic Inc.

• • •

Grokker

Groxis MMV

• • • •

Storage of Documents into Project Sets Integrated Package with Word Rankings, Summarisation of Documents, Thesaurus Features to Enhance Theme/Concept Searches Automatically Draws Maps with Links of Association Between Theme and Concepts Can be set to be purely automatic (Self-Seeding Option) but usually set up with a starting set of themes (simple ontology) Searching based on semantic web technology from Fraunhofer Institute. Able to produce Theme and Concept Lists based on semantic linguistic connections. Can be set to be purely automatic (Self-Seeding Option) but usually set up with a starting set of themes (simple ontology) Produces “S-Net” Summaries of Document Contents Based on Java Technology Can analyse several forms of Web-based Documents and Return Summary Lists, Plus Effective Maps. Includes a suite of analytical applications to perform further analysis in results Automatic and completely Web-based

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

• • •

Delivers MindMap style Maps that are Familiar To Executives Used To MindMap/Brainstorming Sessions Can only be used directly on relatively small numbers of documents, but effective for developing initial ontologies Based on sets of Bayesian Conditional algorithms with capability to factor in wider searches based using Thesaurus options Powerful Range of Mapping Capabilities but difficult to Express In One Easy Map for Executives Very Good Pre-processor For NetMap

•

New generation application based semantic principles that will be included in next generation of World-Wide Web making it a very strong basic technology for future Web-based document analysis

• •

New powerful search and analysis application based on Java Used by Amazon and others for searching matching and making recommendations of search findings. Currently Web-based only but future versions promise substantial improvements in unstructured data analysis and visualisation

•

29

References Banxia Software, 2002. Decision Explorer. V 3.2, Windows XP, www.banxia.com Berners-Lee, Tim,1999. Weaving the Web, London, Orion Business Books. Burawoy, Michael, Burton, Alice, Ferguson, Ann Arnett, Fox, Kathryn J., Gamson, Joshua, Gartrell, Nadine, Hurst, Leslie, Kurtzman, Charles, Salzinger, Leslie, Schiffman, Joseph, Uri, Shiori, 1991, Ethnography Unbound: Power and Resistance in the Modern Metropolis. Berkeley CA: University of California Press. Buzan, Tony, and Buzan, Barry, 1996, The Mind Map Book: How to Use Radiant Thinking to Maximize Your Brain's Untapped Potential. Plume Books Capon, Noel, Hulbert, James, 1975. Decision Systems Analysis In Industrial Marketing. Industrial Marketing Management, 4, 143-60. Copernic (2005), Copernic Summariser, http://www.copernic.com/en/products/summarizer/index.html, Accessed 30 June 2005 Cyert, Richard M., Simon, Herbert A., Trow, Donald B., 1956. Observation of a Business Decision. Journal of Business, 29(4), 237-48. Fraunhofer Institute – IITB (2004), “Wissenschaliche Hilskrafte/Praktikantinnen: Intelligente Softwareagenten, Semantic Web, UML, Java, J2EE, CORBA”, http://www.iitb.fraunhofer.de/ , Accessed 20 February 2005 (Note: there is no English translation for this site). Groxis (2005), Grokker 2, http://www.groxis.com/service/grokker/grokker.html, Accessed 30 June 2005. Howard, John A., Morgenroth, William M., 1968. Information Processing Model of Executive Decision. Management Science, 14 (7), 416-28. Howard, John A., Hulbert, James, Farley, John U., 1975. Organisation Analysis and Information Systems Design: A Decision-Process Perspective. Journal Of Business Research, 3 (2), 133-48. Huff, Anne Sigismund, 1990. Mapping Strategic Thought, New York, John Wiley and Sons. Huff, Anne Sigismund, Huff, James Oran , with Barr, Pamela S., 2000. When Firms Change Direction. New York, Oxford University Press. Huff, Anne Sigismund, Jenkins, Mark Eds., 2002. Mapping Strategic Knowledge, London, Sage. Hulbert, James, 2003. Organisational Analysis And Information Systems Design: A Road Revisited. Journal Of Business and Industrial Marketing, 18 (6/7), 509-13.

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

30

Hulbert, James, Farley, John U., Howard, John A., 1972. Information Processing and Decision-making in Marketing Organisations. Journal of Marketing Research, 9 (February), 75-7. Lindgren, Mats, and Bandhold, Hans, 2003, Scenario Planning: The Link Between Future and Strategy, Palgrave MacMillan, Basingstoke Langley, Ann, Mintzberg, Henry, Pitcher, Patricia, Posada, Elizabeth, Saint-Macary, Jan 1995, Opening Up Decision Making: The View From The Black Stool, Organization Science, 6 (May-June), 269-79. Leximancer (2005), Leximancer, http://www.netmapanalytics.com/, Accessed 30 June 2005 Lloyd-James, Andrew, Chandler, Annmarie, and Pattinson, Hugh M., 2005, Outside The Box, Scenarios for Australian Television in 2015. Work-In-Progress Project. (Second Research Project in this Paper). Mindjet Inc. (2005), MindManager, http://www.mindjet.com/us, accessed 30 June 2005 Mintzberg, Henry, Rasinghani, Duru, Theoret, Andre, 1976. The Structure of "Unstructured" Decision Processes. Administrative Science Quarterly, 21 (June), 246-75. Morgenroth, William M., 1964. A Method for Understanding Price Determinants. Journal of Marketing Research, 2 (August), 17-26. Nath, Deepika, Newell, Stephanie E., 1998. Organisational Responses to a Hypercompetitive Environment: A Case study of Pepsi Canada. Journal of Business Research, 41, 41-48. NetMap Analytics (2005), NetMap Analytics, http://www.netmapanalytics.com/, Accessed 30 June 2005 Pattinson, Hugh M. (2005), Mapping Implemented Strategies Of Bringing Internet-Based Software Applications To Market Using Storytelling Research Methods, PhD Dissertation, University of Technology, Sydney. (First Research Project in this paper) QSR International (2002), NVivo, V 1.3, Windows XP, www.qsr.com.au Schank, Roger C., 1998. Tell Me A Story: Narrative And Intelligence, Evanston Il.: Northwestern University Press. University of Toronto, (1993), TACT, V 2.1, MS-Dos (Runs In MS-DOS Prompt in Windows), www.chass.utoronto.ca Vyas, Niren, Woodside, Arch G., 1984. An Inductive Model of Industrial Supplier Choice Processes. Journal of Marketing, 48 (Winter), 30-45. W3C Worldwide Consortium (2004), “Semantic Web”, http://www.w3.org/2001/sw/, viewed on 20 February 2005. Weick, Karl E., 1995. Sensemaking in Organisations, Thousand Oaks CA, Sage.

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

31

Weick, Karl E., 2001. Making Sense of the Organisation, Oxford, Blackwell Publishing. Wikipedia, 2005, MindMap, http://en.wikipedia.org/wiki/Mindmap . Accessed 14 September 2005 Woodside, Arch G., Samuel, David M., 1981 Observations of Centralised Corporate Procurement. Industrial Marketing Management, 10, 191-205. Woodside, Arch G., 1994. Network Anatomy of Industrial Marketing and Purchasing of New Manufacturing Technologies. Journal of Business and Industrial Marketing, 9 (3), 52-63. Woodside, Arch G., Wilson, Elizabeth J., 2000. Constructing Thick Descriptions of Marketers' and Buyers' Decision-Processes in Business-to-Business Relationships. Journal of Business and Industrial Marketing, 15 (4/5), 354-69. Endnote: Examples of Maps produced by applications discussed in the paper have not been included due to page-limit restrictions, but they may be presented at the actual Conference and a selection is available on request.

ANZMAC 2005 Conference: Marketing Research and Research Methodologies (qualitative)

32