Audio Recording in Software Engineering

Audio Recording in Software Engineering Michael Nagel Jonas Helming Maximilian Koegel Technische Universität München Boltzmannstr. 3 85748 Garching...
Author: Blake Atkins
1 downloads 0 Views 320KB Size
Audio Recording in Software Engineering Michael Nagel

Jonas Helming

Maximilian Koegel

Technische Universität München Boltzmannstr. 3 85748 Garching, Germany

Technische Universität München Boltzmannstr. 3 85748 Garching, Germany

Technische Universität München Boltzmannstr. 3 85748 Garching, Germany

[email protected]

[email protected] Helmut Naughton

[email protected]

Technische Universität München Boltzmannstr. 3 85748 Garching, Germany

[email protected] ABSTRACT

Keywords

In software engineering projects, valuable knowledge is exchanged in informal, unplanned conversations such as watercooler discussions. Projects can benefit if this knowledge is externalized and made accessible to all project participants. Previous solutions for recording project conversation generally do not cover informal communication due to the lack of portability and widespread availability of their audio recording infrastructure. They also do not capture the context of a conversation, which makes it hard to find specific information in a large number of recordings. We propose a solution that overcomes both limitations by employing mobile devices as an ubiquitous recording infrastructure that also enables the user to bridge the gap between informal communication and formal project model elements. The mobile recording application can establish links between a conversation recording in our central audio repository and project model elements in our project management and CASE tool. Project participants can use the application to capture audio and context, create new work items for other participants, engage in asynchronous audio discussions and review project artifacts. We evaluate each of these use cases in several ongoing case studies.

Software engineering, audio, recording, CASE

Categories and Subject Descriptors D.2.1 [Software Engineering]: Requirements/Specifications; D.2.2 [Software Engineering]: Design Tools and Techniques; D.2.9 [Software Engineering]: Management

General Terms Documentation, experimentation, human factors, management

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICSE ’10 South Africa Copyright 2010 ACM X-XXXXX-XX-X/XX/XX ...$10.00.

1.

INTRODUCTION

A vital part of any software engineering project is conversation between participants. During conversations, knowledge is exchanged and built, work is planned, issues are raised and resolved [3]. Therefore, such conversations contain valuable information for and about a project, but their content can easily get lost if it is not externalized in any form, and subsequently accessible to the conversation participants and other project members. Conversations may occur in a structured and planned form, such as staff meetings and requirements elicitation sessions with a customer, or in unplanned, informal forms such as hallway meetings, watercooler discussions or office visits. There are numerous approaches in requirements engineering and meeting management to record the content of team meetings and customer meetings in audio form or video form [3, 6, 8, 16, 17], but unstructured and informal, unplanned conversations are generally neglected. We identified two shortcoming of the existing approaches which may contribute to this. First, these recording approaches usually rely on infrastructure with limited portability and availability (e.g. [1, 3, 8]), therefore they usually only cover structured and planned conversations. Unplanned communication can occur anywhere, and it is generally not feasible to instrument literally every square meter of a building with recording equipment or to move every conversation to designated places with installed equipment. Second, recordings are usually missing context (e.g. [14, 16, 17]). Informal conversations often discuss formal artifacts such as requirements or system design items. If this context is not captured during or directly after the conversation, it will most probably be lost. Furthermore, in order to allow other project participants to find recordings about specific topics, mere availability is not sufficient. Without an index or a search function, it would be very time-consuming to find a recording by listening to all possible candidates. To build an index or search function for recordings, it is necessary to know the context of every recording or to have a plain-text transcript of all content.

We propose to overcome the limited portability and availability of previous solutions by an ubiquitous recording infrastructure as a mobile application on devices that project participants carry all the time (e.g. smartphones). All recordings are transferred into a central repository to make them accessible to all project members. We address the issue of missing context by linking audio items directly to project documentation and project model elements and provide a method to access recordings directly from the linked objects in a CASE tool. Parts of this approach have been proposed in the past, but we claim nevertheless that mobile audio recording in combination with context acquisition is a novelty in software engineering. While ubiquitous computing has been proposed in this application domain, our contribution is its combination with context linking. To show the applicability and benefits of this approach, we define the following use cases and evaluate them in a multi-case study. • Record informal, unstructured communication and capture context information • Create work items and assign them to project members • Record comments while reviewing documents, code and other artifacts • Engage in asynchronous voice discussions Outline: In section 2, we present related work in the field of audio recording and software engineering with mobile devices. In section 3, we describe our approach of mobile recording with links to context in a CASE system. In section 4, we explain the four main use cases we cover with our mobile application approach. In section 5, we describe two ongoing case studies that evaluate the main use cases of our approach. Section 6 outlines the initial results of these case studies. In section 7, we present a conclusion as well as future work.

2.

RELATED WORK

In this section, we present related work. First, we describe approaches that generally rely on audio recording and analyze content. Second, we list approaches that use mobile devices in software engineering, but do not record audio. Hindus et al say that it would be desirable to record all conversations if possible [6]. They call this ”ubiquitous audio”, but do not have any suitable infrastructure at their disposal. They also rely on automated content analysis, which isn’t yet available in a useable form. Cherry et al recorded audio and video of team interactions to measure the impact of different participant types on team dynamics [3]. They do not link their recordings to formal models and require complex infrastructure. Tucker et al recognize the usefulness of storing recorded speech, but try to organize and access audio on the recording device instead of linking it to formal context elements [17]. Stiefelhagen et al record meetings with omnidirectional cameras, but do not cover informal communication and formal model links [16]. Lee et al propose a portable audio/video meeting recorder [8], although this equipment is still expensive, limited and not portable enough for spontaneous recordings. Purver et al recorded meetings to analyze their content automatically. They did not cover informal and unplanned

Figure 1: Mobile recording application

conversations, and focused on content analysis [11], [10]. Ogata et al recorded meetings in audio form and used an automated process that detects laughter, coughs and pauses to identify the meetings’ structure [9]. Basu et al instrumented a room with sensors to observe and influence the behavior of meeting participants [1]. This requires static infrastructure and is unsuitable for informal conversations in situations where those sensors are not present. Richter et al presented a traceability framework to capture and access multimedia sources [12], [13] and showed that access to captured information is beneficial for requirements engineering processes. They analyzed video recordings of meetings and knowledge acquisition sessions and provided an index structure for traceability. However, they relied on complex and limited infrastructure (video recording equipment) and created manual and semiautomatic transcripts of the recordings in order to index them. This process requires manual processing and does not scale well. Sawhney et al propose an infrastructure for asynchronous audio communication, but do not link their audio messages to formal context elements [14]. Fahmi et al researched the use of mobile devices for requirements engineering [4]. They used web-based requirements elicitation tools without audio recording capabilities on PDAs. Seyff et al also investigated web-based requirements engineering tools without audio recording capability on mobile devices (PDAs) [15]. All of the approaches above lack either the mobile recording aspect or the link to context elements.

3.

APPROACH

Our first goal is provide mobility for audio recording. Therefore we built our recording application on the iPhone OS platform. Figure 1 shows a screenshot of the application’s recording screen. The application is capable of recording 5 hours of continuous audio per single session. It offers full dictaphone-style controls to rewind, overwrite and insert audio as well as automatic sound-level based pausing (voice activation) in order to broaden its possible applications. Recordings are stored locally on the device to enable network-independent operation. When the device is in range of a wireless network, the recording application can upload its stored recordings to a central audio repository. We use a WebDAV-enabled web server as our central audio repository.

Capture informal conversation

Audio Repository

Audio recording

UNICASE

Context Element

URL Attachment

Project Participant

Create work item

Record audio

Discuss asynchronously

Link recording to context element

Review project artifact

Requirement

Task

connection implies

Comment

Figure 3: Mobile application use cases Figure 2: Audio context links Our second goal is to enable users to establish a context for their audio recordings. To achieve that, we tied the application to the UNICASE [2] backend. UNICASE includes a repository for storing models, EMFStore [7]. On the one hand this includes artifacts to model the system under construction, such as requirements, UML models and system specifications. On the other hand UNICASE stores project management artifacts like tasks, bug reports and issues [5]. We chose to integrate our approach with UNICASE as we do not limit the context elements of recorded audio to a specific domain. Furthermore, UNICASE captures discussions about specific artifacts. UNICASE offers a rich client based on Eclipse to browse and modify artifacts in the EMFStore repository. We enhanced this client to show recorded audio attached to a context elements and directly play it. The mobile application enables the user to attach a recording to any existing model element in a UNICASE project. It can also create some specific classes of model elements from scratch, such as tasks, action items or bug reports, to accommodate recordings for which no model element exists yet. The recording is connected to UNICASE as a URL attachment that points to the actual audio file in our WebDAV audio repository. Such a URL attachment can be linked to an arbitrary number of model elements, which can be useful when a conversation touches multiple topics. Figure ?? shows the links between audio repository and UNICASE backend. In section 4 we describe how users can access these discussions on a mobile device and answer to them by audio recording.

4.

APPLICATIONS

Mobile audio recording with context acquisition opens up a whole range of applications in a project. Based on the two use cases mobile audio recording and linking model elements to recordings (described in section 3) we identified four use cases for which we offer explicit support. Figure 3 shows the four use cases and how each of them includes the two basic use cases. We will describe them in this section. The first and most obvious application is to record an informal conversation in the context of one or more specific model elements. The user can either record first and afterwards link model elements from the repository, or he can select model elements first and record the conversation in this context. As an example, two developers may engage in a spontaneous hallway discussion about design issue X

regarding class Y in their current project. One of the developers records this conversation. After the discussion, he uploads the recording to their audio repository and attaches it to issue X and class Y. All project participants now see the attached recording when they view either X or Y and can directly listen to it. As another example, informal feedback of an end-user of the system can be attached to a specific use case. From initial experiments we quickly identified the need to create new context elements in addition to being able to select existing ones. This occurred very often when the subject of the voice memo were new tasks or issues. Therefore, we enable the user to link a recording to a context element which he creates within the linking process. We focus on the creation of work items, as they are not formally structured and have a responsible project participant. Furthermore, work items can be attached to any system model element in UNICASE to provide further context. Elements created through this have to be at least initally triaged. The most important thing is to assign the new work item to a specific person. As a consequence, the created work item will appear as an open item on his/her task list in UNICASE and can be further triaged or integrated. UNICASE offers a commenting/discussion facility for all model elements, so we will extend the mobile recording application to create audio comments with recipients on model elements. This facility will make it possible to discuss model elements through voice memos instead of typed text. To remove the last media break in comment trees, we also included a text-to-speech component to play comments that were originally typed text in audio form. With this, we establish the concept of entirely audio-based comment trees. We call this asynchronous voice discussion. As a side effect, participation in a project will be feasible in situations such as while driving a car, which is currently nearly impossible. In a typical session, a user starts participating in audio discussions by instructing his mobile application to download all new comments that list him as one of the recipients. Audio comments are downloaded as-is, text comments are converted to speech by the server first. The user then sees a list of all those comments, annotated with their author, the name of the model element to which they’re attached and their length. He can listen to any of the comments and optionally record a response, which is uploaded to the server as a new audio comment response to the original comment. In

the UNICASE rich client, the user sees a typical tree view of comments, some of which are audio only. Those comments can be played directly in the client. A rich client user can create new comments or responses as either text or audio comments. Another common task for software engineering project participants, especially for team leaders and managers, is reviewing documents or code. These reviews generate reviewer comments for the author(s) of the document. Unless the author is present during the review, these comments need to be captured and transported to the author. Our mobile application can be used to capture reviewer comments on a medium independent of the artifact itself, in order to avoid media breaks and context switches for the reviewer. That way, a reviewer can concentrate on the document and naturally speak his comments without having to switch between applications or media to note them down. This is most useful for electronic documents and artifacts within the CASE tool, because the reviewer can directly attach the audio recording to the document and notify the author. A typical review starts with the reviewer retrieving the document or artifact to review. The medium on which he views the document is not relevant to the process, it can be either on a computer or on paper. First, the reviewer starts a recording, then he proceeds with his review. Whenever he wants to comment on some aspect of the document, he just speaks his comment. At the end, the review will be recorded as an audio file that contains only the actual comments, because the recording software has automatically erased silent intervals from the recording. The review ends with the upload of the recording, which is then either attached to a model element that represents the reviewed document, or inserted into the system as a new work item for the document author.

5.

CASE STUDIES

We are currently conducting a case study in a large student software engineering project. This project has 38 participants, organized into four developer and two management teams with two overall project leads. We supplied all members of two development teams and the two management teams with iPod Touch devices running our mobile recording application to enable them to record all conversations that they deem relevant for the project. In the preparation meetings, we strongly recommended that the students record all relevant informal conversations, not only team meetings. This case study only intends to investigate one use case, informal conversation recording, out of the four described in section 3. The second running case study is the UNICASE project itself. The project consists of 20 developers in a distributed setting. The core UNICASE developer team uses iPhone and iPod Touch devices to record planned and informal oneto-one meetings and to create and assign tasks. Therefore, this study covers the informal conversation recording and work item creation use cases. We will soon extend this case study to include artifact reviews and asynchronous voice communication as well. The first question we are interested in is how project par-

ticipants use the approach to capture audio recordings. We classify the recordings by the situation in which they were recorded: • team meetings • client meetings • presentations • task identification • informal communication Furthermore, we want to compare the different classes of devices. More specifically, we want to investigate the difference in practical use between smartphones (iPhone), which we expect to be reachable all the time, and separate recording devices (iPod Touch) that we expect to be present, but stowed away. We plan to conduct a survey to identify impediments and difficulties of the approach. Finally, to assess the value of audio recordings, we will conduct an experiment. We will present (partial) project models with and without attached audio recordings to independent subjects and measure their understanding of the models.

6.

INITIAL RESULTS

The student project has been running for about two months at the time of writing, and will continue for two more weeks in a different setting. During these two months, students uploaded 31 recordings into our central audio repository. The following table shows a classification of these recordings. Category Team meetings Presentations Client meetings Informal conversations

Count 16 7 7 1

The recordings had a length between about 30 minutes and nearly three hours, and most of them were linked to the model element that represented the respective meeting. Surprisingly, the students recorded only one informal conversation. We interviewed the participants of the case study informally to investigate this, and the participants stated that they did not record more informal conversations because of the inconvenience of having to locate their iPod Touch device and the plug-in microphone and assemble them, then launch the mobile application. This confirms our assumption that any recording devices used for capturing informal communication must be as easily usable and accessible as possible in order to improve cooperation by the users. Furthermore, users need to be better educated about the value of capturing informal conversation. Software engineering beginners, such as students, apparently do not immediately recognize the potential benefits and tend to be discouraged by smaller inconveniences. In the final two weeks, the students will relocate to a typical office setting and work full-time on the project. We intend to make sure that every student has a recording device charged and ready at his/her workplace to remove the aforementioned inconvenience hindrance. In contrast, the UNICASE project participants did not any record team meetings at all. The two project managers used the mobile application to record task descriptions and

add them to UNICASE as work items for other project members. They reported that this approach saves a lot of work in comparison to typing the task descriptions as plain text.

7.

CONCLUSION AND FUTURE WORK

In this paper, we proposed a mobile solution to record informal knowledge and relate it to formal context elements. The approach is based on a mobile application running on smart phones and enables linking audio artifacts to model elements in the unified CASE tool UNICASE. We presented four use cases enabled by this solution. We are currently conducting two case studies to evalute the proposed approach. Initial results show that even if a mobile solution is provided, participants tend to record planned events more than unplanned and informal communication. We believe this is due to two main reasons. First is device availability. We will focus on smart phones rather than separate devices, as initial feedback shows that users are more likely to have the mobile application with them and use it when it is on their phone instead of a separate device that also needs an external microphone attached. Second, we observed a significant difference in motivation between the use cases. Use cases that immediately save the user some effort, such as speaking new work items instead of typing them, tend to see much more use than those that require (even minimal) effort investment now for future benefits, such as conversation recording. This is especially true in the UNICASE project, where the project leads use the approach to assign and explain new tasks. It is necessary to create additional incentives or increase motivation for other use cases. As future work we plan to support the review use case also for document artifacts such as PDF or Microsoft Word documents. We plan to adapt the application to the newly announced Apple iPad device which will allow users to review documents comfortably on a 10” screen and record their comments all on the same mobile device.

[5]

[6]

[7] [8]

[9]

[10]

[11]

[12]

Finally, we conclude that capturing informal communication requires a seamless and very easy to use approach. We also believe that users have to realize the potential uses and possibilities before they will think about recording anything. This is directly related to the need of proving the benefit of recorded audio. We plan an experiment to evaluate the understanding of specific system models with and without attached audio recordings.

[13]

8.

[15]

REFERENCES

[1] S. Basu, T. Choudhury, B. Clarkson, and A. Pentland. Towards measuring human interactions in conversational settings. In Proc. IEEE CVPR Workshop on Cues in Communication, 2001. [2] B. Bruegge, O. Creighton, J. Helming, and M. K¨ ogel. Unicase – an ecosystem for unified software engineering research tools. volume Online, 2008. [3] S. Cherry and P. N. Robillard. Audio-video recording of ad hoc software development team interactions. In Cooperative and Human Aspects on Software Engineering, ICSE Workshop on, pages 13–21, Los Alamitos, CA, USA, 2009. IEEE Computer Society. [4] S. A. Fahmi, A. Ibrahim, and H. Choi. Enhancing requirements engineering activities through the use of

[14]

[16]

[17]

mobile technology devices and tools. In Future Generation Communication and Networking, volume 2, pages 578–581, Los Alamitos, CA, USA, 2007. IEEE Computer Society. J. Helming. Integrating software lifecycle models into a uniform software engineering model. In B. B. Walid Maalej, editor, Software Engineering Conference - Workshop Proceedings, GI-Edition Lecture Notes in Informatics, pages 157–164. Technische Universit¨ at M¨ unchen, Gesellschaft f¨ ur Informatik, 2008. D. Hindus, C. Schmandt, and C. Horner. Capturing, structuring, and representing ubiquitous audio. ACM Trans. Inf. Syst., 11(4):376–400, 1993. M. Koegel and J. Helming. EMFStore. http://emfstore.org, 2009. D.-S. Lee, B. Erol, J. Graham, J. J. Hull, and N. Murata. Portable meeting recorder. In MULTIMEDIA ’02: Proceedings of the tenth ACM international conference on Multimedia, pages 493–502, New York, NY, USA, 2002. ACM. J. Ogata and F. Asano. Stream-Based classification and segmentation of speech events in meeting recordings. In Multimedia Content Representation, Classification and Security, pages 793–800. 2006. M. Purver, J. Dowding, J. Niekrasz, P. Ehlen, S. Noorbaloochi, and S. Peters. Detecting and summarizing action items in multi-party dialogue. In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, 2007. M. Purver, P. Ehlen, and J. Niekrasz. Detecting action items in multi-party meetings: Annotation and initial experiments. Lecture Notes in Computer Science, 4299:200, 2006. H. Richter, G. Abowd, W. Geyer, L. Fuchs, S. Daijavad, and S. Poltrock. Integrating meeting capture within a collaborative team environment. In Ubicomp 2001: Ubiquitous Computing, pages 123–138. 2001. H. Richter, R. Gandhi, L. Liu, and S. W. Lee. Incorporating multimedia source materials into a traceability framework. In Proceedings of the First International Workshop on Multimedia Requirements Engineering, page 7, 2006. N. Sawhney and C. Schmandt. Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments. ACM Trans. Comput.-Hum. Interact., 7(3):353–383, 2000. N. Seyff, P. Gr¨ unbacher, N. Maiden, and A. Tosar. Requirements engineering tools go mobile. In Proceedings of the 26th International Conference on Software Engineering, pages 713–714. IEEE Computer Society, 2004. R. Stiefelhagen, X. Chen, and J. Yang. Capturing interactions in meetings with omnidirectional cameras. International Journal of Distance Education Technologies, 3(3):34–47, 2005. R. C. F. Tucker, M. Hickey, and N. Haddock. Speech-as-data technologies for personal information devices. Personal Ubiquitous Comput., 7(1):22–29, 2003.