We summarize the conclusions of this chapter.
Requirements engineering. Requirements define what a potentially new sys-tem must be capable of to solve a specific problem or need. Stakeholders, such as customers, developers, and users, define these requirements. They do so in the requirements engineering process that formulates, documents, and systematically maintains requirements. Poorly managed requirements can lead to major risks such as exceeding the budget or failing the project. In requirements engineering, one way of coping with the risks is to include change management. Adapting to change is crucial, as the needs of the users may change or the evolution of the software requires changes. Moreover, user satisfaction decides about the rise and fall of the software in the market. As a consequence, requirements engineering must involve users to identify their changing needs.
User involvement. Traditional approaches, such as workshops and observa-tions, help in understanding features users want and how they use them. How-ever, those approaches are usually part of the early development phases and are challenging to perform continuously with a representative sample of users. There-fore, stakeholders must consider new and modern approaches to involve users.
With the increasing popularity of social media and app distribution platforms, users started to write their opinion about software by reviewing and rating them online (explicit user feedback). Explicit user feedback contains valuable informa-tion for requirements engineering, such as encountered problems and requests for new features. Users state their opinion regularly, making explicit user feedback a continuous source for information. Research shows that considering the usage context, in addition to the users’ opinion, is vital for user involvement. If the deployed software collects interaction events like clicks and context data, such as the app and operating system version, stakeholders can, e.g., leverage that information to narrow down the location of bugs (implicit user feedback). The combination of explicit and implicit user feedback can lead to either new require-ments or requirerequire-ments of better quality.
Challenges. Including explicit and implicit user feedback in the requirements engineering process comes with challenges. Popular apps receive explicit user feedback thousandfold daily. Similarly, collecting implicit user feedback generates a million interaction events with only a few users. Both feedback types generate a large amount of noisy data that is unfeasible to analyze manually. Therefore, we need to support stakeholders by automatically and continuously collecting and filtering requirements-relevant feedback. Requirements-relevant feedback is, for example, feedback that describes problems, has inquiries to the stakeholders, or includes requests for features.
Analyzing and matching features. Stakeholders need to analyze the filtered user feedback to understand if users address already existing features or request new features. If stakeholders can extract the particular features users mention in their feedback, they can understand the problem and feature requests better.
They can also gain insights like what the popular features are and how often users address them to understand their impact on user satisfaction. For exam-ple, a feature that occurs in many problem reports may have a higher impact on the app’s success than a feature that only one user reported. For an automated understanding, if users discuss existing features, we have to match them with the features in the requirements documentation. Stakeholders document and ad-vertise the features of apps on app pages. App pages contain a description that
typically lists and describes the features of the app. As a consequence, we can identify and match the features mentioned in user feedback with the documented features on app pages.
Stakeholder needs. So far, stakeholders do not use automated tools as they are not available in the described sense, although, they seek for automated support.
In the next chapter, we present two qualitative studies detailing on stakeholder’s needs for analyzing both types of feedback. Then, we introduce requirements intelligence, a framework to automatically collect, filter, and match features from user feedback with the documented features from app pages. The framework provides an integrated interactive visualization to support stakeholders in their decision-making process.
Stakeholder Needs for Automated User Feedback Analysis
Some people don’t like change, but you need to embrace change if the alternative is disaster.
Elon Musk
Publication. This chapter is partly based on our 2016 publication “On the au-tomatic classification of app reviews”[151] and extends its interview study. My contributions to this publication were to co-design the interviews, interviewing stakeholders, creating tool mockups, and to help analyzing and discussing the paper’s results.
Contribution. This chapter contributes with two qualitative studies to investi-gate how stakeholders perceive the usefulness of user feedback and their needs for automated analysis of that feedback. We present a review of related discussing automated user feedback analysis approaches and an interview study, which fur-ther aims to evaluate an app review analytics mockup with stakeholders.
3.1 Motivation
In the previous chapter, we concluded that continuously involving users by an-alyzing explicit and implicit user feedback is crucial for the success of apps. In this chapter, we aim to investigate the usefulness of user feedback from the per-spective of stakeholders and how they include it in their processes. We
distin-guish between the term stakeholder (any practitioner involved in the decision-making process for the software development) and the term user, who is the end-user of the system.
In Chapter 1, we briefly discussed the need for automated tool support when analyzing explicit and implicit user feedback. More specifically, in Chapter 2, we argued from a data-driven perspective as research highlights the large amount of feedback stakeholders receive. This chapter focuses on the industry perspective and investigates whether stakeholders require automated feedback analyses by performing qualitative research. We reviewed studies on user feedback usefulness and performed interviews to understand stakeholder needs. In total, our studies include the aggregated perspective of 90 stakeholders.
The chapter is structured as follows. First, we describe the research questions and methods followed in Section 3.2. Section 3.4 discusses the results of the interview study. Then, Section 3.3 extends the interviews by summarizing sci-entific evidence of stakeholder needs from related work. After that, we discuss and compare opportunities and challenges stated by stakeholders in Section 3.5.
Eventually, we summarize the findings of this chapter in Section 3.7.