Measuring Software Development Productivity & Incident Management Techniques

JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY ANANTAPUR COLLEGE OF ENGINEERING (Autonomous) DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING ANANTAPURAMU-515002 ANDHRA PRADESH Data Science for Software Engineering Unit 2 By Mr. Muntimadugu VijayKanth Assistant Professor(Ad-hoc) Department of Computer Science and Engineering JNTUA College of Engineering( Autonomous), Ananthapuramu

Data Science for Software Engineering

How To Tame Online Services: During the operation of an online service, a live-site service incident can happen: an unplanned interruption, outage, or degradation in the quality of the service. So, A tool called Service Analysis Studio (SAS) targets real incident management scenarios of large-scale online services provided by Microsoft. It includes a set of data-driven techniques for diagnosing service incidents. The major techniques SAS offers: Identification of incident metrics. Leveraging previous effort for recurrent incidents. Mining suspicious execution patterns. beacons from system

Measuring Induvidual Productivity: No Single and Simple Best Metric for Success/Productivity. The results show that software developers generally want a combination of measures to assess productivity, and these combinations varied significantly across developers Measure the Process, Not Just the Outcome. rather than just focusing on a measure of the outcomes of the development activity, such as the code artifacts, the process of developing the software is important as well. Allow for Measures to Evolve. the activities and responsibilities of an individual developer evolve over time and thus the measures capturing his/her productivity have to evolve.

Cont.: Goodhart s Law and the Effect of Measuring. Goodhart s law states that When a measure becomes a target, it ceases to be a good measure. How to Measure Individual Productivity? Measuring productivity of individual developers is challenging given the complex and multifaceted nature of developing software. There is not a single and simple best measure that works for everyone. Rather, you will have to tailor the measurement to the specific situation and context you are looking at and the specific goal you have in mind with your measurement.

Stack Traces Reveal Attack Surfaces: Stack traces from software crashes have been used for a variety of fault localization purposes. Example: stack mine Another use of stack traces? Security professionals have developed the concept of an attack surface. The open web security project defines attack surface as the paths in and out of a system, the data that travels those paths, and the code that protects both. Attack surface approximation The attack surface of a system is determined by looking at stack traces from crash dumps from the target system. The crash dump stack traces are parsed out into the individual code elements that are seen on each stack trace.

Visual Analytics For Software Engineering Data: Visual analytics is the science of analytical reasoning facilitated interfaces . by interactive visual It focuses on the interactive exploration and manipulation of the data. a user can perform several successive queries and view the data in a variety of formats before satisfactorily identifying the data in which they are most interested.

Visual Analytics For Software Engineering Data: (Cont.) For example, assume that we have an Excel spreadsheet which contains the app sales data of a multinational software corporation. The corporation has three product teams (code named Team A, Team B, and Team C) developing Game and Education Apps. The data includes yearly app sales in different counties, along with the detailed sales of different teams and categories. There are five columns ( Year, Team, Category, Country, Sales ) in the spreadsheet. Some sample records are shown in the following table.

Visual Analytics For Software Engineering Data: (Cont.) In Metro Eyes, users can perform data exploration through direct manipulation of the visual objects. Say the users want to explore the sales of Team A. As illustrated in Fig., users can directly select the Team A bar from the bar chart representing the contribution of each team to Sales. The tool can then extract from the data source the app sales data contributed by TeamA, and display it in a new bar chart.

Gameplay Data Plays Nicer When Divided Into Cohorts: CohortAnalysis as a Tool for Gameplay Data Cohort analysis turns raw player data logs into groups of players, or cohorts. By having groups include enough players, the variation that occurs during a single player s progression with a game will average out among other players. Play to Lose the average players in was playing about 15% more matches than the players who were improving. It appeared that for many players, improvement was not a relevant condition for them to stick with the game. Forming Cohorts Cohorts can be formed from any condition, but a few conditions especially make sense for grouping players. For competitive games, this is often based on an Elo rating system - a method for calculating the relative skill level among players.

Gameplay Data Plays Nicer When Divided Into Cohorts: (Cont.) Case Studies of Gameplay Data by simply dividing the players into yet another cohort grouping of total number of matches played, players who simply played a greater number of matches (while starting at a lower skill level and gaining skill slowly), still overcame the less-frequent players by playing vastly more matches in the same amount of time. Example: real-time strategy game StarCraft 2 Challenges of Using Cohorts When looking at a particular difference between cohorts, also compare these differences within players of a cohort. Cohorts may be biased in ways you do not expect. Mining data from games, whether with cohort analysis or other methods, still requires care when attempting to make causal claims.

A Success Story In Applying Data Science In Practice: Analytics Process Successful deployments of data science and analytics in a project with industry partner followed a process (Fig. 1) and had the following characteristics. Data Collection Exploratory Data Analysis Model Selection Performance Measures and Benefit Analysis

A Success Story In Applying Data Science In Practice: (Cont.)

A Success Story In Applying Data Science In Practice: (Cont.) Communication Process Best Practices Problem Selection Managerial Support Project Management Trusted Relationship

Theres Never Enough Time To Do All The Testing You Want: Software is present in nearly every aspect of our daily lives and also dominates large parts of the high-tech consumer market. software developers have to produce more features in less time while maintaining, or even increasing, product quality. Testing can slow down development processes and cost money in terms of infrastructure and human involvement. Thus, the effort associated with testing must be carefully monitored and managed. The Impact Of Short Release Cycles (There s Not Enough Time) Testing Is More Than Functional Correctness (All the Testing You Want) Learn From Your Test Execution history Test Effectiveness Test Reliability/Not Every Test Failure Points to a Defect TheArt of Testing Less Without Sacrificing Code Quality Tests Evolve Over Time

The Perils Of Energy Mining: Measure A Bunch, Compare Just Once- Let s ENERGISE Your Software Energy Experiments Environment prepare a stable testbed for energy measurement. N-versions run a test across more than one version of the software. Energy or power do we care about total energy consumed of a task, or the per second cost of running a service? Repeat one run is not enough, we need to run our tests multiple times to address background noise. Granularity what level of measurement, how often, and what level of invasiveness of instrumentation can be tolerated? Idle Measurement how do applications and services react with no load: is energy being wasted? Statistical Analysis repeat measures call for summary statistics and comparing distributions. Exceptions errors happen; how do we address them or notice them?

Identifying Fault-prone Files In Large Industrial Software Systems: A common paradigm used at AT&T to develop large systems was to design and create an initial version, and then continue to maintain it with subsequent releases at roughly 3-month intervals. Each new release might contain added functionality, redesign of existing code, removal of parts no longer needed, and fixes for bugs that had been documented since the last release. In short, we could use that information to prioritize our testing efforts to find bugs faster and more efficiently, leading to more reliable software. That is exactly what our software fault prediction algorithms aim to do identify those parts of a software system that are most likely to contain the largest numbers of bugs. To help identify the parts of a system most likely to contain bugs, we needed to identify which file characteristics correlated most closely with files containing the largest numbers of bugs.

A Tailored Suit: The Big Opportunity In Personalizing Issue Tracking- MANY CHOICES, NOTHING GREAT a particular sore point is the weak support provided to developers in building and maintaining a detailed understanding of the current state of their system: what is the current status of the issues? Who is blocking me? How am I blocking others? How many bugs do I need to triage, fix, review, or follow up on? THE NEED FOR PERSONALIZATION Some limitations can be overcome by adding detailed information that is specific to developers and their tasks, and by providing the technical means for developers to create these personalized views themselves.

A Tailored Suit: The Big Opportunity In Personalizing Issue Tracking- DEVELOPER DASHBOARDS OR ATAILORED SUIT Tailor-made suits can be personalized in any number of ways, customization can lead to overfitting, meaning the tool has to change if we change the process. Therefore, using dashboards is a great way to simply blend out the stuff that the team does not use or need, at least at the moment. ROOM FOR IMPROVEMENT Development of next-generation issue tracking that is tailored to individual developers by equipping them with information relevant to their current tasks. Real-time analytics are vital as they allow developers to see the live picture of what is happening with their issues. Support for prioritization by means of enabling developers to set their own or team priorities on issues and tasks and sort them based on their importance. Development of context-aware dashboards that are able to anticipate which information is important for a particular developer at a certain time.

What Counts Is Decisions, Not Numbers Toward An Analytics Design Sheet: DECISIONS EVERYWHERE one wants to select the best alternative based on some evaluation criteria. The alternatives need to be feasible in terms of some hard and soft constraints on cost, time, quality or technical conditions. In the end, all these decisions are made by humans. THE DECISION-MAKING PROCESS people in business must approach decision making with a clear process and plan. We look at this process from an analytics perspective and define what analytics can provide as far as key steps of the decision- making process: Modeling and scoping Information gathering Identify and evaluate alternatives Select one alternative Implement Monitor and adjust to change

What Counts Is Decisions, Not Numbers Toward An Analytics Design Sheet: (Cont.) THEANALYTICS DESIGN SHEET the analytics design sheet (ADS) as a guide for selecting the right analytics to support decision-making. The ADS consist What counts is decisions, not numbers of four quadrants, Q1 to Q4, devoted to the decision problem specified in the heading of the sheet: Context: Description of problem context factors and problem formulation (Q1). Decision: High-level specification of the decision to be made (Q2). Data:Availability of data (Q3). Analytics: Selection of helpful analytics techniques (Q4). EXAMPLE: APP STORE RELEASE ANALYSIS The idea of the ADS is taken as example taken from the domain of app store analytics. With the mass of available explicit and implicit user feedback, synergies between goal- oriented analytics and human expertise are needed to make good decisions. The sample ADS sheet is shown in Fig. 1.

A Large Ecosystem Study To Understand The Effect Of Programming Languages On Code Quality: Comparing languages Comparing programming languages across multiple projects is a non- trivial task. Language choice may depend on many things. For what purpose are you choosing the language, what are the underlying conditions, how big is your team, or who are the programmers? More than that, though, how can the language itself matter? It is the properties of languages that are important. To estimate the language effect on code quality, all these factors need to be considered. Study design and analysis For instance, we focus on projects that are currently under active development, that are widely popular in the open source community, and that are written in somewhat important languages. These choices helped us to mitigate the effect of unmaintained code coming from unskillful development practices. We further excluded some languages from a project that has too few examples within the repository. This ensures that the studied languages have significant activity within the projects.

Code Reviews Are Not For Finding Defectseven Established Tools Need Occasional Evaluation: The process of code reviewing in software engineering is analogous to a plan review in civil engineering or a paper review in academia. In all these, there is the assumption that the quality of output increases by applying differing viewpoints to the subject being reviewed and allowing the author to consider and apply the resulting feedback before finalizing the work. code reviews are a standard part of the modern software engineering workflow. Code reviews take deliberation and are performed by people with a specific set of skills. The social aspect of code reviews cannot be ignored: people s roles on the team and their standing in the team s hierarchy influence the outcome.

Muntimadugu VijayaKanth e-mail: vijaykathphd@gmail.com