Software Failures in Upgrading CA Payroll and DWP Billing Systems

Informatics 43 October 1, 2015 Lecture 1-2 Emily Navarro Duplication of course material for any commercial purpose without the explicit written permission of the professor is prohibited. SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 1 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 2 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 3 Department of Informatics, UC Irvine

Why did the CA payroll system and DWP billing system upgrades fail? A. The task was too large and complex. B. State/city gov t is less efficient than the private sector. C. State/city rules require inadequate programming languages to be used. D. The contractor, SAP Public Services/PricewaterhouseCoopers, has a flawed management structure. E. No one on the software team took Info 43. SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 4 Department of Informatics, UC Irvine

Why did the CA payroll system and DWP billing system upgrades fail? A. The task was too large and complex. B. State/city gov t is less efficient than the private sector. C. State/city rules require inadequate programming languages to be used. D. The software contractor, SAP Public Services/PricewaterhouseCoopers, has a flawed management structure. E. No one on the software team took Info 43. SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 5 Department of Informatics, UC Irvine

What is the real problem? The task is incredibly complex: Payroll system: 160 state departments, 40+ medical/dental plans, $100 millions in costs. DWP: 3.8 million customers Software must conform to the reality of payroll contracts, laws, billing policies. Progress is hard to measure. Part of the goal is to update systems in place since the 1970s. SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 6 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 7 Department of Informatics, UC Irvine

Shopa Failure The company, in a lot of ways, is still trying to figure out what product to build. This leads to massive amount of anxiety and ambiguity as no one knows inside the company what they are building. - There is a lot of unspoken strife between the technical implementation and sales teams - Engineers and Sales team running around as headless chickens. SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 8 Department of Informatics, UC Irvine

Reminder: Top Software Failure Causes Lack of user input/involvement Incomplete requirements and specifications Changing requirements and specifications Lack of discipline in development processes Lack of methodical usage of metrics Lack of resources SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 9 Department of Informatics, UC Irvine

Reminder: Top Software Failure Causes Lack of user input/involvement Incomplete requirements and specifications Changing requirements and specifications Lack of discipline in development processes Lack of methodical usage of metrics Lack of resources Lack of rigor/formality SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 10 Department of Informatics, UC Irvine

Reminder: Top Software Failure Causes Lack of user input/involvement Incomplete requirements and specifications Changing requirements and specifications Lack of discipline in development processes Lack of methodical usage of metrics Lack of resources Requirements issues SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 11 Department of Informatics, UC Irvine

Definition Requirements = what the software should do (without saying how it should do it) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 12 Department of Informatics, UC Irvine

Why Requirements? [We] have grown to care about requirements because we have seen more projects stumble or fail as a result of poor requirements than for any other reason (Kulak and Guiney, in Use Cases: Requirements in Context ) Studies show that many of the key contributors to project failures originate or relate to requirements (The Standish Group CHAOS reports) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 13 Department of Informatics, UC Irvine

Some stats From those CHAOS reports 31% of projects cancelled before they are even completed Many others not delivered or not used ( shelfware ) even if completed Many billions wasted per year on cancelled, unused or unusable projects 52.7% of projects were more than 189% over budget when delivered Requirements defects are expensive They represent more than 70% of rework costs Rework consumes about 30-50% of total project budget Lack of user input/user involvement listed as most frequent problem SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 14 Department of Informatics, UC Irvine

More Stats: Software Life Cycle Costs Analysis 2% Specification 5% Design 6% Maintenance 67% Module Coding 5% Module Testing 7% Integration 8% SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 15 Department of Informatics, UC Irvine

More Stats: Cost of Change Progressively Higher 200 180 160 140 120 100 80 60 40 20 0 Analysis Specification Design Implementation Integration Maintenance SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 16 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 17 Department of Informatics, UC Irvine

Waterfall Req analysis phase Verify Changed requirements Verify Req specification phase Verify Design phase Verify Implementation phase Test Integration phase Test Development Maintenance Operations mode Retirement SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 18 Department of Informatics, UC Irvine

Waterfall Req analysis phase Verify Changed requirements Verify Req specification phase Verify Design phase Verify Implementation phase Test Integration phase Test Development Maintenance Operations mode Retirement SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 19 Department of Informatics, UC Irvine

The RUP Model In an iteration, you walk Phases Process Workflows Inception Elaboration Construction Transition through all workflows Business Modeling Requirements Analysis & Design Implementation Test Deployment Supporting Workflows Configuration Mgmt Management Environment Workflows group activities logically Preliminary Iteration(s) Iter. #1 Iter. #2 Iter. #n Iter. #n+1 Iter. #n+2 Iter. #m Iter. #m+1 Iterations SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 20 Department of Informatics, UC Irvine

Requirements Phase Terminology Requirements analysis/engineering Activity of discovering/observing/gathering customer s needs Requirements specification Activity of describing/documenting customer s needs Note: requirements address what a customer needs, not what a customer wants A customer often does not know what they want, let alone what they actually need Long and arduous, often educational, process And things change under our feet during the requirements process... SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 21 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 22 Department of Informatics, UC Irvine

Techniques for Requirements Analysis Interview customer Create use cases/scenarios Prototype solutions Observe customer Identify important objects/roles/functions Perform research Construct glossaries (Data) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 23 Department of Informatics, UC Irvine

Todays Lecture Software failures Why requirements? Requirements engineering Requirements phase Requirements analysis Requirements specification (documentation) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 24 Department of Informatics, UC Irvine

Requirements Specification Serves as the fundamental reference point between customer and software producer Defines capabilities to be provided without saying how they should be provided Defines the what Does not define the how Defines environmental requirements on the software to guide the implementers Platforms, implementation language(s), Defines constraints on the software Standards, hardware limitations, Defines software qualities maintainability, usability, verifiability SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 25 Department of Informatics, UC Irvine

Non-Functional Requirement Types Non-functional requirements Product requirements Organizational requirements External requirements Efficiency requirements Reliability requirements Portability requirements Interoperability requirements Ethical requirements Usability requirements Delivery requirements Implementation requirements Standards requirements Legislative requirements Performance requirements Space Privacy requirements Safety requirements requirements SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 26 Department of Informatics, UC Irvine

Why Spend a Lot of Time? A requirements specification is the source for all future steps in the software life cycle Lays the basis for a mutual understanding Consumer (what they get) Software producer (what they build) Identifies fundamental assumptions Better get it right Upon delivery, some software is actually rejected by customers Changes are cheap Better make them now rather than later SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 27 Department of Informatics, UC Irvine

Users of a Requirements Document Specify the requirements and read them to check that they meet their needs. They specify changes to the requirements System customers Use the requirements document to plan a bid for the system and to plan the system development process Managers Use the requirements to understand what system is to be developed System engineers Use the requirements to develop validation tests for the system System test engineers Use the requirements to help understand the system and the relationships between its parts System maintenance engineers SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 28 Department of Informatics, UC Irvine

Document Structure Introduction Executive summary Application context Environmental requirements Functional requirements Software qualities Other requirements Time schedule Potential risks Assumptions Future changes Glossary Reference documents SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 29 Department of Informatics, UC Irvine

Introduction What is this document about? Who was it created for? Who created it? Outline SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 30 Department of Informatics, UC Irvine

Executive Summary Short, succinct, concise, to-the-point, description Usually no more than one page Identifies main goals Identifies key features Identifies key risks/obstacles SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 31 Department of Informatics, UC Irvine

Application Context Describes the situation in which the software will be used Home, office, inside, outside, Identifies all things that the system affects Objects, processes, other software, hardware, and people SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 32 Department of Informatics, UC Irvine

Environmental Requirements Platforms Hardware Operating systems, types of machines, memory size, hard disk space Software Is it a Web app? Mobile app? Desktop app? Is it open source? Linux? Apache? PHP/MySQL? Is it enterprise software? .Net? Enterprise Java, J2EE? Programming language(s) Standards SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 33 Department of Informatics, UC Irvine

Functional Requirements Identifies all concepts, functions, features, and information that the system provides to its users Provides an abstraction for each of those, characterizing the properties and functions that are relevant to the user What is the system supposed to do? What information does the system need? What is supposed to happen when something goes wrong? SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 34 Department of Informatics, UC Irvine

Desired Software ilities (Qualities) Correctness Reliability Efficiency Integrity Usability Maintainability Portability Reusability Interoperability Robustness Security This section helps developers assess tradeoffs in the system s implementation SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 35 Department of Informatics, UC Irvine

Other Requirements What about cost? What about documentation? What about manuals? What about tutorials? What about on-the-job training? What about requirements that do not fit in any of the previous categories? SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 36 Department of Informatics, UC Irvine

Time Schedule By when should all of this be done? Initial delivery date Acceptance period Final delivery date What are some important milestones to be reached? Architectural design completed Module design completed Implementation completed Testing completed SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 37 Department of Informatics, UC Irvine

Potential Risks Risks: future uncertain events with a probability of occurrence and a potential for loss (softwaretestinghelp.com) Any project faces risks new methodology requirements new to the group special skills and resource shortage aggressive schedule tight funding It is important to identify those risks up-front so the customer and you (!) are aware of them One of the requirements could be to explicitly address the risks SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 38 Department of Informatics, UC Irvine

Assumptions Factors that are believed to be true during the life cycle of the project If changed, they may affect the project outcomes negatively Examples end-user characteristics known technology infrastructure resource availability funding availability SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 39 Department of Informatics, UC Irvine

Future Changes Any project faces changes over time It is important to identify those changes up-front so the customer and you (!) are aware of them These changes could simply pertain to potential future enhancements to the product One of the requirements could be to build the product such that it can accommodate future changes Note: structure the requirements document in such a way that it easily absorbs changes Define concepts once Partition separate concerns Avoid redundancy SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 40 Department of Informatics, UC Irvine

Glossary Precise definitions of terms used throughout the requirements document SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 41 Department of Informatics, UC Irvine

Reference Documents Pointers to existing processes and tools used within an organization Pointers to other, existing software that provides similar functionality Pointers to literature SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 42 Department of Informatics, UC Irvine

Observations Document is structured to address the fundamental principles Rigor Separation of concerns Modularity Abstraction Anticipation of change Generality Incrementality Not every project requires every section of the document These principles apply to all aspects of software engineering SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 43 Department of Informatics, UC Irvine

Specification Methods Natural language Data flow diagrams Office automation Finite state machines Telephone systems Coin-operated machines Petri nets Production plants Formulas Objects (in object-oriented methods) Use cases (in UML) SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 44 Department of Informatics, UC Irvine

Verification Is the requirements specification complete? Is each of the requirements understandable? Is each of the requirements unambiguous? Are any of the requirements in conflict? Can each of the requirements be verified? Are are all terms and concepts defined? Is the requirements specification unbiased? SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 45 Department of Informatics, UC Irvine

Acceptance Test Plan Accompanies a requirements specification Specifies, in an operational way, consistency between the requirements specification and the system that will be delivered Binds a customer to accept the delivered system if it passes all the tests Covers all aspects of the requirements specification May include: some specific test cases the number of test cases that must pass SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 46 Department of Informatics, UC Irvine

Videos Requirements gathering 1: https://www.youtube.com/watch?v=l_GTTyE9i9Y Requirements gathering 2: https://www.youtube.com/watch?v=lXNu0VBVCUc SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 47 Department of Informatics, UC Irvine

Quiz Tuesday, October 6 Closed book, no notes, calculators, phones Covers all readings and lectures through Thursday 10/1 (today) No scantrons, no blue books How to study: Different from other CS courses Software engineering as much about people as it is about software Shifts away from technical thinking of a CS student Many ways to analyze topics, especially definitions, links between different concepts Attend lecture, take notes, spend time going over them carefully, analyzing, discussing Do readings carefully, take notes, analyze, and discuss Focus more on high-level understanding of main points than details of concepts SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 48 Department of Informatics, UC Irvine

Quiz Tuesday, October 6 - Topics Memorize one definition of software engineering (word for word) 3 essential ingredients of software engineering Know and understand the 3 perspectives on software engineering we talked about Know and understand the Inf43 Recurring, Fundamental Principles of software engineering, and the overall ideas behind the other principles No Silver Bullet Know and understand the essential difficulties of software engineering Know and understand the potential silver bullets on the essential difficulties SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 49 Department of Informatics, UC Irvine

Quiz Tuesday, October 6 - Topics Know and understand software failure causes and how they relate to requirements issues Know and understand the main ideas of the online failure readings Make sure you have watched the videos I show in class Textbook: High-level understanding of the readings The quiz will focus on these topics, but I reserve the right to ask about any other lecture/reading information as well SDCLCollaboration Laboratory Software Design and sdcl.ics.uci.edu 50 Department of Informatics, UC Irvine