Effective Test Requirements Matter

Test Requirements: Why Requirements Matter Amadori Courses: Delivering an Effective Test Process 1

Without clear requirements effective testing is NOT possible After all . Introduction Amadori Courses: Delivering an Effective Test Process 2

if no one can tell US what an application is supposed to do .. Introduction Amadori Course 3

if no one can tell US what an application is supposed to do .. Introduction How can we tell THEM if it is doing it correctly? Amadori Course 4

What is a testable Requirement ? Amadori Course 5

A requirement is testable if you can write a test against it, What is a testable Requirement ? which if executed by 2 separate individuals at the same time would give the same result. Amadori Course 6

1. The application must support normal levels of usage 2. The application must be able to process 10,000 transactions in one hour Testable or Non- Testable 3. The application should provide timely and accurate performance metrics Amadori Courses: Delivering an Effective Test Process 7

1. The application must support normal levels of usage 2. The application must be able to process 10,000 transactions in one hour 3. The application should provide timely and accurate performance metrics Testable or Non- Testable All are non-testable in their current form . Amadori Courses: Delivering an Effective Test Process 8

We ask questions How do we make them testable? We obtain clarifications We agree what assumptions should be used when writing tests and executing them WE AGREE ACCEPTANCE CRITERIA Amadori Courses: Delivering an Effective Test Process 9

The application must support normal levels of usage From Non- Testable to Testable (1) Agree a definition of normal in terms of Number of transactions being processed at any one time Type of transactions being processed The setup and capacity of the production environment Validate your test results on this basis Amadori Courses: Delivering an Effective Test Process 10

The application must be able to process 10,000 transactions in one hour What types of transactions are to be included for this test? What else is running on the system at the same time? How do we validate if the transactions have been processed correctly? How do we count the number of transactions processed? What are the start and end points of the test? From Non- Testable to Testable (2) Validate your test results on this basis Amadori Courses: Delivering an Effective Test Process 11

The application should provide timely and accurate performance metrics Agree a definition of timely with stakeholders Agree a definition of accurate with stakeholders Test on this basis From Non- Testable to Testable (3) Amadori Courses: Delivering an Effective Test Process 12

Testers do not define the detail here Stakeholders must tell us WHAT they expect HOW we should determine if their criteria are met Remember We may not agree with their goals BUT Provided their requirements are clear and the criteria for acceptance agreed that s what we need to test Amadori Courses: Delivering an Effective Test Process 13

Testers are provided with a clear specification in which every single requirement is testable Ideal World Situation Amadori Courses: Delivering an Effective Test Process 14

Things are rarely so simple Requirements arrive late or not all Requirements are ambiguous contradictory or incomplete Requirements are subject to change throughout the delivery lifecycle Is this just the way things have to be? Or is there anything that testers can do to improve the situation? And in the Real World Amadori Courses: Delivering an Effective Test Process 15

Amadori Courses: Delivering an Effective Test Process Is this just the way things have to be? And in the Real World Or is there anything that testers can do to improve the situation? 16

Step 1: work out WHY there are issues with requirements Step 2: assess whether testers can help in addressing the underlying issKey Addressing the situation Amadori Courses: Delivering an Effective Test Process 17

Possible Root Causes 1. The requirements have not been defined 2. The requirements are defined but not documented 3. Different parts of the project have conflicting requirements 4. Requirements are only defined at a high level but not in detail Amadori Courses: Delivering an Effective Test Process 18

Ask stakeholders the following questions What do you like about the current system? What don t you like? What benefits are you expecting from the new system? Don t simply ask them to define requirements Document their responses and send out this document for review How to address these issues Amadori Courses: Delivering an Effective Test Process 19

Why this review process is important Asking for this review will 1. Highlight differences of opinion 2. Identify gaps in what is being proposed 3. Show where the focus of delivery is misplaced 4. Provide a firm basis for later assessing whether the software is ready to go-live Amadori Courses: Delivering an Effective Test Process 20

Also Provide examples to stakeholders of what you think is being asked for A single worked example can often answer resolve many issues Ask stakeholders to provide real-life examples of things which Work well Do not work well Hold workshops at which stakeholders explain what They currently do What they expect to do when the system goes live Amadori Courses: Delivering an Effective Test Process 21

A Worked Example Hopefully the next few minutes will show you that it IS possible to extract testable requirements from even the least promising source .. Amadori Courses: Delivering an Effective Test Process 22

The Scenario A change which was cancelled has brought back from the dead and is required at very short notice. Amadori Courses: Delivering an Effective Test Process 23

The Scenario A change which was cancelled has brought back from the dead and is required at very short notice. No formal spec. is available just some handwritten notes jotted down by the development manager during a conference call the previous week Amadori Courses: Delivering an Effective Test Process 24

Your Spec Amadori Courses: Delivering an Effective Test Process 25

Your Mission Review the specification List what you believe to be the requirements here State which of these requirements you believe are testable and which are non testable List any assumptions you make to allow this area to be testable Are there areas where more information is required before you can progress? Amadori Courses: Delivering an Effective Test Process 26