Basic definitions
An overview of software failures, defects, and errors, along with the importance of inspections in identifying defects. Learn about different roles in inspection teams and principles to follow for effective inspections.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Basic definitions A failure is an unacceptable behaviour exhibited by a system The frequency of failures measures the reliability An important design objective is to achieve a very low failure rate and hence high reliability A failure can result from a violation of an explicit or implicit requirement A defect is a flaw in any aspect of the system that contributes, or may potentially contribute, to the occurrence of one or more failures could be in the requirements, the design and the code It might take several defects to cause a particular failure An error is a slip-up or inappropriate decision by a software developer that leads to the introduction of a defect into the system 2/20/2025 Lecture 9 1
Exercise Categorize the following according to whether each describes a failure, a defect or an error: 1. A software engineer working in a hurry, unintentionally deletes an important line of source code 2. On 1 January 2040 the system reports the date is January 1940 3. No design documentation or source code comments are provided for a complex algorithm 4. A fixed size array of length 10 is used to maintain the list of courses taken by a student during one semester. The requirements are silent about the maximum number of courses a student may take at any one time 2/20/2025 Lecture 9 2
Inspections An inspection is an activity in which one or more people systematically Examine source code or documentation, looking for defects. Normally, inspection meeting... Although participants can also inspect alone at their desks involves a 2/20/2025 Lecture 9 3
Roles on inspection teams The author The moderator Calls and runs the meeting Makes sure that the general principles of inspection are adhered to The secretary Responsible for recording the defects when they are found Must have a thorough knowledge of software engineering Paraphrasers Step through the document explaining it in their own words 2/20/2025 Lecture 9 4
Principles of inspecting Inspect the most important documents of all types code, design documents, test plans and requirements Choose an effective and efficient inspection team between two and five people Including experienced software engineers Require that participants prepare for inspections They should study the documents prior to the meeting and come prepared with a list of defects Only inspect documents that are ready Attempting to inspect a very poor document will result in defects being missed 2/20/2025 Lecture 9 5
Principles of inspecting Avoid discussing how to fix defects Fixing defects can be left to the author Avoid discussing style issues Issues like are important, but should be discussed separately Do not rush the inspection process A good speed to inspect is 200 lines of code comments) or ten pages of text per hour per hour (including 2/20/2025 Lecture 9 6
Principles of inspecting Avoid making participants tired It is best not to inspect for more than two hours at a time, or for more than four hours a day Keep and use logs of inspections You can also use the logs to track the quality of the design process Re-inspect when changes are made You should re-inspect any document or code that is changed more than 20% 2/20/2025 Lecture 9 7
A peer-review process Managers are normally not involved This allows the participants to express their criticisms more openly, not fearing repercussions The members of an inspection team should feel they are all working together to create a better document Nobody should be blamed 2/20/2025 Lecture 9 8
Conducting an inspection meeting 1. The moderator calls the meeting and distributes the documents 2. The participants prepare for the meeting in advance 3. At the start of the meeting, the moderator explains the procedures and verifies that everybody has prepared 4. Paraphrasers take turns explaining the contents of the document or code, without reading it verbatim Requiring that the paraphraser not be the author ensures that the paraphraser say what he or she sees, not what the author intended to say 5. Everybody speaks up when they notice a defect 2/20/2025 Lecture 9 9
Software Testing Definition: Testing is a process of executing a program with data with the sole intention to find errors in the program Can only reveal the PRESENCE of ERRORS, and NOT the ABSENCE of ERRORS A successful test is a one which discovers more, yet-undiscovered, errors Consequence: An ideal testing process will never terminate 2/20/2025 Lecture 9 10
Effective and Efficient Testing To test effectively, you must use a strategy that uncovers as many defects as possible. To test efficiently, you must find the largest possible number of defects using the fewest possible tests Testing is like detective work: The tester must try to understand how programmers and designers think, so as to better find defects The tester must not leave anything uncovered, and must be suspicious of everything It does not pay to take an excessive amount of time; tester has to be efficient 2/20/2025 Lecture 9 11
Objectives of testing To find more errors in the design and implementation of a software product To conform that the product meets (satisfies) the requirements To conform performance requirements Testing is one type of validation Validation is a process that answers to the question Are we developing the right product? Consequence: The product must be ready (available) in some form before exercising the testing process 2/20/2025 Lecture 9 12
Two Types A fault of commission: representation is incorrect A fault of omission: fail to enter correct information, example: missing a requirement Specification fault of omissions fault of commission Correct Code Implementation 2/20/2025 Lecture 9 13
Relationship Specification (expected) Program (observed) P 6 S 5 2 1 3 4 7 T Testing 2/20/2025 Lecture 9 14
Testing and Debugging Definitions: Testing is a process of finding the presence of errors Debugging is a process of finding the location and cause of errors Debugging requires testing efforts to find the errors first, and then retest the program after correcting the errors Thus, testing occurs before and after debugging Testing may be done manually, but debugging will be done effectively with the help of tool support 2/20/2025 Lecture 9 15
Testing Process Testing activities: test planning test case development running test cases evaluating test results 2/20/2025 Lecture 9 16
Test case Definition: A test case is a description of input/output relationship of one particular testing activity Example: Let y = f(x) be the function to be tested; a test case will indicate for what values (range of values) of x the function will promptly return an output, and for what values the function will not return an output Generally expressed as a set of predicates (boolean expressions) 2/20/2025 Lecture 9 17
Test Data Definition: Test data is a set of values that are used to test a program Example: In the y = f(x) example, test data could be x = 12.34, x = 14.56, x = -0.3487 etc. Test data are generally derived from the application domain, particularly from the requirements 2/20/2025 Lecture 9 18
Test Case Information Test Case ID Purpose Preconditions Inputs Expected Outputs Postconditions Execution History Date Result Version Run By 2/20/2025 Lecture 9 19
Testing principles All tests should be traceable to requirements Otherwise, testing efforts will not be useful Testing activity should be planned well before testing process is exercised Requires preparation of test cases, test data and test oracles Exhaustive testing is not possible Often, testers are required to select (may be randomly) a subset of test cases to exercise To be most effective, testing should be conducted by a third party not by the developers Quality Assurance team 2/20/2025 Lecture 9 20
Test case design methods Definitions: Black-box testing: Program is assumed to be a black-box and test cases are derived from sources other than the program itself; such sources can be requirements document, design document, problem description etc. White-box testing: Test cases are derived from the actual source code of the program Both black-box and white-box testing are required for a successful testing process 2/20/2025 Lecture 9 21
Black-box and White-box testing Test cases can be derived before the program is written Objective is to ensure that the program meets (satisfies) the requirements Needs knowledge of the application domain but nothing about the program or the programming language Test cases can be derived only after the program is complete Objective is to ensure that the logic of the program is correct Needs knowledge of the programming knowledge, and may not need application domain information 2/20/2025 Lecture 9 22
White-box testing Also called glass-box testing and structural testing Derivation of test cases from the program (source code); knowledge of the application domain may be used to derive additional test cases 2/20/2025 Lecture 9 23
White-box Testing Identify test cases based on how the program is actually implemented Test coverage metrics Provide a way to explicitly state the extent to which a software item has been tested Make the testing management more meaningful 2/20/2025 Lecture 9 24
Test Coverage Metrics Goal Derive sufficient number of test cases to cover one or more of the following: Statement Coverage Branch Coverage / Condition Coverage Path Coverage 2/20/2025 Lecture 9 25
Example computing factorial int factorial (int n) { int i, ret = 0; if (n < 0) { return ret; } i = n; ret = 1; while (i > 0) { ret = ret * i; i = i - 1; } return ret; } 2/20/2025 Lecture 9 26
Example computing factorial (continued) Test cases: case 1: n is negative; return value must be 0 case 2: n is positive; return value must be n! Test data: n = -2, n = 5 Test oracles run the program for the above test data, get the actual outputs and check against the expected outputs shows in test cases 2/20/2025 Lecture 9 27
Is Statement Coverage Sufficient? Example: Consider the following part of the code that is supposed to compute the absolute value of y: if (y >= 0) abs = y; then y = 0 - y; 2/20/2025 Lecture 9 28
Branch Coverage / Condition Coverage Branch coverage: focus on testing branches in code Test cases are selected in such a way that each branch in the code is executed at least once. This requires that each decision box is evaluated to be true and false at least once. 2/20/2025 Lecture 9 29
Branch Coverage / Condition Coverage Example: If ((x < -5 ) && (y > 11)) z = compute (x,y); else z = compute_altern (x,y); Consider the test cases: x = -4, y = 10 and x = -6 , y = 12 2/20/2025 Lecture 9 30
Another Example Consider: No Yes X> 0 z = someF2(x) z = someF1(x) Test cases: x = 2, z = 6 Yes No z > 10 x = 0, z = 12 z = doSth1(); z = doSth2(); 2/20/2025 Lecture 9 31
Path coverage Path coverage: considers all possible logical paths in a program and leads to test cases aimed at exercising a program along a path. the use of the path coverage criterion may help detect faults easily omitted by branch coverage testing. However, in many cases, this criterion can be too impractical, especially when it comes to loops in the program that may easily lead to a very high number of paths. 2/20/2025 Lecture 9 32
Loop Coverage Loop Coverage Ensure that every loop is covered for all possible execution scenarios Loop skipped Loop is iterated through at the first time Loop is iterated through many times Loop is iterated at its last time Independent loops are easier to test than nested loops 2/20/2025 Lecture 9 33
Flow graph notation Also called program graph Describes the logical control flow in the program Consists of nodes (represented as circles) and transitions (represented by arrows) nodes represent executable statements or expressions transitions represent control flow 2/20/2025 Lecture 9 34
sequence Do-while if if-else while Flow graph notations Switch-case for 2/20/2025 Lecture 9 35
Try-catch example Line1 int age; Line2 try { Line3 Line4 Line5 Line6 System.out.print( Enter your age: ); age = Keyboard.readInt(); if(age < 0 || age > 120) { throw new Exception( Invalid age: + age); } Line7 System.out.println( Your age is + age); } Line8 catch (Exception e) { Line9 System.out.println(e.getMessage()); } Line10 System.out.println( Done. ); 2/20/2025 Lecture 9 36
Flow graph for the factorial example ret = 0 1 3 n < 0 return ret 2 Notice that this is a special case; return is a terminal statement and so the arrow goes out rather than connecting back to the next statement i = n; ret = 1 4 while (i > 0) 5 ret = ret * i; i = i - 1 6 7 return ret 2/20/2025 Lecture 9 37
Test cases: Case 1: n < 0 will cover statements 1, 2, 3 and return 0; Case 2: n = 0 will cover statements 1, 2, 4, 5, 7 and return 1. Case 3: n > 0 will cover statements 1, 2, 4, 5, 6, 7 and return n! All statements are executed at least once with these test cases. 2/20/2025 Lecture 9 38
Another example To remove an integer from an array and squeeze the array; add 0 at the end void squeeze (int k) { boolean flag = false; int j, index = 0; while (index < arr.length && !flag) { if (arr[index] == k) { for (j = index; j < arr.length-1; j++) arr[j] = arr[j+1]; arr[arr.length-1] = 0; flag = true; } else index = index + 1; } } Draw the flow graph for this example line1 line2 line3 line4 line5 line6 line7 line8 line9 Identify representative path/independent path 2/20/2025 Lecture 9 39
Cyclomatic complexity Definition: Cyclomatic complexity is a software metric (quantitative) that denotes the number of independent paths derived from the flow graph Definition: An independent path in a flow graph is any path that introduces at least one new set of processing statements or a new conditional expression In terms of flow graph, there must be at least one new edge that is not traversed. Example: for the factorial method, it is 3. 2/20/2025 Lecture 9 40
Computing cyclomatic complexity Gives a measure of number of test cases to be considered Cyclomatic complexity V(G) of a flow graph G is defined as V(G) = E N + 1 where E denotes number of edges, and N denotes number of nodes Check this formula for the factorial and squeeze methods 2/20/2025 Lecture 9 41
Exercise Cyclomatic complexity differences 2/20/2025 Lecture 9 42
Comparing White-box testing methods Specification Program Specification Program Test Cases (Method A) Test Cases (Method B) 2/20/2025 Lecture 9 43
Questions Are branch and path coverage the same? Is 100% coverage the same as exhaustive testing? When can I stop testing? 2/20/2025 Lecture 9 44
Black-box testing Testers provide the system with inputs and observe the outputs They can see none of: The source code The internal data Any of the design documentation describing the system s internals 2/20/2025 Lecture 9 45
Example for black-box testing An automated teller machine (ATM) consider the functional requirement withdraw test cases for this function will include withdraw an amount within the balance withdraw an amount equal to the balance withdraw an amount more than the balance ??? withdraw a negative amount (though not applicable in practice, you can do so in software) no matter how the withdraw function is implemented, these test cases will effectively test whether the implementation is satisfactory with regard to the requirements 2/20/2025 Lecture 9 46
Objectives of black-box testing Identify incorrect or missing functionalities Identify errors in interfaces among the components of the software Identify errors in links with other programs such as databases a special case of the previous objective Identify performance errors Identify initialization and/or termination errors 2/20/2025 Lecture 9 47
Equivalence partitioning A method to generate test data for black- box testing Divides the input domain (the set of values that can be used as input) into mutually disjoint subsets these subsets are called equivalence classes Each member in a subset has the same characteristic as that of the other elements in the same subset consequence: it is sufficient to select one data from each subset reduce the time for testing 2/20/2025 Lecture 9 48
Mathematical Definition of Equivalence Classes Given a set S, the equivalence classes of S are the subsets S1, S2, , Sn such that S1 S2 Sn =S Si Sj = {}, for 1<i, j<=n, i j The first condition asserts that the subsets are partitions of the given set S The second condition asserts that the subsets are mutually exclusive 2/20/2025 Lecture 9 49
Input domain Software system Important: Equivalence partitioning does not impose any constraints on the output domain Output domain 2/20/2025 Lecture 9 50
