Software Engineering and Testing: Maximizing Technology for Human Capability
Software engineering involves leveraging technology to optimize human potential. Jeff Offutt, a professor of software engineering, emphasizes the importance of creating reliable software to prevent catastrophic failures like those experienced by NASA and Toyota. Testing plays a critical role in ensuring software quality and safety, with researchers constantly innovating new methods to detect faults before they impact users. Learn about the significance of software testing and how it saves time and resources in the long run.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
(some of) My Research Engineering is about getting technology to do what it does well so humans can do what they do well Jeff Offutt Professor of Software Engineering https://www.cs.gmu.edu/~offutt/
Software is the nervous system of our civilization Quote partially due to Dr. Mark Harman of 15 CS 700 September 2018 2 Jeff Offutt
Spectacular software failures NASA s Mars lander : September 1999, crashed due to a units integration fault Ariane 5: exception-handling bug : forced self destruct on maiden flight (64-bit to 16- bit conversion: about 370 million $ lost) Mars Polar Lander crash site? Toyota brakes : Dozens dead, thousands of crashes Major failures : Ariane 5 explosion, Financial engines, Intel s Pentium FDIV bug Poor testing of safety-critical software can kill : THERAC-25 radiation machine: 3 dead Northeast blackout : 50 million people, $6 billion USD lost alarm system failed THERAC-25 design Software testers try to find faults before the faults find users of 15 CS 700 September 2018 3 Jeff Offutt
What is testing & why is it hard? A program can have quadrillions of inputs. And will work correctly on most! If a million inputs trigger the fault. Only .00000000025 chance of finding it randomly ... Software testers design test inputs to trigger faults before they get to users of 15 CS 700 September 2018 4 Jeff Offutt
Cost of testinglate 60 50 40 30 20 10 Fault origin (%) Detected (%) Unit cost (X) 0 Unit testing saves a LOT of money of 15 CS 700 September 2018 5 Jeff Offutt
What do testing researchers do? Software testing researchers invent clever ways to design inputs that will find software faults Build software tools to create tests Run experiments to find out how well the testing ideas work Tell industry how to use the ideas Teach students the next generation of software testers of 15 CS 700 September 2018 6 Jeff Offutt
A few recent PhD students Towards automatically localizing and repairing SQL faults, 2018, Cvent Generating cost- effective criteria-based tests from behavioral models, 2014, Medidata Solutions Mutation testing for android apps, 2017, Towson University Towards evasive attacks: Anomaly detection resistance analysis on the Internet, 2013, Intuit Testing web applications with mutation analysis, 2017, University of Virginia Testability of dynamic real-time systems, 2009, University of Sk vde of 15 CS 700 September 2018 7 Jeff Offutt
Model-based testing process Model Criterion Test criterion Test Requirements Test requirements (subpaths) Abstract Tests Abstract tests (test paths) Concrete tests Concrete Tests Extra Info Test oracles Test Execution Test Reports of 15 CS 700 September 2018 8 Jeff Offutt
Test oracle strategies 28,881,000 test executions Edge coverage tests on FSM models % faults found that the oracle revealed 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 NOS SIOS OS1 OS2 OS3 OS5 OS6 More precise OSes are not always more effective of 15 CS 700 September 2018 9 Jeff Offutt
Test oracle strategy findings Crash testing (NOS) wastes testing effort Penny wise, pound foolish We should check state invariants (SIOS) Only one check needed Can be partially derived automatically from the model A lot of JUnit test assertions are broken they do not check the right thing Joint work with Dr. Nan Li of 15 CS 700 September 2018 10 Jeff Offutt
Current researchsmart tests Old style tests Values invented by humans Scripts were pieces of paper with steps Turn on computer Type : Run myProgram Enter name : George P. Burdell Enter age : -25 Simple directions to humans Slow! Error prone! Limited repeatability! These tests are as dumb as single-cell organisms !! Almost impossible to use test criteria of 15 CS 700 September 2018 11 Jeff Offutt
Modern dumb tests Test values Created by a mix of humans and test data generators Satisfy well-documented goals, test criteria, or specialized domain needs Integrated into automated test scripts (eg, JUnit) Includes a small amount of brain power these tests know what results to expect (eg, JUnit assertions) Fast repeatable These multi-cellular tests show the first signs of intelligence! of 15 CS 700 September 2018 12 Jeff Offutt
Multicellular tests But this test does not know Before values Test values After values Why is it there? Expected results When should it run? When should it change? When should it die? of 15 CS 700 September 2018 13 Jeff Offutt
Intelligent tests Self-awareness Self-determination Each test should encode traceability what it covers Tests should check what has changed, and rerun if necessary Tests should alert tester when they no longer match the software Tests should quietly go away when no longer needed of 15 CS 700 September 2018 14 Jeff Offutt
Googles motto Debugging sucks Testing rocks Jeff Offutt https://cs.gmu.edu/~offutt/ of 15 CS 700 September 2018 15 Jeff Offutt
