Revolutionizing Workplace Innovation with Collaborative Solutions
Empower organizations to drive workplace innovation through collaborative thinking. Our agile frameworks foster organic growth and a consumer-first mindset. Addressing a market gap, our product uniquely caters to Gen Z, offering cost savings and easy usability. We are the first to market with a stylish and tested product designed with expert input. Enhance community connections with our cool and stylish product, available through our online store and market swap. Utilize our abstract business model based on market trends and research, focusing on minimalist design and user-friendly products. Join us in tapping into a multi-billion-dollar market, offering freedom to invent and build with few competitors.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Experiments in Benchmarking Software Test Cases Produced by Generative AI ITEA AI in T&E Workshop March 19, 2025 Dr. Mark L. Gillenson MIS Department University of Memphis Mark.Gillenson@memphis.edu Dr. Pavankumar Mulgund MIS Department University of Memphis pmulgund@memphis.edu Dr. Ankur Arora MIS Department University of Memphis aarora3@memphis.edu
Systems Testing Excellence Program (STEP) Created in 2006 in collaboration with the FedEx Corp. A unit of the FedEx Institute of Technology at the University of Memphis. Includes faculty of the University of Memphis plus faculty of several other universities. Most have industrial backgrounds. Is the largest academic organization devoted exclusively to software testing. Has performed training and research projects for industry, DOD (JITC), and DHS. step.memphis.edu Systems Testing Excellence Program FedEx Institute of Technology University of Memphis
Background, Goal, and Method Background: Most efforts involving AI and T&E are aimed at exploring how to test AI applications. Goal: Our project s goal was to discover whether AI, specifically Generative AI, can be used to improve the efficiency and effectiveness of testing any application. Method: Benchmark the efficiency and effectiveness of test cases created by Generative AI against test cases created by traditional methods, in their ability find defects in code. Systems Testing Excellence Program FedEx Institute of Technology University of Memphis
This Project Created specifications for two basic applications: package shipping pricing and payroll, and coded them in Python. Created three sets of test cases: Manually (TFD), Pairwise Analysis (MS PICT), ChatGPT4o. Devised ten defects for each application. Inserted the defects one-at-a-time and executed the applications with each set of test cases to find which sets of test cases found the most defects. Increased the complexity of the now enhanced applications, revised the defects, and repeated the process. Systems Testing Excellence Program FedEx Institute of Technology University of Memphis
Results Results for the basic applications were mixed and inconclusive. For the enhanced applications we ran four sets of test cases: TFD, MS PICT, and ChatGPT4o with and without a sample test case. Results for the enhanced package application were mixed. Results for the enhanced payroll application were very encouraging. ChatGPT4o with and without a sample test case performed substantially better than TFD and MS PICT. Systems Testing Excellence Program FedEx Institute of Technology University of Memphis
Contributions and Future Research We believe we have made a novel contribution to the question of whether Generative AI can be an aid in software testing in general. Future Research: Continue to conduct experiments of this type with more complex applications. Experiment with different kinds and combinations of Generative AI prompts and attachments. Experiment with trying to have Generative AI determine the expected output of test cases. Systems Testing Excellence Program FedEx Institute of Technology University of Memphis
