Software Engineering Project Management: Risk Quantification Basics

Project Management for Software Engineers (Summer 2017) LECTURE 12 Risk Quantification Basics July 19, 2016 (9:00 am 11:40 pm PST) 1 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Risk = Uncertainty Risk quantification means: How much uncertain are we about a particular outcome Supplier Input Process Output Customer: Focusing on the I.P.O: If we know how much uncertain we are about the Inputs, we can agglomerate all uncertainties to calculate the uncertainty about the output, using statistical techniques (simulation) 2 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

A Crash Course on Probabilities A + Ac = 1 P(A or B) = P(A) + P(B) P (A and B) Mutually Exclusive events: P(A and B) =0 Conditional Probability P(A|B) = P(A and B) / P(B) P(A and B) = P(A|B)xP(B) Joint Probabilities: Decision Tree (pp. 257-258) Independent Events: P(A|B)=P(A) P(A and B) =P(A)xP(B) 3 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

A Crash Course on Statistics Descriptive Statistics Current Data attributes (central tendency & dispersion) Histograms: Location, Spread, Shape Inferential Statistics (Enumerative Studies) Sample Population (Hypothesis Testing) Key: Proper Sampling Representative Sample Predictive Statistics (Analytic Studies) DOE, Regression, ANOVA 4 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Descriptive Statistics Central tendency: Mean, median, mode Dispersion: Variance, Min, Max, Range Shape: Histogram, Skewness, Kurtosis Others: Count, Sum Excel Example: Tools Data Analysis Descriptive Statistics Tools Data Analysis Histogram 5 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Statistical Distributions Discrete: Binomial, Hypergeometric, Poisson, Continuous: Normal, Lognormal, Exponential, Interrelationships Binomial Poisson Normal Lognormal Poisson Exponential Sources of abnormality: Recording & Measurement errors, Multiplicity, truncation, natural skewness, tampering (Deming experiments) Understand data & sources of variation before sampling 6 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Inferential Statistics Sampling: Select an unbiased subset of an entire group Simple Random Sampling (SRS): For homogeneous sets Stratified sampling: Group by some attributes (internally homogeneous) Clustering: Similar groups, different components ( Naturally clustered, internally heterogeneous) Less Precise & More Economic Systematic: Every (N/n)th sample, or some other rules (Easy, but prone to bias) Examples: Random, NOT arbitrary! Stratified: Sampling City population, in age groups Stratified: Exponential Distribution Clustered: Sampling random pallets of packed products Systematic: Sampling every 8th house of a street with 120 houses

Representative Samples If the sampling is performed properly, the sample represents the population Parameter Parameter Sample Sample Size Size n Mean Mean x x St. Dev. St. Dev. s s Population Population N N m m s s CLT For very large population (n/N<0.05) and Truly representative samples: E(x x ) = m m & s s(x x )=s s / n x x ~ N ( m m , , s s / n ) Example: P. 280, Examples 6.13 & 6.14 N(x, m m, s s) == Ns[(x-m m)/s s, 0, 1] In CLT: In CLT: x x ~ Ns [(x x --m m) )/ /( (s s / n ),0,1] University of Southern California, Industrial & Systems Engineering Note: Ns Table (pp.A3&A4) 8 4/19/2025

From Point To Range Estimate Confidence Level (CL) = 1 - a a a a = Probability of a parameter falling outside a given range. (e.g. probability of defect) Example: 90% CI for m m is x x + q q means In 100 samples, for 90 of them: the range [x x - q qx x + q q ] contains the true value of m m Equations: P. 281 Examples 6.15 & 6.16 (p. 282) 9 University of Southern California, Industrial & Systems Engineering 4/19/2025

What is simulation? Problem: Flipping a coin as many times as we get 3 heads; Cost: $1 per coin-flip, reward: $8 for 3 heads Can you develop a formula to get the answer? Yes you can, but it s a tedious and complicated process. Can it be done easier? Just walk the talk: Generate a random integer (0, 1) Excel: Int((Rand()+0.5)) Repeat until you get three 1 s (heads) If we repeat this experiment many many times, we can develop a statistical distribution for the sample size (i.e. number of flips that returns 3 heads) Estimating reality by experimentation Example calculations in Excel (handout) 10 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Monte-Carlo Simulation Process: If Y= (X1,X2,X3, ) and statistical distribution of Xi s are known as F(Xi), then: U = F(Xi) Xi = F-1(U), where U is a probability, a random number between 0 and 1 (100%) With numerous iterations, statistical distribution of Y and its statistical characteristics can be modeled and the probability of Y<=Y0 can be estimated with a very high accuracy The more iterations, the more accurate the statistical distribution for Y For each Xi: For the entire model (Y): N=100 N=200 N=500 N=1000 Source: Queueing Methods, Randolph W. Hall (1991), P. 59 11 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

A Basic Example A Typical customer service: Poisson Process ? ? = 1 ? l?, SET: R[0 1] = ? ? ? =ln ? l Assume l=4 customers / hour based on historical data How many customers will arrive in 2 hours? Trial #1 Trial #2 Trial #3 Trial #4 Trial #5 n RAND 1 0.184117 2 0.162222 3 0.835792 4 0.795659 5 0.179481 6 0.200616 Arrival Time 0.423045546 0.423046 0.454698087 0.877744 0.044843731 0.922587 0.057146139 0.979734 0.429422115 1.409156 0.401590104 1.810746 Ellapsed Time N=6 N=5 #1 #2 N=2 #3 #4 #5 N=4 N=5 12 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025

Simulating PM Metrics NPV : pp 310-318 CPM/PERT Scheduling: pp 367-372 13 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 4/19/2025