Accountability in Machine Learning Systems via Program Analysis
This presentation delves into the critical topic of accountability in machine learning systems through program analysis. Explore the ubiquitous nature of these systems, the threats they pose to fairness, and the formalization of explicit use. Discover challenges related to correlated inputs and transparency queries, along with the results of quantitative input influence techniques for measuring system input-output relationships.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Accountability in Machine Learning Systems via Program Analysis Gihyuk Ko PhD Candidate, Department of Electrical and Computer Engineering Carnegie Mellon University *slides were borrowed and modified from Anupam Datta s MIT Big Data@CSAIL Data Privacy Series talk on Accountable Big Data Systems
Machine Learning Systems are Ubiquitous Law Web services Education Credit Healthcare Enforcement 2
Machine Learning Systems Threaten Fairness Explicit Use [Datta, Tschantz, Datta 2015] Address Online Advertising System 1816 High paying Job Ads Gender Major 311 3
Formalizing Explicit Use Quantitative Input Influence [Datta, Sen, Zick 2016] Credit Decision Algorithm Age Age 27 27 Workclass Workclass Private Private Education Education Preschool Preschool How much causal influence do various inputs (features) have Marital Status Marital Status Married Married Occupation Occupation Farming-Fishing Farming-Fishing on a classifier s decision about individuals or groups? Race Race White White Because Okay but why? Gender Gender Male Male Negative Factors: Occupation Education Level Capital gain Capital gain $41310 $41310 .. .. Positive Factors: Capital Gain Transparency Query Transparency Report 4
Challenge Challenge | Correlated Inputs Credit Decision Algorithm (only uses income) Age Decision correlated Income Conclusion: Measures of association not informative! 5
Challenge Challenge | General Class of Transparency Queries Which input had the most influence in my credit denial? Individual What inputs have the most influence on credit decisions of women? Group What inputs influence men getting more positive outcomes than women? Disparity
Result | Result | Quantitative Input Influence (QII) A technique for measuring the influence of an input of a system on its outputs. Causal Intervention Deals with correlated inputs Quantity of Interest Supports a general class of transparency queries 7
Key Idea 1 Key Idea 1| Causal Intervention 23 45 64 30 Credit Decision Algorithm (only uses income) Age 30 Decision $40K Income $10K $90K $100K $30K Age Income 23 $10K 45 $90K Replace feature with random values from the population, and examine distribution over outcomes. 64 $100K 30 $40K Joe 8
?? QII for Individual Outcomes Inputs: ? ? ? ? ??? ?1?2 ?? ?1?2 ?? ?? ? ? Classifier Outcome ?? ? ? = 1 ? = ?Joe] ?? ? ? ??? = 1 ? = ?Joe] Causal Intervention: Replace feature with random values from the population, and examine distribution over outcomes. 9
Key Idea Key Idea 2 2| Quantity of Interest Various statistics of a system: Classification outcome of an individual ?? ? ? = ?(?0) ? = ?0] ?? ? ? ??? = ?(?0) ? = ?0] Classification outcomes of a group of individuals ?? ? ? = 1 ? is female] ?? ? ? ??? = 1 ? is female] Disparity between classification outcomes of groups ?? ? ? = 1 ? is male] ?? ? ? = 1 ? is female] ?? ? ? ??? = 1 ? is male] ?? ? ? ??? = 1 ? is female] 10
QII QII | Definition The Quantitative Input Influence (QII) of an input ? on a quantity of interest ??(?) of a system ? is the difference in the quantity of interest, when the input is replaced with random value via an intervention. ???? = ??? ??(? ???) 11
Result | Result | Quantitative Input Influence (QII) A technique for measuring the influence of an input of a system on its outputs. Causal Intervention Deals with correlated inputs Quantity of Interest Supports a general class of transparency queries 12
Experiments Experiments | Test Applications Predictive policing using the National Longitudinal Survey of Youth (NLSY) Features: Age, Gender, Race, Location, Smoking History, Drug History Classification: History of Arrests ~8,000 individuals arrests Income prediction using a benchmark census dataset Features: Age, Gender, Relationship, Education, Capital Gains, Ethnicity Classification: Income >= 50K ~30,000 individuals income Implemented with Logistic Regression, Kernel SVM, Decision Trees, Decision Forest 13
Personalized Explanation Personalized Explanation | Mr X Age Age 23 23 Workclass Workclass Private Private 11th 11th Education Education Marital Status Marital Status Never married Never married Occupation Occupation Craft repair Craft repair Relationship to household income Relationship to household income Child Child Race Race Asian-Pac Island Asian-Pac Island Gender Gender Male Male Capital gain Capital gain $14344 $14344 Capital loss Capital loss $0 $0 Work hours per week Work hours per week 40 40 Can assuage concerns of discrimination. Country Country Vietnam Vietnam income 14
Personalized Explanation Personalized Explanation | Mr Y Age Age 27 27 Workclass Workclass Private Private Education Education Preschool Preschool Marital Status Marital Status Married Married Occupation Occupation Farming-Fishing Farming-Fishing Relationship to household income Relationship to household income Other Relative Other Relative Race Race White White Gender Gender Male Male Capital gain Capital gain $41310 $41310 Capital loss Capital loss $0 $0 Explanations of superficially similar people can be different. Work hours per week Work hours per week 24 24 Country Country Mexico Mexico income 15
Result | Result | Quantitative Input Influence (QII) A technique for measuring the influence of an input of a system on its outputs. Causal Intervention Deals with correlated inputs Quantity of Interest Supports a general class of transparency queries 16
Machine Learning Systems Threaten Privacy Proxy Use [Datta, Fredrikson, Ko, Mardziel, Sen 2016] Protected Attribute Pregnancy Associated Scent-free Lotion Target s Advertising System Causally Used Diaper Coupons Pre-natal Vitamins Stationaries 17
Challenge Challenge | Proxy Variables Race correlated Race correlated Credit Decision Algorithm A (only uses income) Credit Decision Algorithm B (directly uses race) Income Income Decision Conclusion: Impossible to detect information use through proxies in black-box manner! 18
Key Idea Key Idea | Learning Systems as Programs Scent-free Lotion Diaper Coupons Pre-natal Vitamins Subprogram How much causal influence do various StationariesHow much associated are the subprogram s outputs to the individuals protected attribute? subprogramshave on a classifier s decision? A. Use an existing association measure! OR ( IF { AND ( Shopped(Pre-natal vitamins), Shopped(Scent-free lotion), ), Program Diaper Coupons, Nothing } IF { Shopped(Stationaries), Notebooks, Nothing } } ) 19
Recall: Causal Intervention 23 45 64 30 Credit Decision Algorithm (only uses income) Age 30 Decision $40K Income $10K $90K $100K $30K Age Income 23 $10K 45 $90K Replace feature with random values from the population, and examine distribution over outcomes. 64 $100K 30 $40K Joe 20
?? Recall: QII for Individual Outcomes Inputs: ? ? ? ? ??? ?1?2 ?? ?1?2 ?? ?? ? ? Classifier Outcome ?? ? ? = 1 ? = ?Joe] ?? ? ? ??? = 1 ? = ?Joe] Causal Intervention: Replace feature with random values from the population, and examine distribution over outcomes. 21
Program Decomposition OR ( IF { AND ( Shopped(Pre-natal vitamins), Shopped(Scent-free lotion), ?(?) ? (?,?(?)) ?(?) ), ? ? Diaper Coupons, Nothing } IF { Shopped(Stationaries), Notebooks, Nothing } } ) OR ( If { U ?(?,?) ? (?,?) ? (?,?) Diaper Coupons, Nothing } IF { Shopped(Stationaries), Notebooks, Nothing } } ) Define decomposition for each subprogram by replacing it with as the random variable. 22
QII for Subprograms ? ?1?2 ?? Inputs: ? ? ? ? ?1?2 ?? ?1?2 ?? p(X) ? p(Y) ? ? Classifier c c Outcome ??[? ?,?(?) = 1] ??[? ?,?(?) = 1] Causal Intervention: Replace subprogram output with outcome of the same subprogram with different sample from the population, and examine distribution over outcomes. 23
Proxy Use Proxy Use | 2-phase definition A subprogram that is associated to the protected attribute Use existing association measure! Protected Attribute Associated A subprogram that is influential to the output of the program Quantitative Subprogram Influence measure! Causally Used ?,? -proxy use With association measure ?, influence measure ? 24
Experiments Experiments | Test Applications Advertisement targeting using the Indonesian Contraception Dataset Features: Education, Children, Husband s Job, etc Classification: Contraception Methods Protected attribute: Religion ~6,000 individuals contra Prediction of Academic Performance using Portuguese Student Alcohol dataset Features: Failures, Studytime, Father s education level, Health status, etc Classification: Grade Protected attribute: Weekly alcohol consumption ~7,000 individuals student Implemented with Logistic Regression, Decision Trees, Random Forest 25
Proxy Use Detection Proxy Use Detection | contra Proxy use violation in submodel for religion: ite(education < 3.5, ite(children < 2.5, ite(age < 30.50, true, false) education level is an indicator for religion: This is concerning as certain discrimination or bias could be done based on the education level contra 26
Proxy Use Detection Proxy Use Detection | student Proxy use violation in submodel for week_alco_consumption: ite(studytime < 2.5, ite(Fedu < 3.50, false, true) studytime being less than 2.5 is an indicator for high weekly alcohol consumption: As study time is widely accepted to be directly correlated with the success in school, it s okay to use! student 27
Use Restrictions in Machine Learning Systems [Datta, Fredrikson, Ko, Mardziel, Sen, Tschantz 2017] Do not use a protected information type (explicit or proxy use) for certain purposes with some exceptions Accountability in Machine Learning Systems Use Privacy: Do not use health information for purposes other than those of healthcare context; exceptions for law enforcement Correction to prevent future violations using program analysis techniques Provide oversight to detect violations and explain behaviors Non-discrimination: Do not use race or gender for employment decisions; business necessity exceptions 28
Related Work: Quantitative Input Influence Randomized Causal Intervention Feature Selection: Permutation Importance [Breiman 2001] Importance of Causal Relations [Janzing et al. 2013] Do not consider marginal influence or general quantities of interest Associative Measures Quantitative Information Flow: Appropriate for secrecy FairTest [Tram r et al. 2015] Correlated inputs hide causality Interpretability-by-design Regularization for simplicity (Lasso) Bayesian Rule Lists [Letham et al. 2015] Potential loss in accuracy 29
Related Work: Accountability for Use Restrictions Accounting for proxies and their causal use is missing Usage control in computer security, Sandhu and Park 2002 Information accountability, Weitzner et al. 2008 Audit algorithms for privacy policies, Garg, Jia, Datta 2011 Enforcing purpose restrictions in privacy policies, Tschantz, Datta, Wing 2012 Privacy compliance of big data systems, Sen, Guha, Datta, Rajamani, Tsai, Wing 2014 Fairness in big data systems Group fairness [Feldman+ 2015]: detection and repair of disparate impact; does not account for proxy usage in general Individual fairness [Dwork et al. 2011]: focus on correctness by construction not accountability 30
Future Directions Accountable deep learning to ensure non-discrimination and use privacy Efficient detection of proxy use via use of inference techniques in probabilistic programming 31
Challenge Challenge | Single Inputs have Low Influence Only accept old, high-income individuals Young Age Decision Classifier Low Income 34
Nave Approach | Na ve Approach | Set QII Replace ?? with a independent random value from the joint distribution of inputs ? ?. ? ? = ? ? ?(? ???) ? ?1?2 ?? ?1?2 ?1 ?? ?? ? ? 35
Marginal QII Not all features are equally important within a set. Marginal QII: Influence of age and income over only income. ? {age,income} ? {income} ? {age,gender,job} ? {gender,job} But age is a part of many sets! ? {age} ? {} ? {age,gender} ? {gender} ? {age,gender,job} ? {gender,job} ? {age,job} ? {job} ? {age,gender,income} ? {gender,income} ? {age,gender,income,job} ? {gender,income,job} 36
Key Idea Key Idea 3| Set QII is a Cooperative Game Cooperative game set of agents value of subsets Voting Revenue Sharing Input Influence agents features value influence 37
Shapley Value [Shapley 53] For cooperative games, the only aggregation measure that satisfies symmetry, dummy, and monotonicity is: Need to compute sum over all subsets of features: Efficient approximation by randomly sampling sets 38
Result | Result | Quantitative Input Influence (QII) A technique for measuring the influence of an input of a system on its outputs. Causal Intervention Deals with correlated inputs Quantity of Interest Supports a general class of transparency queries Cooperative Game Computes joint and marginal influence Performance QII measures can be approximated efficiently 39
Program Syntax Supports various families of classifiers including Decision Trees, families of Linear Classifiers, Random Forests, and so on.
Program Decomposition OR ( IF { AND ( Shopped(Pre-natal vitamins), Original program Shopped(Scent-free lotion), ), Diaper Coupons, Nothing } Sub-program IF { Shopped(Stationaries), Notebooks, Nothing } } ) OR ( If { New program U Diaper Coupons, Nothing } IF { Shopped(Stationaries), Notebooks, Nothing } } )
Formalizing Proxy Use: 2-phase definition A sub-program that is influential to the original program Qualitatively: Change [?1], does [?2] change? Quantitatively: QII A sub-program that is associated to the protected attribute Association measure: NMI ?,? -proxy use: exhaustively search for every sub-programs
