Achieving Positive Outcomes in Data Science and Social Impact

datascience-project.eu Data Science & Social Impact: Achieving Positive Outcomes By Women in AI Austria The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Data Science & Social Impact: Achieving Positive Outcomes What you can find in this course: 1. Some examples of using data science for social good Deep dive into Amnesty International Italy s Social Media Monitoring 2. Understand main risks of the technology Examples of negative consequences of data science applications 3. The characteristics of trustworthy AI 4. Understand the challenges of measuring fairness Discussion of the COMPAS recidivism risk algorithm The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Data Science & Social Impact: Achieving Positive Outcomes Why should you learn this: Data science and AI have a huge variety of applications with positive social impact. For example, data science is useful to investigate how social media impacts human rights. On the other hand, data science and AI applications also carry risks to health, safety, the environment and human rights. Bias and discrimination, privacy concerns, and harmful environmental impacts are just some of the possible effects. In order to ensure that data science applications benefit people and the planet, it is necessary to understand both its capabilities and its risks. In this course, you will be introduced to both, and will also be introduced to some methods for addressing the risks. The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Index 1 3 2 Using data science for social good 1. Overview of possible data science for good use cases 2. Amnesty Italy Use Case Data science isn t always good 1. Major known examples 2. Overview of main risks Trustworthy AI 1. EU HLEG characterisation 2. Focus on bias and discrimination 3. Fairness Metrics The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 1: Overview of possible data science for good use cases Industry examples Skills adjacency detection and targeted training of missing skills: SkillsFuture Singapore, https://www.skillsfuture.gov.sg/AboutSkillsFuture AI & digital twins - simulating and practicing for resilience in the supply chain: https://www.technologyreview.com/2021/10/26/1038643/ai-reinforcement-learning-digital-twins-can-solve-supply-chain- shortages-and-save-christmas/ Reducing the footprint of recycled steel: Fero Labs uses AI to help steel manufacturers reduce the use of mined ingredients by up to 34%, preventing an estimated 450,000 tons of CO2 emissions per year: https://gpai.ai/projects/responsible- ai/environment/climate-change-and-ai.pdf The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 1: Overview of possible data science for good use cases More industry examples Adaptive charging breaks down barriers to electric vehicle adoption. Bi-directional charging & Vehicle to Grid technologies need smart scheduling algorithms. https://ev.caltech.edu/info Using AI to detect forced labor in the supply chain: https://www.altana.ai/blog/illuminating-xinjiang-forced-labor- ecosystem Machine learning can boost the value of wind energy: https://www.deepmind.com/blog/machine-learning-can- boost-the-value-of-wind-energy The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Barometro dell Odio: Annual Social Media Monitoring Campaigns since 2018. What is the tone of online discourse, in particular political and human rights relevant discussion? To what extent does intolerance and discrimination shape the social media landscape, and what is the impact on disadvantaged groups? Data Science at the service of human rights impact assessment. The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Modalities: Public content downloaded via Twitter and Facebook API Collected from a list of public accounts/profiles determined by Amnesty Italy 4-8 weeks of monitoring (exception: 2021 16 weeks of monitoring) Random sampling of comments from most active accounts Between 30 100 thousand comments selected in this manner Manual labelling by between 50 150 trained Amnesty volunteers: topic and level of offensiveness Cross-checking (same comment labelled by 2-3 volunteers) of all comments Resolution of inconsistent labelling & determination of target and type of offense: by Amnesty experts ( Tavolo dell Odio ) The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Example: European Parliamentary Elections 2019 The public Twitter and Facebook profiles of 461 candidates, plus the party leadership, were monitored. 6 weeks (April 15 to May 24, 2019) Over 27 thousand posts, and 4 million comments were collected The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Example: European Parliamentary Elections 2019 The public Twitter and Facebook profiles of 461 candidates, plus the party leadership, were monitored. 6 weeks (April 15 to May 24, 2019) Over 27 thousand posts, and 4 million comments were collected The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Example: European Parliamentary Elections 2019 Due to the large volume of comments, a selection had to be made of which politicians feeds to evaluate. The criteria were: extent of social media activity, while ensuring a representation of all parties, all regions, and at least one woman/man per party. As a result, the feeds of 77 politicians were evaluated. 80% of posts, and a random sampling of 100 thousand comments were tagged by 150 Amnesty Italy volunteers The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Hate Speech is not randomly distributed: Each rectangle has the exact same number and size of blue dots Random Clustered The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case According to Amnesty Italy data, the prevalence of online hate speech is about 1%. But it tends to be focussed on particular groups and topics. And, it also has peaks of concentration in time. Random Clustered The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Hot Rod Topics: Immigration/Asylum/Refugees; Roma; Religious minorities;Women & women s rights, ... Random Clustered The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 1: Using data science for social good Section 2: Amnesty Italy Use Case Offensive speech is: A catalyst for more offensive speech More popular: on average, offensive posts get more interactions, shares, responses An obstacle to freedom of expression: during the monitoring period (November-December 2019) for the Sessismo da Tastiera edition, we observed three women being targeted and two of them pushed off of social media platforms via hate campaigns https://www.amnesty.it/barometro-dellodio-sessismo-da-tastiera/#sintesi Report, p.20 The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 1: Major known examples Bias, discrimination, stereotypes Ziad Obermeyer et al. Dissecting racial bias in an algorithm used to manage the health of populations. https://science.sciencemag.org/content/366/6464/447 The Guardian, Amazon ditched AI recruiting tool that favored men for technical jobs, October, 2018. https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine After Google s Gorillas comes Facebook s Primates: Facebook Apologizes After A.I. Puts Primates Label on Video of Black Men, September 2021. https://www.nytimes.com/2021/09/03/technology/facebook- ai-race-primates.html The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 1: Major known examples Bias, discrimination, stereotypes labor, and the environment Semuels, A., The Internet Is Enabling a New Kind of Poorly Paid Hell, in The Atlantic, January 23, 2018. https://www.theatlantic.com/business/archive/2018/01/amazon-mechanical-turk/551192/ Geiger, G., Court Rules Deliveroo Used 'Discriminatory' Algorithm, Motherboard, January 2021. https://www.vice.com/en/article/7k9e4e/court-rules-deliveroo-used-discriminatory-algorithm Hao, K., Training a single AI model can emit as much carbon as five cars in their lifetimes, in MIT Technology Review, June 6, 2019 https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars- in-their-lifetimes/ The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 1: Major known examples Be a bias detective Yes, you can try this experiment out at home!* Enter the following text into Google Translate, and translate from English into German: English: My doctor is clever. She immediately found the solution Google Deutsch: English: My secretary is clever. He immediately found the solution Google Deutsch: *Hat tip Liad Magen for the idea The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 2: Overview of main risks Data science applications are neither objective, nor neutral: From using bots to create deepfake nudes on Telegram, generating sexualised avatars of women (but not men), not developing functionalities useful to a specific group of people, or undermining gender identity through binary classification, data science applications can cause harm. Reflect on what your application can do, what it is used for, who is included/excluded and who might be affected in different ways the consequences can be widespread! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 2: Overview of main risks Data science applications are not perfect, and their errors are not randomly distributed: Gender classification algorithms using facial recognition routinely misclassify darker-skinned women more frequently than lighter-skinned men (and women). This is because the datasets on which the two models investigated by Joy Buolamwini and Timnit Gebru were trained contain a disproportionate share of images of light- skinned women. Check out the 2018 study of these two rock star data scientists! Algorithms used to detect offensive speech on online platforms were more likely to classify patterns of speech common amongst Black US Americans as offensive and datasets similarly displayed widespread bias against African American English. This shows how important labelling of the dataset is: if the data is labelled in a biased way, the outcomes will be biased too. Check out the 2019 article and the two studies it links to! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 2: Data science isnt always good Section 2: Overview of main risks In addition, data science applications can be very data intensive, carrying with them issues of Privacy Data protection Poor data quality: garbage in, garbage out The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

... Oh no... Ethics Guidelines to the rescue! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Ethics Guidelines OECD https://www.oecd.ai/ai-principles UNESCO: https://en.unesco.org/news/unesco-launches-worldwide-online- public-consultation-ethics-artificial-intelligence UNICEF: https://www.unicef.org/globalinsight/reports/policy-guidance-ai- children EU: https://ec.europa.eu/info/publications/white-paper-artificial- intelligence-european-approach-excellence-and-trust_en Microsoft AETHER: https://www.microsoft.com/en-us/ai/responsible-ai Google s (former) Ethics Committee: https://www.reuters.com/article/us- alphabet-google-ai-idUSKCN1RH00S Partnership on AI: https://www.partnershiponai.org/ ... over 80 ethics guidelines are available today The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Ethics Guidelines or Washing? Nice, but lets talk when they become binding ... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

... Oh no... EU Regulations to the rescue! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 1: Trustworthy AI The EU HLEG has established the following characteristics of a trustworthy AI system, based on the EU Charter of Fundamental Rights: (1) human agency and oversight, (2) technical robustness and safety, (3) privacy and data governance, (4) transparency, (5) diversity, non-discrimination and fairness, (6) environmental and societal well-being and (7) accountability The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 2: Bias, fairness, non-discrimination What is bias? In the context of data science and machine learning in general, many different definitions of bias collide (colloquial usage vs. Statistics vs. deep learning). In the context of trustworthy AI, we will take bias to be a prejudice that favors one group over another. What is fairness? The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 2: Bias, fairness, non-discrimination Societal Bias Confirmation Bias In-group Bias Automation Bias Temporal Bias Omitted Variable Bias Sampling Bias Representation Bias Measurement Bias Evaluation Bias ... and many more ... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 2: Bias, fairness, non-discrimination How to detect and measure all this bias? First, check the quality of your data. And then ... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 2: Bias, fairness, non-discrimination ... actually, we can usually only detect its effect on the result of the model By measuring it with a Fairness Metric! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics Group fairness Conditional statistical parity False positive error rate balance False negative error rate balance Conditional use accuracy equality Overall accuracy equality Test-fairness Well-calibration Fairness through unawareness Counterfactual fairness ... and many more ... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics The proportion of correctly predicted high risks is the same regardless of demographic Predictive parity (All groups have equal PPV) The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics Within each true risk category, the percentage of false predictions is equal for each demographic Equalized Odds (All groups have equal FNR and equal FPR) The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics Which definition would you call fair? The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics What happens when the prevalence of high risk is greater for one group, than for another? The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics If p is the proportion of high risk individuals in a population: p .1 - PPV 1-p PPV FPR = (1 FNR) ... then this formula tells us that we cannot have both Equalized Odds and Predictive Parity In order to see why, suppose that both Equalized Odds and Predictive Parity hold. Plug into the formula, and some algebra will show you that then the prevalence p also has to be the same for both populations ... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics Lets recall a much-debated example: In May 2016, ProPublica published an article indicating that the predictions of a widely-used recidivism modelling model (COMPAS), were biased: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal- sentencing For a good explanation, see: https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can- an-algorithm-be-racist-our-analysis-is-more-cautious-than- propublicas/?noredirect=on&utm_term=.24b3907c91d1 Julia Dressel and Hany Farid, The accuracy, fairness, and limits of predicting recidivism, Science Advances, 17 Jan 2018: Vol. 4, no. 1. https://advances.sciencemag.org/content/4/1/eaao5580.full The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics PREDICTED LOW RISK TRUE LOW RISK PREDICTION MODEL DATA PREDICTED HIGH RISK TRUE HIGH RISK The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics PREDICTIVE PARITY The proportion of correctly predicted high risks is the same regardless of demographic All groups have equal PPV Northpointe says ... it s fair, because within each risk category, the proportion of defendants who reoffend is approximately the same regardless of race The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics EQUALIZED ODDS All groups have equal FNR and equal FPR ProPublica says it s unfair, because among defendants who ultimately did not reoffend, blacks were more than twice as likely as whites to be classified as medium or high risk (42 percent vs. 22 percent) The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics The overall recidivism rate for black defendants is higher than for white defendants (52 percent vs. 39 percent) The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics The overall recidivism rate for black defendants is higher than for white defendants (52 percent vs. 39 percent) And as we saw above, when two populations have different prevalences, then Equalized Odds and Predictive Parity cannot both hold true. The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics So which definition of fair is fair? The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics The problem ... The overall recidivism rate for black defendants is higher than for white defendants (52 percent vs. 39 percent) ... is systemic bias It can be dangerous to deploy automated systems to make or support decisions in societal contexts where prejudice is deeply embedded. The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Unit 3: Trustworthy AI Section 3: Fairness metrics Involving the various stakeholders, and experts from different fields, in the decision of which fairness metric to consider is vital. And finally, keep in mind that sometimes, data science might not always be the right approach... The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Summing up Fairness of outcomes in data science and AI applications can be measured in many different ways. Data science can be used in many applications with positive social impact Data science and AI applications also carry risks to health, safety, and human rights Building trustworthy AI applications requires intensive interdisciplinary collaboration Bias and discrimination, privacy concerns, and harmful environmental impacts are just some of the possible effects. For example, data science is useful in investigating human rights impacts of social media The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

Self-assessment test 1. Name three different examples of using data science for good A adaptive charging 2. Which of the following is not one of the principles of trustworthy AI? 3. The Equalized Odds fairness metric requires that A Robustness A TPR is the same for all demographic groups B skills matching B Reproducibility C monitoring social media for human rights impacts B FPR is the same for all demographic groups C Transparency C All of the above The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."

datascience-project.eu Thank you! The European Commission support for the production of this publication does not constitute endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein."