Unveiling the Deceptive Nature of Risk Studies

Camouflage or scary monsters: deceiving others about risk Ross Anderson Cambridge

Security psychology In the 1990s information security was seen as a problem in maths and engineering From about 2001 we got economics on board: if Alice guards a system and Bob pays the cost of failure, you can expect trouble Psychology was seen as a usability issue But we could not explain the privacy gap between stated and revealed preferences

Security psychology (2) Realisation that behavioural economics maybe had something to contribute, we organised the first Workshop on Security and Human Behavior at MIT in 2008 Invitees: half security engineers, half psychologists, anthropologists, philosophers Why do people worry too much about terrorism, and not enough about computer security?

Nick Humphreys analysis It would be easy to make people more relaxed about flying. Just hide the gun-toting cops and have lots of pastel colours, nice sofas, soothing music It would also be easy to make people worry more about their computers. Just force everyone to use a Jaws screensaver But governments won t allow the first and computer companies won t allow the second

The flaw in many risk studies Most risk studies overlook the adversaries! Health: British Sugar, Imperial Tobacco Computer security: not just the Rustock gang but Microsoft, Apple, Google who want you to not worry Terrorism: not just Daesh but the agencies, ministers, the media who want you to worry as much as possible

Murphy and Satan are different In reliability engineering you worry about stuff that can go wrong with low probability, even one in a billion for widely used products In security engineering you worry about a one-in-a-billion case being publicised and then used as a loophole by everybody The odds shorten dramatically! A growing challenge for cars, medical devices What can we do?

What frameworks exist? Adversarial risk analysis (David Rios Insua) looks at two-player multiround games e.g. Spanish fishermen / Somali pirates Terrorism studies (John Mueller et al) Behavioural economics, e.g. of privacy (Alessandro Acquisti et al) Criminologists looking at cybercrime in various ways, e.g. pathways into crime What else?

Recent case in UK Wannacry infected unpatched Vista machines in 5% of UK hospitals where SMB open Reactions NHS: Microsoft s fault for charging for XP patches Microsoft: NSA s fault for not telling us of the vuln GCHQ: not our fault for developing cyberweapons; everybody does that; but give us money to protect .uk from clever evil gangs Us: looks like a 14-year-old did it

How do people think about this? Case study: browser warnings Your browser warns you if you re about to go to a site with an expired certificate (who cares?) or a site containing malware (when you should care very much!) Most people ignore warnings even when they shouldn t Google: what can people do about this?

Browser warnings Work with David Modic, and initially with Adrienne Porter-Felt of Google Classic psychology: can we use images? Chance discovery at Google: two warnings with a cartoon head / missing Chrome logo had response elevated from 30% to 60% Melissa Bateson s famous coffee jar! Followup with millions of impressions: nope

Browser warnings (2) So what can work then? David Modic and I tested response to Appeal to authority Social compliance Concrete vs vague threats Based on much research on psychology of persuasion, and on scam compliance Psychology might suggest authority / social

Browser warnings (results) The most significant effect was giving concrete warnings as opposed to vague ones Some way behind was appeal to authority Factors other than our treatments: trust in the browser vendor was strongest, then mistrust of authority All factors together explain 60% of the effect

Which warning do you follow? Warning: the website you re trying to go to is a bad website The website you re trying to go to will attempt to load the Zeus botnet malware on your computer, which will attempt to steal your bank credentials and do you no good whatsoever

Is this effect adversarial? Every day we see many warnings that are written for other people s benefit In the IT industry, much of it s not just liability dumping, but demeaning and calculated to induce learned helplessness The action you re about to take carries huge risks you re too stupid to understand. But click here to get on with your work anyway Do you buy this?

More research needed! That specific warnings work better than pictures suggests that conflict economics is stronger here than psychology But it d be good to disentangle the effects. Will warnings work differently where adversarial risk messages are clearly likely? When is it more efficient to sensitize the public to deception than to teach them stats?

A final thought One of the big questions in deception research is self-deception Psychologists ask: does it exist, or is it just about suggestibility? Economists ask: what s the value of belief? (B nabou & Tirole s Mindful Economics ) These debates can obviously have some impact here too