Understanding Signal Detection Theory and Eyewitness Identification Procedures
Explore the concepts of signal detection theory, receiver operating characteristic analysis, and eyewitness identification procedures using mock-crime laboratory studies. Gain insights into decision-making processes in detecting signals and identifying suspects in lineups.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
SIGNAL-DETECTION THEORY AND RECEIVER OPERATING CHARACTERISTIC (ROC) ANALYSIS Psych 218 (Week 2)
LIST Diagnostic Decision honey candy dinner present sword belief shore kitchen cradle snake Absent Present Hit Miss .20 (Correct ID) .80 Present True State Correct Rejection .70 False Alarm (False ID) .30 Absent TARGETS FOILS honey candy dinner present sword belief shore kitchen cradle snake drama folly thorn message drink ground doctor woods journal sister FA Rate= .30 Hit Rate= .80
Eyewitness Identification Procedures Simultaneous Lineup Suspect: Innocent or Guilty? Fillers: All are known to be innocent
Eyewitness Identification Procedures Simultaneous Lineup Sequential Lineup Suspect: Innocent or Guilty?
Mock-Crime Laboratory Studies Each participant (n = 200) first watches a simulated crime TARGET-ABSENT LINEUP (N=100) TARGET-PRESENT LINEUP (N=100) Simultaneous Lineup Simultaneous Lineup
Mock-Crime Laboratory Studies Each participant (n = 200) first watches a simulated crime TARGET-ABSENT LINEUP (N=100) TARGET-PRESENT LINEUP (N=100) Simultaneous Lineup Simultaneous Lineup
Mock-Crime Laboratory Studies 100 participants tested using a target-present lineup Imagine that 58 pick the suspect CORRECT ID RATE = .58 100 participants are tested using a target-absent lineup Imagine that 43 pick the suspect FALSE ID RATE = .43
Target-Present Lineup 1 2 3 4 5 6 7 8 9 10 S S S S S S S S S S 1 S S S S S S S S S S 2 Suspect ID Rate: 0.58 (Guilty) S S S S S S S S S S 3 S S S S S S S S S S 4 Filler ID Rate: 0.12 S S S S S S S S S S 5 S S S S S S S S F F 6 No ID rate: 0.30 F F F F F F F F F F 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Absent Lineup 1 2 3 4 5 6 7 8 9 10 S S S S S S S S S S 1 S S S S S S S S S S 2 (Innocent) Suspect ID Rate: 0.43 S S S S S S S S S S 3 S S S S S S S S S S 4 Filler ID Rate: 0.15 S S S F F F F F F F 5 F F F F F F F F N N 6 N N N N N N N N N N No ID rate: 0.42 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Lindsay & Wells (1985) Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43 .58 .43 = 1.35 Diagnosticity Ratio Sequential lineup .50 .17 Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17 = 2.94
Lindsay & Wells (1985) Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43 0.8 Correct ID Rate 0.6 Simultaneous Sequential Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Lindsay & Wells (1985) Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43 0.8 1.35 Correct ID Rate 0.6 2.94 Simultaneous Sequential Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
"Absent" "Present" Correct ID Rate = 0.98 Target Absent Target Present False ID Rate = 0.50 Receiver Operating Characteristic (ROC) Weak Signal Strong Signal 1.0 0.8 Correct ID Rate "Absent" "Present" 0.6 Target Absent Target Present Correct ID Rate = 0.84 0.4 False ID Rate = 0.16 0.2 Weak Signal Strong Signal 0.0 0.0 0.2 0.4 0.6 0.8 1.0 False ID Rate "Absent" "Present" Correct ID Rate = 0.50 Target Absent Target Present False ID Rate = 0.02 Weak Signal Strong Signal
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Instead of manipulating response bias across conditions to trace out the ROC, you can trace out the same ROC more easily by collecting confidence ratings for each decision. Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Target-Present Lineup 1 2 3 4 5 6 7 8 9 10 S S S S S S S S S S 1 S S S S S S S S S S 2 Suspect ID Rate: 0.58 (Guilty) S S S S S S S S S S 3 S S S S S S S S S S 4 Filler ID Rate: 0.12 S S S S S S S S S S 5 S S S S S S S S F F 6 No ID rate: 0.30 F F F F F F F F F F 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Absent Lineup 1 2 3 4 5 6 7 8 9 10 S S S S S S S S S S 1 S S S S S S S S S S 2 (Innocent) Suspect ID Rate: 0.43 S S S S S S S S S S 3 S S S S S S S S S S 4 Filler ID Rate: 0.15 S S S F F F F F F F 5 F F F F F F F F N N 6 N N N N N N N N N N No ID rate: 0.42 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Present Lineup 1 2 3 4 5 6 7 8 9 10 1 1 2 2 2 2 2 3 3 3 1 3 3 3 3 3 3 4 4 4 4 2 4 4 5 5 5 5 5 5 5 6 Suspect ID Rate: 0.58 (Guilty) 3 6 6 6 6 6 7 7 7 7 7 4 7 7 7 7 7 7 7 7 7 7 Filler ID Rate: 0.12 5 7 7 7 7 7 7 7 7 F F 6 F F F F F F F F F F No ID rate: 0.30 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Present Lineup 1 2 3 4 5 6 7 8 9 10 N N 2 2 2 2 2 3 3 3 1 3 3 3 3 3 3 4 4 4 4 2 4 4 5 5 5 5 5 5 5 6 Suspect ID Rate: 0.56 (Guilty) 3 6 6 6 6 6 7 7 7 7 7 4 7 7 7 7 7 7 7 7 7 7 Filler ID Rate: 0.12 5 7 7 7 7 7 7 7 7 F F 6 F F F F F F F F F F No ID rate: 0.32 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Absent Lineup 1 2 3 4 5 6 7 8 9 10 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 2 3 3 3 3 3 4 4 4 4 4 Suspect ID Rate: 0.43 (Innocent) 3 4 5 5 5 5 5 6 6 6 6 4 7 7 7 F F F F F F F Filler ID Rate: 0.15 5 F F F F F F F F N N 6 N N N N N N N N N N No ID rate: 0.42 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Target-Absent Lineup 1 2 3 4 5 6 7 8 9 10 N N N N N N N N N N 1 2 2 2 2 2 2 2 3 3 3 2 3 3 3 3 3 4 4 4 4 4 Suspect ID Rate: 0.33 (Innocent) 3 4 5 5 5 5 5 6 6 6 6 4 7 7 7 F F F F F F F Filler ID Rate: 0.15 5 F F F F F F F F N N 6 N N N N N N N N N N No ID rate: 0.52 7 N N N N N N N N N N 8 N N N N N N N N N N 9 N N N N N N N N N N 10
Varying Response Bias from liberal to conservative 0.8 1 2 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 1 2 Correct ID Rate 3 0.6 4 Simultaneous Sequential 5 0.4 6 7 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 1.35 1.81 2.28 2.94 0.4 3.90 5.14 0.2 7.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Varying Response Bias from liberal to conservative 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Diabetes 1.25 3.31 16.3
The Concept of Discriminability 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
The Concept of Discriminability 0.8 Correct ID Rate 0.6 Simultaneous Sequential 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Lindsay & Wells (1985) Simultaneous lineup Correct Suspect ID rate = 0.58 False Suspect ID rate = 0.43 0.8 Correct ID Rate 0.6 Simultaneous Sequential Sequential lineup Correct Suspect ID rate = 0.50 False Suspect ID rate = 0.17 0.4 0.2 0.0 0.0 0.1 0.2 0.3 0.4 0.5 False ID Rate
Results from ROC Analysis (#1) Simultaneous vs. Sequential 0.8 0.7 Simultaneous Sequential 0.6 Correct ID Rate 0.5 0.4 0.3 0.2 0.1 0.0 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 False ID Rate Mickes, L., Flowe, H. D., & Wixted, J. T. (2012). Journal of Experimental Psychology: Applied, 18, 361 376.
Results from ROC Analysis (#2) Simultaneous vs. Sequential SIM SEQ Gronlund et al. (2012). Journal of Applied Research in Memory and Cognition, 1, 221 228.
Results from ROC Analysis (#3) Simultaneous vs. Sequential SIM x 4 SEQ x 4 SIM x 2 SEQ x 2 Dobolyi, D. G., & Dodson, C. S. (2013). Journal of Experimental Psychology: Applied, 19, 345-357.
Results from ROC Analysis (#4) Simultaneous Sequential Correct ID Rate False ID Rate Carlson, C. A. & Carlson, M. A. (2014). Journal of Applied Research in Memory and Cognition.
Results from ROC Analysis (#5) Correct ID Rate Simultaneous Sequential False ID Rate Andersen, S. M., Carlson, C. A., Carlson, M. A. & Gronlund (2014). Personality and Individual Differences, 60, 36-40.
Diagnosticity Ratio or ROC Analysis? Despite its merits, a single diagnosticity ratio thus conflates the influences of discriminability and response bias on binary classification, which muddies the determination of which procedure, if any, yields objectively better discriminability in eyewitness performance. Perhaps the greatest practical benefit of recent debate over the utility of different lineup procedures is that it has opened the door to a broader consideration of methods for evaluating and enhancing eyewitness identification performance. ROC analysis is a positive and promising step, with numerous advantages. The committee concludes that there should be no debate about the value of greater discriminability to promote a lineup procedure that brings less discriminability would be akin to advocating that the lineup be performed in dim instead of bright light.
What About Recent ROC Analyses? a small set of recent studies using ROC analysis has reported that discriminability (area under the ROC curve) for simultaneous lineups is as high, or higher, than that for sequential lineups. Amendola and Wixted re-analyzed a subset of the data for which proxy measures of ground truth were available Their analyses suggested that identification of innocent suspects is less likely and identification of guilty suspects is more likely when using the simultaneous procedures. While future studies are needed, these latter findings raise the possibility that diagnosticity is higher for the simultaneous procedures.
LIST Diagnostic Decision honey candy dinner present sword belief shore kitchen cradle snake Absent Present Hit Miss .20 (Correct ID) .80 Present True State Correct Rejection .70 False Alarm (False ID) .30 Absent TARGETS FOILS honey candy dinner present sword belief shore kitchen cradle snake drama folly thorn message drink ground doctor woods journal sister FA Rate= .30 Hit Rate= .80
Collect Confidence Ratings New Old ------------------------- ------------------------ High Medium Low Low Medium High 1 2 3 4 5 6
Confidence Ratings "New" "Old" 1 2 3 4 5 6 Low Low Targets Lures Medium Medium High High Memory Strength
"New" "New" "New" "New" "New" "New" "New" "Old" "Old" "Old" "Old" "Old" "Old" "Old" 1 2 3 4 5 6 1 2 3 4 5 6 Targets Targets Targets Targets Targets Targets Targets Lures Lures Lures Lures Lures Lures Lures Low Low Low Low Low Low Low Medium Medium Medium Medium Medium Medium Medium HR = .31 HR = .69 HR = .84 FA = .01 FAR = .01 FAR = .07 FAR = .07 FAR = .16 FAR = .16 High High High High High High High
"New" "New" "New" "New" "Old" "Old" "Old" "Old" 1 2 3 4 5 6 1 2 3 4 5 6 Targets Targets Targets Targets Lures Lures Lures Lures Low Low Low Low Medium Medium Medium Medium HR = .93 HR = .99 FAR = .31 FAR = .31 FAR = .69 FAR = .69 High High High High
.99, .69 .93, .31 1.0 .84, .16 0.8 0.6 Hit Rate .69, .07 0.4 0.2 .31, .01 0.0 0.0 0.2 0.4 0.6 0.8 1.0 FA Rate
Signal Detection Theory HR = .84 FAR = .16 "New" "Old" Targets Lures Memory Strength
