Measurement Scales in Experimental Psychology

Psychology 3450W: Experimental Psychology Fall, 2019 Professor Delamater

Measurement Issues 1. Measurement Scales 2. Evaluating Measurements 3. Statistics Primer Descriptive Inferential Lets look at each of these in more detail

Measurement Issues Measurement Scales What quantitative information is conveyed by our measuring instrument about the psychological construct? Our instrument is a readout of the underlying construct. ?

Measurement Issues 4 Measurement Scales 1. Nominal 2. Ordinal 3. Interval 4. Ratio ?

Measurement Issues 4 Measurement Scales (e.g., numbers on a uniform). Thus, they contain no numerical information at all. 1. Nominal Scale Numbers are used to signify categories ?

Measurement Issues 4 Measurement Scales (e.g., numbers on a uniform). Thus, they contain no numerical information at all. 1. Nominal Scale Numbers are used to signify categories ?

Measurement Issues 4 Measurement Scales relative magnitude, i.e., >, <, or = relations, but no more. For example, Turnpike exit numbers tell you that exit 9 is further away from you than exit 7 (if you are at 1), but you cannot infer that the difference between 7 and 9 is equal to the difference between 4 and 6. 2. Ordinal Scale Numbers can convey information about ?

Measurement Issues 4 Measurement Scales relative magnitude, i.e., >, <, or = relations, but no more. For example, Turnpike exit numbers tell you that exit 9 is further away from you than exit 7 (if you are at 1), but you cannot infer that the difference between 7 and 9 is equal to the difference between 4 and 6. 2. Ordinal Scale Numbers can convey information about ?

Measurement Issues 4 Measurement Scales relative magnitude, i.e., >, <, or = relations, but no more. For example, Turnpike exit numbers tell you that exit 9 is further away from you than exit 7 (if you are at 1), but you cannot infer that the difference between 7 and 9 is equal to the difference between 4 and 6. 2. Ordinal Scale Numbers can convey information about Measure Anxiety with Heart Rate (HR) in the presence of 3 different stimuli. Your results are as follows: Stimulus 1: 60 beats per min Stimulus 2: 80 beats per min Stimulus 3: 100 beats per min ? Stim 3 > Stim 2 > Stim 1. It seems that Stim 3 is more anxiety-provoking than Stim 2 which is more than Stim 1.

Measurement Issues 4 Measurement Scales relative magnitude, i.e., >, <, or = relations, but no more. For example, Turnpike exit numbers tell you that exit 9 is further away from you than exit 7 (if you are at 1), but you cannot infer that the difference between 7 and 9 is equal to the difference between 4 and 6. 2. Ordinal Scale Numbers can convey information about Measure Anxiety with Heart Rate (HR) in the presence of 3 different stimuli. Your results are as follows: Stimulus 1: 60 beats per min Stimulus 2: 80 beats per min Stimulus 3: 100 beats per min ? If the anxiety differences between 3 and 2 and 2 and 1 are not equal, then HR is only an ordinal scale measure of anxiety.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, IQ and intelligence. 3. Interval Scale Numbers convey the same information ?

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, IQ and intelligence. 3. Interval Scale Numbers convey the same information Measure IQ from different people. The scores are as follows: Person 1: score of 60 Person 2: score of 90 Person 3: score of 120 ? Person 3 > Person 2 > Person 1. Person 3 has more intelligence than Person 2 and Person 2 has more intelligence than Person 1. That s an ordinal scale statement.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, IQ and intelligence. 3. Interval Scale Numbers convey the same information Measure IQ from different people. The scores are as follows: Person 1: score of 60 Person 2: score of 90 Person 3: score of 120 ? Person 3 > Person 2 > Person 1. If we can say that: Person 3 has as much more intelligence than Person 2 as Person 2 has over Person 1, that would be an interval scale statement.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, IQ and intelligence. 3. Interval Scale Numbers convey the same information Measure IQ from different people. The scores are as follows: Person 1: score of 60 Person 2: score of 90 Person 3: score of 120 ? However, many psychologists would argue that IQ cannot be taken as a true interval scale measure of intelligence.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, RT and stimulus processing vs expectation. 3. Interval Scale Numbers convey the same information Measure RT in a Posner cueing task: Valid Trials: RT = 325 msec (dot target) Invalid Trials: RT = 350 msec (dot target) ? In this task spatial cues speed up processing of the dot target.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, RT and stimulus processing vs expectation. 3. Interval Scale Numbers convey the same information Measure RT in a Posner cueing task: Valid Trials: RT = 325 msec (dot target) Invalid Trials: RT = 350 msec (dot target) Valid Trials: RT = 675 msec (word target) Invalid Trials: RT = 700 msec (word target) ? Suppose we also looked at word targets

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, RT and stimulus processing vs expectation. 3. Interval Scale Numbers convey the same information Measure RT in a Posner cueing task: Valid Trials: RT = 325 msec (dot target) Invalid Trials: RT = 350 msec (dot target) Valid Trials: RT = 675 msec (word target) Invalid Trials: RT = 700 msec (word target) ? The same processing advantage occurs with spatially predicted dots as with words. RT is an interval scale measure of stimulus processing, i.e., the added efficiency in processing produced by spatial cues.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, RT and stimulus processing vs expectation. 3. Interval Scale Numbers convey the same information Measure RT in a Posner cueing task: Valid Trials: RT = 325 msec (dot target) Invalid Trials: RT = 350 msec (dot target) Valid Trials: RT = 675 msec (word target) Invalid Trials: RT = 700 msec (word target) ? However, if we were to use this RT difference as a measure of the psychological construct of expectation, then we d likely only assert ordinal scale properties to RT.

Measurement Issues 4 Measurement Scales about relative magnitude as in the ordinal scale, but, in addition, equal differences are psychologically equal. For example, RT and stimulus processing vs expectation. 3. Interval Scale Numbers convey the same information Measure RT in a Posner cueing task: Valid Trials: RT = 325 msec (dot target) Invalid Trials: RT = 350 msec (dot target) Valid Trials: RT = 675 msec (word target) Invalid Trials: RT = 700 msec (word target) ? This is because the same amount of expectation may translate into differences in RT at different points along the RT scale. Thus, an equal numeric difference may not be psychologically equivalent.

Measurement Issues 4 Measurement Scales the interval scale, but, in addition, ratio statements can be made because a true 0 point can be identified. For example, scale for height and weight. 4. Ratio Scale Numbers convey the same information as in ? You can say when someone is twice as tall or heavy as someone else.

Measurement Issues 4 Measurement Scales convey the same information as in the interval scale, but, in addition, ratio statements can be made because a true 0 point can be identified. For example, temperature scales 4. Ratio Scale Numbers ? Compare Fahrenheit, Celsius, and Kelvin Scales of temperature. Are these interval or ratio scales? Why?

Measurement Issues 4 Measurement Scales convey the same information as in the interval scale, but, in addition, ratio statements can be made because a true 0 point can be identified. For example, temperature scales... 4. Ratio Scale Numbers ? Kelvin Scale: Absolute 0 is theoretically meaningful zero heat energy.

Measurement Issues 4 Measurement Scales convey the same information as in the interval scale, but, in addition, ratio statements can be made because a true 0 point can be identified. For example, temperature scales... 4. Ratio Scale Numbers ? Kelvin Scale: Absolute 0 is theoretically meaningful zero heat energy. In Psychological research we d need to know how the total absence of a construct translates into a numeric value. Very Difficult.

Measurement Issues Measurement Scales in Psychology ? Generally speaking, measuring instruments in psychology experiments assess psychological constructs at either the ordinal or interval scale of measurement. The problems are (a) that we can rarely, if ever, identify a true 0 point for some psychological construct, and (b) it is sometimes difficult to determine if equal differences at different points on the measuring scale are psychologically equivalent.

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Reliability How consistent is the measuring instrument at measuring whatever it measures? ?

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Reliability How consistent is the measuring instrument at measuring whatever it measures? Suppose we used this type of scale to measure intelligence. ? Is it consistent? If so, then it is reliable BUT, is it valid?

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Validity How accurate is the measuring instrument at measuring a particular psychological construct? Suppose we used this type of scale to measure intelligence. ? Is it consistent? If so, then it is reliable BUT, is it valid? Well, NO, this is not a valid measure of intelligence.

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Validity How accurate is the measuring instrument at measuring a particular psychological construct? Three types of validity: Face, Criterion (predictive), Construct ?

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Face Validity How intuitive is it that a measuring instrument measures the psychological construct? e.g., RTs in a line length discrimination task as a measure of intelligence ? Press Button 1 if same Press Button 2 if different

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Face Validity How intuitive is it that a measuring instrument measures the psychological construct? e.g., RTs in a line length discrimination task as a measure of intelligence ? Press Button 1 if same Press Button 2 if different

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Face Validity How intuitive is it that a measuring instrument measures the psychological construct? e.g., RTs in a line length discrimination task as a measure of intelligence ? Press Button 1 if same Press Button 2 if different Etc Is RT in this task a good measure of intelligence? On the face of it, we d say NO.

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Face Validity How intuitive is it that a measuring instrument measures the psychological construct? e.g., RTs in a line length discrimination task as a measure of intelligence ? Press Button 1 if same Press Button 2 if different Etc But, in fact, RT in this task negatively correlates with IQ. Thus, it has low Face Validity, but that doesn t mean it isn t useful.

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Criterion Validity Can the measuring instrument be used to predict future performance related to the construct being measured? e.g., Can SAT, GRE scores be used to predict academic success? ? SAT correlates positively with 1st yr GPA, but the correlation goes down thereafter. Thus, there is some criterion (predictive) validity to these measure, but it seems limited.

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Criterion Validity Can the measuring instrument be used to predict future performance related to the construct being measured? e.g., Can a measure of depression predict whether people are likely to attempt suicide or to actually commit suicide? ? The Beck Depression inventory (BDI) is one popular measuring scale for depression.

Measurement Issues Beck Depression Inventory 1. 0 1 2 3 2. 0 1 2 3 I do not feel sad. I feel sad I am sad all the time and I can't snap out of it. I am so sad and unhappy that I can't stand it. ? I am not particularly discouraged about the future. I feel discouraged about the future. I feel I have nothing to look forward to. I feel the future is hopeless and that things cannot improve. 3. 0 1 2 3 4. 0 1 2 3 I do not feel like a failure. I feel I have failed more than the average person. As I look back on my life, all I can see is a lot of failures. I feel I am a complete failure as a person. BDI Depression I get as much satisfaction out of things as I used to. I don't enjoy things the way I used to. I don't get real satisfaction out of anything anymore. I am dissatisfied or bored with everything. etc

Measurement Issues Beck Depression Inventory 1. 0 1 2 3 2. 0 1 2 3 I do not feel sad. I feel sad I am sad all the time and I can't snap out of it. I am so sad and unhappy that I can't stand it. ? I am not particularly discouraged about the future. I feel discouraged about the future. I feel I have nothing to look forward to. I feel the future is hopeless and that things cannot improve. 3. 0 1 2 3 I do not feel like a failure. I feel I have failed more than the average person. As I look back on my life, all I can see is a lot of failures. I feel I am a complete failure as a person. BDI Depression etc 1-10____________________These ups and downs are considered normal 11-16___________________ Mild mood disturbance 17-20___________________Borderline clinical depression 21-30___________________Moderate depression 31-40___________________Severe depression over 40__________________Extreme depression

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Criterion Validity Can the measuring instrument be used to predict future performance related to the construct being measured? e.g., Can a measure of depression predict whether people are likely to attempt suicide or to actually commit suicide? ? The Beck Depression inventory (BDI) is one popular measuring scale for depression. Green et al (2015, J Clin Psychiatry) demonstrated a clear predictive relationship between BDI and suicide deaths and attempts. Shows criterion validity

Measurement Issues Evaluating Measurements Aside from quantitative considerations, we can ask how Reliable and Valid are our measuring devices. Construct Validity Is the measuring instrument really assessing the underlying psychological construct of interest? This is the most difficult to establish because it relates to the extent that we are actually measuring the construct of interest. ? The way to establish construct validity is: predictable research outcomes. 1. This increases the validity of our measuring instrument, and 2. The construct itself. 3. This is related to the converging operations idea. We must be on the right track if we can run multiple studies with predictable outcomes.

Measurement Issues Evaluating Measurements concepts. It is possible for a measure to be highly reliable, but not valid. However, a measure that is valid must also be reliable (because it is actually measuring the thing it is supposed to measure). ? Notice that Reliability and Validity are dissociable

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Inferential Statistics

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies (mean, median, mode)

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies (mean, median, mode) Spread

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies (mean, median, mode) Spread (range, variance, standard deviation, SEM)

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies (mean, median, mode) Spread (range, variance, standard deviation, SEM)

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? Descriptive Statistics Central Tendencies (mean, median, mode) Spread (range, variance, standard deviation, SEM) Standard Dev = Square Root (variance) 68% of normally distributed scores fall within +/- 1 sd, 95% within +/- 2 sd SEM = St Dev / Square Root (N)

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? experimental hypothesis. We can ask about the likelihood of observing some difference by chance. If the observation is unlikely to have occurred by chance, then it is likely that there is a real difference between our groups or experimental conditions. Inferential Statistics A tool for assessing the status of an

Measurement Issues Statistical Review Once we collect data, how do we know whether we have real differences? experimental hypothesis. We can ask about the likelihood of observing some difference by chance. If the observation is unlikely to have occurred by chance, then it is likely that there is a real difference between our groups or experimental conditions. But what does a real difference mean? Inferential Statistics A tool for assessing the status of an

Measurement Issues Null vs Alternative Hypotheses Some variable is normally distributed in the population. This means that the distribution of scores has a mean ( ) and a standard deviation ( ).

Measurement Issues Null vs Alternative Hypotheses Some variable is normally distributed in the population. This means that the distribution of scores has a mean ( ) and a standard deviation ( ). For example, body weights of males is normally distributed with a mean close to 200 pounds.