Psychological Assessment Test Reliability

M.A. Semester IV Paper II Psychological Assessment Test Reliability Topics covered- The concept of Reliability Methods of Reliability Test Retest Reliability Parallel Form Reliability Split Half Reliability Spearman Brown formula Kuder Richardson formula Coefficient Alpha Scorer Reliability Consulted Books Prof. Dhananjay Kumar Department of Psychology DDU Gorakhpur University

The Concept of Reliability Reliability refers to the consistency of scores obtained by the same persons when re-examined with the same test on different occasions, or with different sets of equivalent items, or under other variable examining conditions. ( Anastasi, 1976) The concept of test reliability underlies the estimation of the error of measurement of test score, where one can predict the range of fluctuation likely to occur in the score of the testee as a result of some uncontrolled factors or chance factors. The concept of reliability of a test has been used to cover several aspects of score consistency. Test reliability indicates the extent to which individual differences in test scores are attributable to true differences in the characteristics under consideration and the extent to which they are attributable to chance errors . ( Anastasi, 1976)

the reliability coefficient is defined as the ratio of true score variance to the total variance of test scores. It denotes the proportion of variance of test scores that varies due to, or accounted for by, variability in true score. In other words, measures of test reliability also yield an estimate about what proportion of the total variance of test scores is error variance. Factors considered error variance in one test would be considered true variance for another test. For example, if we are interested in measuring mood, then the changes in scores on the items of different mood state would be relevant for that test and would be part of the true variance of the scores, on the other hand, if the test is designed to measure anxiety the same mood states would fall under the heading of error variance.

Methods of Reliability Test Retest Reliability A. Parallel Form Reliability B. Split Half Reliability C. Spearman Brown formula Kuder Richardson formula D. Internal consistency : Coefficient Alpha E. Scorer Reliability

Test Retest Reliability The method provide the reliability of test scores by repeating the identical test on a second occasion. The reliability coefficient here is simply the correlation between the scores obtained by the same tests, administered on two occasions on identical group of subjects. The error variance gives an assessment of the random fluctuations of performance from one test session to the other. These variations can be attributed in part to uncontrolled testing conditions, to some extent, however, they arise from changes in the condition of the subject himself, as illustrated by illness, fatigue, emotional strain etc. This reliability tells the extent to which scores on a test can be generalized over different occasions; the higher the reliability, there are less susceptibility that the scores will vary randomly due to daily changes in the condition of the subject or of the testing environment.

Retest correlations decrease progressively as the interval of administration of test lengthens, Short-range, random fluctuations that occur during intervals ranging from a few hours to a few months are generally included under the error variance of the test score. Thus, in checking this type of test reliability, an effort is made to keep the interval short. the interval between retests should rarely exceed six months.

Parallel Form Reliability To avoid the difficulties faced in test-retest reliability parallel (alternate) forms of the test is used. The same subjects are tested with one form of the test on the first occasion and with another, comparable form on the second occasion. The correlation between the scores obtained on the two forms represents the reliability coefficient of the test. Such a reliability coefficient is a measure of both temporal stability and consistency of response to different item samples ( parallel test forms). This coefficient thus combines two types of reliability.

Alternate-form reliability should also be reported the length of the interval between test administrations. If the two forms are administered in immediate succession, the resulting correlation shows reliability across forms only, not across occasions. Thus, the error variance only represents fluctuations in performance from one form of test to another, but it does not tells about the fluctuations over time. In the alternate forms, care should be taken to ensure that they are truly parallel. The tests should contain the equal number of items, and the items should cover the same type of content. The level of difficulty of the items should also be equal. Instructions, time limits, illustrative examples, format, and all other aspects of the parallel test must be same to each other.

Split Half Reliability In split-half reliability, a test is divided into two halves using some pre-specified criteria and are scored separately. The results of one half of the test are then correlated with the results of the other. This type of reliability is also called a coefficient of internal consistency, since only a single administration of a single form is conducted. Generally people uses two methods to find split-half reliability: divide the test in first-half and second-half or divide the test in odd-even items. For ease in computing scores for the different halves, people prefer to calculate a score for the first half of the items and another score for the second half. But, this method can face problems when items on the second half of the test are more difficult than items on the first half. If the items of the test are devised so that they are progressively more difficult, then researcher should use the odd-even system, whereby one sub-score is obtained for the odd-numbered items in the test and another for the even-numbered items.

After getting the scores of both halves obtained for each person, they may be correlated with each other. however, this correlation actually gives the reliability of only a half-test. For example, if a test consists of 50 items, the correlation is computed between two sets of scores based on two halves of 25 items. Whereas, In both test-retest and alternate-form reliability, on the other hand, each score is based on the full number of items in the test. There are different methods to find split-half reliability Spearman Brown formula Kuder Richardson formula

Spearman Brown Formula In split- half reliability scores are obtained by the two splitted halves of a single test, and they are correlated. But this would be a reliability coefficient for a half test To estimate the reliability of whole test, Spearman- brown formula is used. It estimates what the reliability would be if each test half were made twice as long.

Kuder Richardson formula Many psychological tests are developed using dichotomous items in which individual pass or fail on the test items. A 1 will be assigned for pass items and 0 for failed items. The score will be number of passed items. To assess reliability coefficient in this type of situation a formula was developed by Kuder and Richardson. The Kuder-Richardson formula 20 is a measure of the internal consistency, or homogeneity of the test items. It has been shown that if a test is split in all possible ways, the average of all the split half reliability coefficient with the Spearman Brown correction is the Kuder-Richardson formula.

Internal Consistency Method: Coefficient Alpha Internal consistency methods estimate reliability of a test based solely upon the number of items in the test and the average correlation among them. Thus the internal consistency methods are mathematically linked to the split half method. Coefficient alpha is most widely used and general form of internal consistency estimate. It utilizes the mean reliability coefficient one would obtain from all possible split half. What is the difference between split half methods and coefficient alpha? Split half tests compare one half of the test to another; whereas internal consistency estimates compare each item to every other item.

Scorer Reliability Some tests are fairly subjective in their scoring and require considerable judgment on the part of the scorer. Scorer reliability is the degree of agreement or consistency between two or more scorers (or judges or raters) with regard to a particular measure. The two sets of scores obtained by each examiner are completed in the usual way and the resulting correlation coefficient is known as scorer reliability. Reference to levels of inter-scorer reliability for a particular test may be published in the test s manual or elsewhere. the scorers need to be trained to score the protocols, especially with scoring sophisticated psychological techniques such as the Rorschach inkblot test, and the resulting correlation coefficient can be in part reflective of the effectiveness of the training. The source of error variance in scorer reliability is interscorer difference.

Consulted Books Anastasi, A. (1976). Psychological Testing (4th Ed.). Macmillan Publishing Co., Inc. New York. Cohen & Swerdlik (2009). Psychological Testing and Assessment: An Introduction to Tests and Measurement (7th Ed.). TMH. Domino, G. & Domino, M.L., (2006). Psychological Testing: An Introduction. Cambridge University Press. Kaplan, R.M. & Saccuzzo, D.P., (2009). Psychological Testing: Principles, Applications, and Issues (7th Ed.). Wadsworth, Cengage Learning.