Characteristics of Experimental Studies: Manipulation, Control, Randomization

Research Design Unit SIX Part 1 EXPERIMENTAL QUASI-EXPERIMENTAL

What study is an experimental study? Experimental studies are ones where researchers introduce an intervention and study the effects. Experimental studies are usually randomized, meaning the subjects are grouped by chance. Randomized controlled trial (RCT): Eligible people are randomly assigned to one of two or more groups

Characteristics of True Experiments A true experimental design or randomized controlled trial (RCT) is characterized properties: 1. Manipulation the experimenter does something to some subjects that is, there is some type of intervention. (the researcher independent variable by introducing a treatment or intervention); by the following manipulates the

2. Control : (including the use of a control group that is not given the intervention and represents the comparative counterfactual); Experimenters can expose the control group to various conditions, including no treatment; an alternative treatment; a placebo or pseudo intervention; standard care ); different doses of the treatment; and a wait-list (delayed treatment) condition. treatment ( usual

3. Randomization: the experimenter assigns subjects to a control or experimental condition on a random basis. Random assignment can be accomplished by flipping a coin or pulling names from a hat. Researchers typically either use computers to perform the randomization or rely on a table of random numbers.

Experimental designs are considered by many to be the gold standard because they come closer than any other design in meeting the criteria for inferring causal relationships.

How can you tell if a study is Experimental? Researchers usually indicate in the method section of their reports that they have used an experimental design, but they may also say they used a randomized design or an RCT. If such terms are missing, you can conclude that a study is experimental if the report says that the study purpose was to test, evaluate, or examine the effectiveness of an intervention, treatment, or innovation, AND if individual participants were put into groups (or conditions) at random. exposed to different

Experimental Designs Symbolic presentation of research design Experimental designs can be explained using symbols to represent features of the design. In these diagrams, the convention is that R= random assignment of subjects to groups, O = observation or measurement of dependent variable X =experimental treatment or intervention graphically

Three types of true experimental designs : The pretest - posttest control group design. The posttest-only control group design, and The Solomon four group design.

The pretestposttest design The most frequently used experimental design, is the pretest posttest design, in this design, 1- the subjects are randomly assigned to groups, 2- a pretest is given to both groups 3- the experimental group receives the experimental treatment and the comparison group receives the routine treatment or no treatment and 4- a post test is given to both groups . So, for example, a pretest posttest design would be depicted as follows: R O1 X O2 (experimental group) R O1 O2(comparison group)

Post test-only control group design The most basic randomizing subjects to different groups and then measuring the dependent variable. In this design, 1- subjects are randomly assigned to groups 2- the experimental group receives the experimental treatment, 3-the comparison group receive the ordinary treatment or no treatment and 4- a posttest is given to both groups R X O1 (experimental group) R O1(comparison group) experimental design involves

Solomon Four-Group Design In the Solomon four-group design, (a) subjects are randomly assigned to one of the four groups; (b) two of the groups, experimental group 1 and comparison group 1, are pretested; (c) two of the groups, experimental group 1 and experimental group 2, receive the experimental treatment, whereas two of the groups, comparison group 1 and comparison group 2, receive the routine treatment or no treatment; and (d) a posttest is given to all four groups.

R O1 X O2 (Experimental group 1) R O1 O2 (Comparison group 1) R X O2 (Experimental group 2) R O2 (Comparison group 2)

Group Data collection before Data collection after Experimental with pretest x x Experimental without pretest x Control with pretest x x Control without pretest x Solomon four-group experimental design.

Sometimes researchers are not able to randomly assign subjects to groups, or for various reasons no comparison group experimental study. Quasi-experimental designs are those in which there is either no comparison group or subjects are not randomly assigned to groups is available for an

QUASI-EXPERIMENTS The word Quasi indicates similarity. A quasi- experimental design is similar to experimental but it is not the same. Quasi-experiments (called controlled trials without randomization in the medical literature), also involve an intervention; however, quasi-experimental designs lack randomization, the signature of a true experiment. Some quasi-experiments even lack a control group. The signature of a quasi-experimental design, then, is an intervention randomization. in the absence of

Quasi-experimental design is a useful tool in situations where true experiments cannot be used for ethical or practical reasons.

How experimental? can you tell if a study is quasi- Researchers do not always identify their studies as quasi-experimental. If a study involves an intervention and if the report does not explicitly mention random assignment, it is probably safe to conclude that the design is quasi-experimental. Oddly, some researchers experimental designs as quasi-experimental. If individual subjects are randomized to groups or conditions, the design is not quasi-experimental. misidentify true

Example: A researcher might choose a group of patients with diabetes on one hospital experimental group and a group of patients with diabetes on another floor for comparison group. The experimental treatment would be administered to the experimental group ;the comparison group would receive no treatment or some alterative treatment. floor for the

Two types of quasi-experimental designs are discussed next: Nonequivalent control group design and Time-series design

Non-equivalent Control Group Design The nonequivalent control group design is similar to the pretest-posttest control group design except there is no random assignment of subjects to the experimental and comparison groups.

O1 X O2 (Experimental group) O1 O2 (Comparison group) A researcher might choose a group of patients with diabetes on one hospital floor for the experimental group and a group of patients with diabetes on another floor for the comparison group. The experimental administered to the experimental comparison group would treatment or some alternative treatment. treatment would group; the be the receive routine

Threats nonequivalent control group design are history, testing, maturation, and instrumentation change. The biggest threat to internal validity is selection bias. The two groups may not have been similar at the beginning of the study. It is possible, however, to test statistically for differences in the groups. For example, it could be determined if the ages and educational backgrounds of the subjects in both groups were similar. If the groups were similar at the beginning of the study, more confidence could be placed in a cause-and-effect relationship between variables. to internal validity controlled by the

2-Time Series Designs In a time series design, the researcher periodically observes or measures the subjects, the experimental treatment is administered between two of observations. there is no comparison group; information on the dependent variable is collected over a period of time before and after the treatment. O1 O2 O3 X O4 O5O6 Example A researcher might assess the pain levels of a group of clients with low back pain. After 3 weeks of pain assessment (O1 O2 O3 ), subjects could be taught a special exercise to alleviate low back pain (X). During the next 3 weeks, pain levels would again be measured (O4 O5O6)

The results of this study would help the researcher determine if low back pain persists, if a specific exercise is effective in reducing low back pain, and if the effectiveness of the exercise persists. The time-series design observations or measurements of the dependent variable helps strengthen the validity of the design. The greatest threats to validity are history and testing with its numerous

In evaluating the results of quasi-experiments, it is important to ask whether it is plausible that factors other than the intervention caused or affected the outcomes (i.e., whether there are rival hypotheses for explaining the results).

Validity of the experimental designs Study validity concerns the extent to which appropriate inferences can be made. Threats to validity are reasons that an inference could be wrong. Internal validity of an experimental design concerns the degree to which changes variable(effect)can be attributed to the independent variable (cause). in the dependent External validity concerns the degree to which study results can be generalized to other people and settings.

Six threats to internal validity Threats to internal validity are factors other than the independent variable that influence the dependent variables. 1. Selection bias occurs when the study results are attributed to the experimental treatment but in fact , the results occur because of subject differences before the treatment. 2. History occur when experimental treatment occurs during the course of a study, and this event influence the dependent variable. some event besides the

3- Maturation is threat to internal validity when changes that occur within the subjects during an experimental study influence the study results . people may become older, taller, or sleeper from the time of pretest to the post test 4- The testing threat may occur in studies where the pretest is given or where subjects have knowledge of baseline data , testing refers to influence of the pretest on the post test scores .

For example if subjects were weighted and told their weight before experimental weight reduction program, these subjects might make some effort on their own to lose weight because they have overweight. This knowledge of baseline data could be considered If subjects were a pretest. discovered they are

5. Instrumentation between pretest and posttest caused by a change in the accuracy of the instrument or the judges ratings rather than as a result of the experimental treatment. training sessions for judges. Also , if mechanical instruments are sphygmomanometers, these instruments should be checked for accuracy through out the study. change, involves difference measurement used, such as

6- The mortality threat occurs when the subject do not complete the study, dropout rate is different between the comparison groups, and this difference in dropout rate influences the posttest results . There is no research design that will control for mortality because participants can never be forced to remain in a study . experimental and

Three threats to external validity 1- Hawthorne effect occur when the study participants respond in a certain manner because they are aware that they are being observed. It might be possible to control this threat by a double-blind experiment. 2- The experimenter effect. A researcher s characteristics or behaviors influence the subject behaviors(facial expression , clothing, age , gender, body build) 3- The reactive effects of the pretest threat, some times called measurement effect, responses to the experimental treatment are indirectly influenced by the pretest. occurs when subjects

34 elmor@hotmail.com