Quantitative Research Designs Experiments, Quasi-Experiments, & Factorial Designs Experimental research in communication is conducted in order to establish causal relationships between variables. Alternate explanations can be eliminated only when high control is exercised. Control, therefore, is the key characteristic of an experiment. The degree of control is contingent upon 1) the manipulation of the independent variable; 2) the creation of equivalent experimental groups; and 3) the control of possible extraneous variable influence. These three issues determine whether a design is a true experiment or quasi-experiment. When communication researchers investigate the causal effects of MORE THAN ONE INDEPENDENT VARIABLE on a DEPENDENT VARIABLE, factorial designs are created. The difference between a true experiment and a quasi-experiment is random assignment to groups. Random assignment means that each participant has an equal chance of being a member of an experimental group. In a true experiment, participants are randomly assigned to groups. Random assignment controls for possible variations in participants by equally distributing those variations in the different groups. If those variations are equally distributed in the groups, then there should be no discernable group mean differences based on those variations allowing us to isolate the variable(s) of interest. For example, if we are interested in whether or not a particular message design strategy is effective among freshman girls regardless of IQ, then by randomly assigning girls to our treatment and control groups and thus equalizing the group IQ’s, we can rule out IQ as a factor and make the claim that the message design strategy was effective (regardless of any other possible explanations). Without random assignment to groups, it is difficult to know whether the message design strategy was what was influencing freshman girl learning outcomes or some other factor. So, random assignment to groups allows us make stronger claims about the relationships between variables (in this case, the relationship between the message design strategy and freshman girl cognitive learning).
Experimental Designs In true experiments, participants are randomly assigned to groups. Usually these groups are 1) the treatment group and 2) the control group. The treatment group
receives the stimulus (the treatment; such as exposure to a message, some kind of training, or a new drug), but the control group does not receive the treatment. The two groups are compared to see if significant differences exist between them (that resulted from the treatment). In order to see if any changes have taken place based on the existence or nonexistence of the treatment, measurements are taken before and after administration of the treatment. Unfortunately, sometimes the pretest sensitizes people to the issue, and thus it is difficult to see if the treatment had an effect or not. For example, if I am interested in measuring people’s attitudes toward abortion after exposure to a Public Service Announcement about abortion, the pretest in which I ask a lot of questions about abortion may stimulate some people to go out and find out more information about abortion on their own. So what caused the change, the treatment or the pretest? If this is a worry, then one can use a posttest only design, or best yet, a Solomon Four design in which all possibilities are included. Of course, a Solomon Four design is expensive, so it might not be feasible to use a Solomon Four. Decisions, decisions… Pretest-Posttest Design Pretest
Treatment
Posttest
Treatment
O
O
O
Control
O
X
O
Pretest
Treatment
Posttest
Treatment
X
O
O
Control
X
X
O
Pretest
Treatment
Posttest
Treatment
O
O
O
Control
O
X
O
Treatment
X
O
O
Control
X
X
O
Posttest Only
Solomon Four
O = Given X = Not Given
Summary of True Experiments BASIC ASSUMPTION of Experimental Approach: Human behavior is NOT Random. IV is the suspect. DV is the victim. GOAL: To determine whether changes in an independent variable produce changes in a dependent variable. CONTROL is the central characteristic of an experiment. ESTABLISHING CAUSATION ...three requirements necessary for establishing causal relationship between IV and DV 1. IV must precede DV [time: A is before B]. 2. IV and DV must be shown to covary [not a spurious relationship: A ↔ B]. 3. Changes observed in DV must be result of changes in the IV (and not some other unknown variable) [isolate influence: only A causes the change in B and NOT C, D, E…]. EXERCISING CONTROL IN EXPERIMENTAL RESEARCH ...contingent on three issues 1. Manipulating exposure to an Independent Variable 2. Observing exposure to an Independent Variable 3. Ruling out initial differences between the conditions (creating equivalent experimental groups; similar types of people in similar settings tested at similar times) *** Random Selection ≠ Random Assignment Random Selection is a way of sampling, getting people to participate in your study. Random Assignment is a way of distributing participants to either a treatment or control group in order to control for extraneous variables.
Quasi-Experimental Designs 1. Posttest Only Design: there are two groups of people (but the participants were not separated into groups using random assignment). One group receives the treatment and the other group does not. Group Treatment Control
Pretest
Treatment
Posttest
X
O
O
X
X
O
2. Pretest/Posttest Nonequivalent Group Design: participants are not randomly assigned to the treatment and control groups. Both groups receive a pretest and a posttest, but only the treatment group receives the treatment. Group Treatment Control
Pretest
Treatment
Posttest
O
O
O
O
X
O
3. Time Series Design: multiple observations are made on one group of people both before and after the treatment. Is change because of the treatment or because of the natural changes that occur over time? A time series design can also be done using a control group/treatment group comparison. Pretest 1
Pretest 2
Pretest 3
Treatment
Posttest 1
Posttest 2
Posttest 3
O1
O2
O3
O4
O5
O6
O7
Factorial Design See http://www.socialresearchmethods.net/kb/expfact.php. Basically, the idea here is that you have at least 2 factors (IV’s) that potentially affect a DV. For example, IV’s = 1) SENSATION SEEKING and 2) NEED FOR COGNITION, DV = SAFE SEX PRACTICE. We would like to know who is more likely to practice safe sex. Is it people who are high or low in sensation seeking, high or low in need for cognition, or low sensation seeking/low need for cognition, or high sensation seeking/low need for cognition, or low sensation seeking/high need for cognition, or high sensation seeking/high need for cognition. Basically we have 2 possible main effects [the effects of the individual IV’s]: 1) high or low in need for cognition AND 2) high or low in sensation seeking. And we have 1 possible interaction effect [an effect from the combination of IV’s]: 1) low sensation seeking/low need for cognition, 2) high sensation seeking/low need for cognition, 3) low sensation seeking/high need for cognition, or 4) high sensation seeking/high need for cognition. Each IV is a factor. Factors can have multiple levels [amounts, intensities, quantities…]. For example, SENSATION SEEKING could be conceptualized as having multiple levels instead of just 2 (EXTREMELY LOW, LOW, MEDIUM, HIGH, EXTREMELY HIGH). The conceptualization is based on theory. The following is a 2 x 2 Factorial Design [the 2 represents the number of levels of a factor]: A 2 x 2 Factorial Design Need for Cognition
Sensation Seeking
Low
High
Low
Group 1 Average
Group 3 Average
High
Group 2 Average
Group 4 Average
A 3 x 2 Factorial Design Need for Cognition
Sensation Seeking
Low
Medium
High
Low
Group 1 Average
Group 3 Average
Group 5 Average
High
Group 2 Average
Group 4 Average
Group 6 Average
You need to have equal numbers of people in each cell. You need to have a sufficiently large sample so that each cell gets filled. You need a strategy for distinguishing levels of a factor. Factorial Designs are tested using regression.
