Jackson S.L. Research Methods and Statistics: A Critical Thinking Approach
Подождите немного. Документ загружается.
312
■ ■
CHAPTER 11
of words. When rote rehearsal is used, word
type makes a large difference; concrete words
are recalled very well and abstract words very
poorly.
Graph for Critical Thinking Check 11.1, item 3
0
2
4
6
8
10
ConcreteAbstract
Number of Words
Recalled
Word Type
Rote
Rehearsal Type
Imagery
11.2
1. Tukey’s HSD would not need to be calculated
for the main effect of the variable with two lev-
els because if there is a significant main effect for
a variable with two levels, then we know that
the difference between those two levels is signifi-
cant. We would need to calculate Tukey’s HSD
for the variable with four levels to determine
exactly which groups among the four differed
significantly from each other.
2. a. This is a 2 3 design.
Gender Row Means
Practice Female Male (Practice)
1 hour 1,778.125 1,763.375 1,770.75
2 hours 1,512.375 1,764.25 1,638.31
3 hours 1,182.75 1,662 1,422.38
Column 1,491.08 1,729.88
means
(gender)
b. Two-Way ANOVA Summary Table
Source df SS MS F
Factor A 1 684,264 684,264 9.68
(gender)
Factor B 2 989,504 494,752 7.00
(practice)
A × B 2 489,104 244,552 3.46
Error 42 2,967,768 70,661.143
Total 47 5,130,640
c. Gender: F(1, 42) 9.68, p < .01
Practice: F(2, 42) 7.00, p < .01
Interaction: F(2, 42) 3.46, p < .05
d. The significant main effect of gender indicates
that females performed more quickly than
males. The significant main effect of practice
indicates that as the amount of time spent
practicing increased, reaction time decreased.
The significant interaction effect indicates
that practice affected only females; the more
females practiced, the more quickly they
responded. However, practice did not affect
males; reaction times for males were consist-
ent across the various practice conditions.
e. Eta-squared was .13 for gender, .19 for
practice, and .095 for the interaction. Thus,
overall, the proportion of variance in the
dependent variable accounted for by the
independent variables is .415, or 41.5%.
f.
1,000
1,200
1,400
1,600
1,800
2,000
321
Reaction Time (ms)
Hours of Practice
Females
Males
Check your knowledge of the content and key terms
in this chapter with a practice quiz and interactive
flashcards at http://academic.cengage.com/
psychology/jackson, or, for step-by-step practice and
information, check out the Statistics and Research
Methods Workshops at http://academic.cengage
.com/ psychology/workshops.
WEB RESOURCES
10017_11_ch11_p290-315.indd 312 2/1/08 1:32:18 PM
Complex Experimental Designs
■ ■
313
For hands-on experience using the research methods
described in this chapter, see Chapters 7 “Factorial
Designs I” and 8 “Factorial Designs II” in Research
Methods Laboratory Manual for Psychology, 2nd ed.,
by William Langston (Wadsworth, 2005) or Lab 11
“Factorial Designs” in Doing Research: A Lab Manual
for Psychology, by Jane F. Gaultney (Wadsworth,
2007).
LAB RESOURCES
For hands-on experience using statistical software to
complete the analyses described in this chapter, see
Chapter 9 “Two-Factor ANOVA” and Exercises 9.1
“Interpreting Results,” 9.2 “Creating Patterns,” 9.3
“Computational Method for the Two-Factor
ANOVA,” and 9.3 “The Mixed Two-Factor Design”
in The Excel Statistics Companion Version 2.0 by
Kenneth M. Rosenberg (Wadsworth, 2007).
STATISTICAL SOFTWARE RESOURCES
STUDY GUIDE REVIEW EXERCISES
(Answers to exercises appear in Appendix C.)
STUDY GUIDE FILL-IN SELF TEST
Answer the following questions. If you have trouble
answering any of the questions, restudy the relevant
material before going on to the multiple-choice self
test.
1. The notation that indicates how many independ-
ent variables were used in a study and how
many levels there were for each variable is called
.
2. An effect of a single independent variable is a
.
3. In a 4 × 6 factorial design, there are
independent variables, one with
levels and one with levels.
4. In a two-way randomized ANOVA, there is the
possibility for
main effect(s)
and interaction effects.
Chapter 11
■
Study Guide
CHAPTER 11 SUMMARY AND REVIEW: COMPLEX EXPERIMENTAL DESIGNS
In this chapter, designs using more than one indepen-
dent variable were described. The chapter discussed
several advantages of using such designs and intro-
duced the concepts of factorial notation, main effects,
and interaction effects. After reading the section on
main and interaction effects, you should be able to
graph data from a factorial design and interpret what
the graph means. Additional topic coverage included
the statistical analysis of such designs using a two-
way ANOVA. The various calculations necessary to
compute a two-way randomized ANOVA were pre-
sented along with the assumptions of the test and a
description of how to interpret the results.
10017_11_ch11_p290-315.indd 313 2/1/08 1:32:19 PM
314
■ ■
CHAPTER 11
STUDY GUIDE MULTIPLE-CHOICE SELF TEST
Select the single best answer for each of the following
questions. If you have trouble answering any of the
questions, restudy the relevant material.
1. When we manipulate more than one independent
variable in a study, we:
a. will have significant main effects.
b. will have at least one significant interaction
effect.
c. are using a factorial design.
d. All of the above
2. In a study examining the effects of time of day
(morning, afternoon, or evening) and teaching
style (lecture only versus lecture with small
group discussion) on student attentiveness, how
many main effects are possible?
a. 3
b. 6
c. 5
d. 2
3. In a study examining the effects of time of day
(morning, afternoon, or evening) and teaching
style (lecture only versus lecture with small
group discussion) on student attentiveness, how
many interaction effects are possible?
a. 1
b. 2
c. 6
d. 5
4. In a study examining the effects of time of day
(morning, afternoon, or evening) and teaching
style (lecture only versus lecture with small
group discussion) on student attentiveness, the
factorial notation would be:
a. 2 2.
b. 2 3.
c. 2 5.
d. 3 3.
5. A 2 4 5 6 factorial design has
potential main effects.
a. 2
b. 3
c. 4
d. 24
6. An experiment with three independent variables,
each with three levels is a
design.
a. 2 3
b. 3 3
c. 2 2 2
d. 3 3 3
7. If the lines in a graph are not parallel, then there
is most likely a(n):
a. main effect of variable A.
b. main effect of variable B.
c. interaction effect.
d. All of the above
8. A two-way randomized ANOVA is to
as a
two-way repeated measures ANOVA is to .
a. two independent variables manipulated
between-participants; two dependent
variables manipulated within-participants
b. two dependent variables manipulated
between-participants; two independent
variables manipulated within-participants
c. two independent variables manipulated
between-participants; two independent
variables manipulated within-participants
d. two dependent variables manipulated
between-participants; two dependent
variables manipulated within-participants
9. When the effect of one independent variable
depends on the level of the other independent
variable we have observed a(n):
a. main effect of one variable.
b. main effect of a level of an independent
variable.
c. interaction effect.
d. All of the above
10. How many conditions would there be in a facto-
rial design with three levels of factor A and three
levels of factor B?
a. 6
b. 3
c. 9
d. Unable to determine
5. In a two-way ANOVA, the sum of the squared
deviations of each score minus its cell mean is the
.
6. In an ANOVA, we use to meas-
ure effect size.
10017_11_ch11_p290-315.indd 314 2/1/08 1:32:20 PM
Complex Experimental Designs
■ ■
315
STUDY GUIDE SELF-TEST PROBLEMS
1. The following ANOVA table corresponds to an
experiment with two factors; 1) Time of Day
(morning, afternoon, or evening) and 2) Type of
Teaching Method (lecture only or lecture with
small group activities). The attention level (on a
0-10 scale) of college students during the morn-
ing, afternoon, or evening is measured in each of
the teaching method conditions. This is a com-
pletely between-participants design. The scores
for the five participants in each group follow.
Lecture Only/Morning: 8, 9, 9, 9, 10
Lecture Only/Afternoon: 5, 6, 7, 8, 9
Lecture Only/Evening: 5, 5, 6, 7, 7
Lecture-Small Group/Morning: 3, 4, 5, 6, 7
Lecture-Small Group/Afternoon: 5, 6, 6, 6, 7
Lecture-Small Group/Evening: 7, 7, 8, 9, 9
The sums of squares are provided in the follow-
ing table. However, for practice, see if you can
correctly calculate them by hand.
ANOVA Summary Table
Source df SS MS F
A (Time) 1.67
B (Teaching Method) 7.50
A B 45.02
Within 32.00
Total
a. Construct the matrix showing the means in
each cell and provide the factorial notation.
b. Complete the ANOVA Summary Table.
c. Are the F
obt
significant at .05?
d. What conclusions can be drawn from the
F-ratio?
e. What is the effect size and what does this
mean?
f. Graph the means.
11. In a study with two levels of factor A, four levels
of factor B, and 5 participants in each condition,
the df error would be:
a. 39.
b. 32.
c. 8.
d. 40.
12. In a study with two levels of factor A, four levels
of factor B, and 5 participants in each condition,
the dfs for factors A and B, respectively, would
be
and .
a. 2; 4
b. 4; 4
c. 1; 4
d. 1; 3
10017_11_ch11_p290-315.indd 315 2/1/08 1:32:20 PM
316
Quasi-Experimental and
Single-Case Designs
12
CHAPTER
Conducting Quasi-Experimental Research
Nonmanipulated Independent Variables
An Example: Snow and Cholera
Types of Quasi-Experimental Designs
Single-Group Posttest-Only Design
Single-Group Pretest/Posttest Design
Single-Group Time-Series Design
Nonequivalent Control Group Posttest-Only Design
Nonequivalent Control Group Pretest/Posttest Design
Multiple-Group Time-Series Design
Internal Validity and Confounds in Quasi-Experimental Designs
Statistical Analysis of Quasi-Experimental Designs
Developmental Designs
Cross-Sectional Designs
Longitudinal Designs
Sequential Designs
Conducting Single-Case Research
Types of Single-Case Designs
Reversal Designs
ABA Reversal Designs • ABAB Reversal Designs
Multiple-Baseline Designs
Multiple Baselines Across Participants • Multiple Baselines Across
Behaviors • Multiple Baselines Across Situations
Summary
10017_12_ch12_p316-338.indd 316 2/1/08 1:34:06 PM
Quasi-Experimental and Single-Case Designs
■ ■
317
Learning Objectives
• Describe how quasi-experimental designs differ from correlational and
experimental designs.
• Explain what a participant variable is.
• Differentiate single group designs and nonequivalent control group
designs.
• Describe advantages and disadvantages of posttest-only designs and
pretest/posttest designs.
• Explain a time-series design.
• Describe the differences among cross-sectional, longitudinal, and sequen-
tial developmental designs.
• Describe advantages and disadvantages of ABA versus ABAB reversal
designs.
• Differentiate multiple-baseline designs (i.e., across participants, across
behaviors, and across situations).
Q
uasi-experimental research can be thought of as an intermediate
point between correlational and true experimental research. As
such, it allows us to draw slightly stronger conclusions than we
might with correlational research. We can say there is more than a simple
relationship between variables, but we cannot draw as strong a conclusion
as we can with true experimental research. We cannot say that we have
observed a causal relationship between variables. Quasi-experimental
research frequently fits into the category of field research; that is, it often
involves conducting research in more naturalistic settings. In this chapter,
we will begin with a brief discussion of what quasi-experimental research
is and how it differs from correlational and experimental research. We will
then describe various types of quasi-experimental research designs and
discuss how the quasi-experimental method limits internal validity in a
study. Next, we will discuss three types of quasi-experimental designs used
by developmental psychologists—cross-sectional designs, longitudinal
designs, and sequential designs—which all use age as a nonmanipulated
independent variable. Finally, we will discuss another type of field research,
single-case research. As with quasi-experimental designs, we will discuss
the value of single-case research and describe various types of single-case
designs.
Conducting Quasi-Experimental Research
The term quasi (meaning “having some but not all of the features”) preced-
ing the term experimental indicates that we are dealing with a design that
resembles an experiment but is not exactly an experiment. How does a
quasi-experimental design differ from an experimental design? Sometimes
the difference is the lack of any control group or comparison group. In other
10017_12_ch12_p316-338.indd 317 2/1/08 1:34:07 PM
318
■ ■
CHAPTER 12
words, only one group is given a treatment and then assessed. At other times,
the independent variable is not a true manipulated independent variable;
instead, it is a participant variable or a nonmanipulated independent vari-
able. Finally, there may be designs considered quasi-experimental because
participants are not randomly assigned to conditions. In other words, they
are already part of a group, and the researcher attempts to manipulate a vari-
able between preexisting groups.
Nonmanipulated Independent Variables
In some quasi-experiments, the researcher is interested in comparing
groups of individuals (as is done in an experiment), but the groups occur
naturally. In other words, participants are not assigned randomly to the
groups. Notice the difference between this and correlational research.
We are not simply looking for relationships between variables—such
as between smoking and cancer. In quasi-experimental research, we are
testing a hypothesis—for example: Individuals who have smoked for
20 years have a higher incidence of respiratory illness than nonsmokers.
We would then randomly select a group of individuals who had smoked
for 20 years and a group of individuals who had never smoked to serve
as a control. Thus, rather than simply looking for a relationship between
smoking and cancer/illness, we are comparing two groups to test a
hypothesis.
The independent variable is referred to as a nonmanipulated independ-
ent variable because participants are not randomly assigned to the two
groups. We are not truly manipulating smoking; participants come to the
study as either smokers or nonsmokers. However, we do make compari-
sons between the groups. Thus, the study has the intent and “flavor” of an
experiment without being a true experiment. Nonmanipulated independent
variables are also known as participant variables. A participant variable, you
may recall from Chapter 1, is a characteristic of the participant that cannot
be changed, such as ethnicity, gender, age, or political affiliation. If a study
is designed to assess differences in individuals on some participant vari-
able, by default it is a quasi-experiment and not a true experiment because
it uses a nonmanipulated independent variable; that is, participants are not
randomly assigned to conditions.
An Example: Snow and Cholera
In the 1850s in London, England, there were frequent outbreaks of cholera,
an infection of the small intestine. The cause at the time was unknown, but
the common theory was that cholera was somehow spread as people came
in contact with cholera victims and shared or breathed the same air. This
was known as the effluvia theory. John Snow, in his quest for the cause of
cholera, had an alternative hypothesis (Goldstein & Goldstein, 1978). Snow
thought that people contracted cholera by drinking contaminated water. He
based his hypothesis on the observation that, of the several different water
companies serving London, some provided water from upstream (it had not
nonmanipulated inde-
pendent variable The
independent variable in a
quasi-experimental design in
which participants are not ran-
domly assigned to conditions
but rather come to the study as
members of each condition.
nonmanipulated inde-
pendent variable The
independent variable in a
quasi-experimental design in
which participants are not ran-
domly assigned to conditions
but rather come to the study as
members of each condition.
10017_12_ch12_p316-338.indd 318 2/1/08 1:34:08 PM
Quasi-Experimental and Single-Case Designs
■ ■
319
yet passed through the city and possibly become contaminated), whereas
others used water from downstream (after it had passed through the city
and possibly become contaminated).
To test this hypothesis, Snow used a quasi-experimental design.
Obviously, it was not feasible to use a true experimental design because it
would have been impossible to randomly assign different houses to contract
with a specific water company. Snow therefore had to look at houses that
already received their water from a downstream company versus houses
that received water from upstream. You should begin to see some of the
problems inherent in quasi-experimental research. If people chose their
water company, then there was most likely a reason for the choice. In most
cases, the reason was socioeconomic: The wealthier neighborhoods used
upstream (more costly) companies, whereas the poorer neighborhoods used
downstream (less costly) companies. This obviously presented a problem
for Snow because he had no way of knowing whether differences in cholera
incidence were due to the different water companies or to something else
related to socioeconomic level, such as diet, living conditions, or medical
care.
Luckily for Snow, he was able to find one neighborhood in which socio-
economic status was stable but different houses received water from two
different companies in an unsystematic manner. In other words, the choice
of water companies in this neighborhood appeared to be random. It was
so random that, in some cases, the choice of water company varied from
house to house on a single street. Here was a naturally occurring situation
in which socioeconomic level was controlled and water company varied. It
was important, however, to ensure that not only the water company but also
the contamination level of the water varied. Snow was lucky in this respect,
too, because one company had moved upstream after a previous cholera
epidemic, whereas the other company had stayed downstream. Snow calcu-
lated the number of deaths by cholera for individuals receiving water from
upstream versus those receiving water from downstream. He found that
there were 37 deaths per 10,000 households for the upstream company and
315 deaths per 10,000 households for the downstream company. Therefore,
it appeared that water contamination was responsible for the spread of
cholera.
As a review, the nonmanipulated independent variable in Snow’s study
was water company. This was a participant variable because individuals
came to the study with their choice of water company already established.
The dependent variable was number of deaths by cholera. Snow observed
a difference in death rates between the two companies and concluded that
the type of water (more contaminated versus less contaminated) appeared
to be the cause. Snow was particularly lucky because of the naturally
occurring situation in which socioeconomic level was controlled but water
company varied. This type of control is often lacking in quasi-experimental
research. However, even with such control, there is still not as much con-
trol as in an experiment because participants are not randomly assigned
to conditions. Therefore, it is still possible for uncontrolled differences
between the groups to affect the outcome of the study.
10017_12_ch12_p316-338.indd 319 2/1/08 1:34:08 PM
320
■ ■
CHAPTER 12
Types of Quasi-Experimental Designs
The quasi-experimental design has several possible variations (Campbell
& Stanley, 1963). One distinction is whether there are one or two groups of
participants. A second distinction has to do with how often measurements
are taken. We will begin by discussing quasi-experimental designs in which
only one group of participants is observed. These designs include the single-
group posttest-only design, the single-group pretest/posttest design, and the
single-group time-series design. We will then consider designs that use two
groups—referred to as nonequivalent control group designs. These include the
nonequivalent control group posttest-only design, the nonequivalent control
group pretest/posttest design, and the multiple-group time-series design.
Single-Group Posttest-Only Design
The single-group posttest-only design is the simplest quasi-experimental
design. As the name implies, it involves the use of a single group of
participants to whom some treatment is given. The participants are then
single-group posttest-only
design A design in which
a single group of participants
is given a treatment and then
tested.
single-group posttest-only
design A design in which
a single group of participants
is given a treatment and then
tested.
IN REVIEW Quasi-Experimental Versus Correlational Methods
METHOD VARIABLES CONCLUSIONS CAUTIONS
Correlational method Two measured variables. The variables may be We cannot conclude
related in some way. that the relationship
is causal.
Quasi-experimental Typically one nonmanipulated Systematic differences have Due to confounds inherent
method independent been observed between two in the use of nonmanipulated
variable and one or more groups, but we independent variables, there
measured dependent cannot say that the may be alternative
variable. nonmanipulated independent explanations for the results.
variable definitely caused
the differences.
CRITICAL
THINKING
CHECK
12.1
1. Which of the following variables are always participant variables
(nonmanipulated independent variables)?
gender ethnicity
religious affiliation visual acuity
amount of time spent studying amount of alcohol consumed
2. How does the quasi-experimental method allow us to draw slightly
stronger conclusions than the correlational method? Why is it that
the conclusions drawn from quasi-experimental studies cannot be
stated in as strong a manner as those from a true experiment?
10017_12_ch12_p316-338.indd 320 2/1/08 1:34:09 PM
Quasi-Experimental and Single-Case Designs
■ ■
321
assessed on the dependent variable. Research in education is frequently of
this type. For example, some new educational technique—such as interactive
learning, outcomes learning, or computer-assisted learning—is proposed,
and school systems begin to adopt this new method. Posttest measures are
then taken to determine the amount learned by students. However, there is
neither a comparison group nor a comparison of the results to any previous
measurements (usually because what is learned via the new method is so
“different” from the old method that the claim is made that comparisons are
not valid). You should see the problem with this type of design. How can we
claim a method is better when we cannot compare the results for the group
who participated with the results for any other group or standard? This
design is open to so many criticisms and potential flaws that results based
on this type of study should always be interpreted with caution.
Most frequently, you will see single-group posttest-only designs reported
in popular literature, where they are frequently misinterpreted by those
who read them. How many times have you read about people who lived
through a certain experience or joined a particular group claiming that the
experience or the group had an effect on their lives? These are examples of
single-group posttest-only designs. Single-group posttest-only designs can-
not be used to draw conclusions about how an experience has affected the
individuals involved. The change in their lives could be due to any number
of variables other than the experience they lived through or the program
they went through.
Single-Group Pretest/Posttest Design
The single-group pretest/posttest design is an improvement over the
posttest-only design in that measures are taken twice—before the treatment
and after the treatment. The two measures can then be compared, and any
differences in the measures are assumed to be the result of the treatment. For
example, if we had a single group of depressed individuals who wanted to
receive treatment (counseling) for their depression, we would measure their
level of depression before the treatment, have them then participate in the
counseling, and then measure their level of depression after the treatment.
Can you think of possible problems with this design? The greatest problem
is the lack of a comparison group. With no comparison group, we do not
know whether any observed change in depression is due to the treatment
or to something else that may have happened during the time of the study.
For example, maybe the pretest depression measure was taken right after
the holidays, when depression is higher for many people. Therefore, the
participants might have scored lower on the posttest depression measure
regardless of whether they received counseling or not.
Single-Group Time-Series Design
The single-group time-series design involves using a single group of partici-
pants, taking multiple measures over a period of time before introducing the
treatment, and then continuing to take several measures after the treatment.
single-group pretest/post-
test design A design in
which a single group of partici-
pants takes a pretest, then re-
ceives some treatment, and then
takes a posttest measure.
single-group pretest/post-
test design A design in
which a single group of partici-
pants takes a pretest, then re-
ceives some treatment, and then
takes a posttest measure.
single-group time-series
design A design in which a
single group of participants is
measured repeatedly before and
after a treatment.
single-group time-series
design A design in which a
single group of participants is
measured repeatedly before and
after a treatment.
10017_12_ch12_p316-338.indd 321 2/1/08 1:34:09 PM