Jackson S.L. Research Methods and Statistics: A Critical Thinking Approach
Подождите немного. Документ загружается.
302
■ ■
CHAPTER 11
the number of people in each of the factor B conditions. The SS
B
calculated
in Table 11.5 is 98. Notice that, as with factor A, n
B
is 16—the total number
of scores that contribute to the row means. In addition, as with factor A,
the mean square for factor B is calculated by dividing SS
B
by df
B
. The df
B
is
derived by taking the number of levels of factor B minus 1. This is 2 1 1,
and MS
B
is therefore 98/1 98.
We also have to consider the variance due to the interaction of factors
A and B, which will tell us whether or not there is a significant interaction
effect. The sum of squares interaction (SS
A B
) is the sum of the squared
difference of each condition mean minus the grand mean times the number
of scores in each condition. Because this gives us an estimate of the amount
of variance of the scores about their respective condition means, it includes
the amount of variance due to factor A, factor B, and the interaction. Thus,
after this sum is calculated, we must subtract the variance due solely to
factor A and the variance due solely to factor B. The definitional formula is
thus SS
A B
[(
X
C
X
G
)
2
n
C
] SS
A
SS
B
, where
X
C
is the mean for each
condition (cell),
X
G
is the grand mean, and n
C
is the number of scores in each
condition or cell. The calculation of SS
A B
is illustrated in Table 11.6. As can
be seen, the sum of squares for the interaction term is 50. We must divide
this number by its corresponding degrees of freedom. The degrees of free-
dom for the interaction are based on the number of conditions in the study.
In the present study, there are four conditions. To determine the degrees of
freedom across the conditions, we multiply the degrees of freedom for the
factors involved in the interaction. In the present case, factor A has 1 degree
of freedom, and factor B also has 1 degree of freedom. Thus, df
A B
(A 1)
(B 1) 1. Using this to determine MS
A B
, we find that 50/1 50.
sum of squares
interaction The sum of the
squared difference of each condi-
tion mean minus the grand mean
times the number of scores in
each condition. SS
A
and SS
B
are
then subtracted from this sum.
sum of squares
interaction The sum of the
squared difference of each condi-
tion mean minus the grand mean
times the number of scores in
each condition. SS
A
and SS
B
are
then subtracted from this sum.
TABLE 11.5 Calculation of SS
B
Using the Definitional Formula
SS
B
[(
X
B
X
G
)
2
n
B
]
[(4.5 6.25)
2
16] [(8 6.25)
2
16]
[(1.75)
2
16] [(1.75)
2
16]
[(3.0625)16] [(3.0625)16]
49 49
98
TABLE 11.6 Calculation of SS
A B
Using the Definitional Formula
SS
AB
[(
X
C
X
G
)
2
n
C
] SS
A
SS
B
[(4 6.25)
2
8] [(5 6.25)
2
8] [(10 6.25)
2
8] [(6 6.25)
2
8] 18 98
[(2.25)
2
8] [(1.25)
2
8] [(3.75)
2
8] [(0.25)
2
8] 18 98
[(5.0625)8] [(1.5625)8] [(14.0625)8] [(0.0625)8] 18 98
40.50 12.50 112.50 0.50 18 98
50
10017_11_ch11_p290-315.indd 302 2/1/08 1:32:11 PM
Complex Experimental Designs
■ ■
303
Last, we have to determine the amount of variance due to error—the
within-groups variance. As in a one-way ANOVA, the within-groups vari-
ance is an indication of the amount of variance of the scores within a cell
or condition about that cell mean. The sum of squares error (SS
Error
) is the
sum of the squared deviations of each score from its condition (cell) mean.
The definitional formula is SS
Error
(X
X
C
)
2
. The calculation of SS
Error
is
illustrated in Table 11.7. In the present study, SS
Error
is 36. We can now check
all of our calculations by summing SS
A
, SS
B
, SS
A B
, and SS
Error
: 18 98
50 36 202. We previously found that SS
Total
202, so we know that our
calculations are correct.
The df
Error
is determined by assessing the degrees of freedom within each
of the design’s conditions. In other words, the number of conditions in the
study is multiplied by the number of participants in each condition minus the
one score not free to vary, or AB(n – 1). In the present example, this is 4(8 – 1),
or 28. As a check, when we total the degrees of freedom for A, B, A B, and
error, they should equal df
Total
. In this case, df
A
1, df
B
1, df
A B
1, and
df
Error
28. They sum to 31, which is the df
Total
we calculated previously. To
determine MS
Error
, we divide SS
Error
by its degrees of freedom: 36/28 1.29.
Now that we have calculated the sum of squares, degrees of freedom, and
mean squares for each term, we can determine the corresponding F-ratios.
In a two-way ANOVA, there are three F-ratios: one for factor A, one for fac-
tor B, and one for the interaction of factors A and B. Each of the F-ratios is
determined by dividing the MS for the appropriate term by the MS
Error
. Thus,
for factor A (word type), the F-ratio is 18/1.29 13.95. For factor B (rehearsal
type), the F-ratio is determined in the same manner: 98/1.29 75.97. Last,
for the interaction, the F-ratio is 50/1.29 38.76. The definitional formulas
for the sums of squares and the formulas for the degrees of freedom, mean
squares, and F-ratios are summarized in Table 11.8. Table 11.9 shows the
ANOVA summary table for the data from the present study.
sum of squares error The
sum of the squared deviations of
each score from its group (cell)
mean; the within-groups sum of
squares in a factorial design.
sum of squares error The
sum of the squared deviations of
each score from its group (cell)
mean; the within-groups sum of
squares in a factorial design.
TABLE 11.7 Calculation of SS
Error
Using the Definitional Formula
ROTE/CONCRETE ROTE/ABSTRACT IMAGERY/CONCRETE IMAGERY/ABSTRACT
X (X
X
C
)
2
X (X
X
C
)
2
X (X
X
C
)
2
X (X
X
C
)
2
4 0 5 0 10 0 6 0
5 1 4 1 12 4 5 1
3 1 5 0 11 1 6 0
6 4 6 1 9 1 7 1
2 4 4 1 8 4 6 0
2 4 5 0 10 0 6 0
6 4 6 1 10 0 7 1
4 0 5 0 10 0 5 1
18 4 10 4
SS
Error
18 4 10 4 36
10017_11_ch11_p290-315.indd 303 2/1/08 1:32:12 PM
304
■ ■
CHAPTER 11
Interpreting the Two-Way Randomized ANOVA. Our obtained F-ratios
are all greater than 1.00. To determine whether they are large enough
to let us reject the null hypotheses, however, we need to compare our
obtained F-ratios with F
cv
. As we learned in Chapter 10, the underlying F
distribution is actually a family of distributions, each based on the degrees
of freedom between and within each group. Remember that the alterna-
tive hypotheses are that the population means represented by the sample
means are not from the same population. Table A.8 in Appendix A pro-
vides the critical values for the family of F distributions when .05 and
when .01. We use this table exactly as we did in the previous chapter.
That is, we use the df
Error
(remember, df
Error
is the degrees of freedom within
groups, or the degrees of freedom for error variance) running down the
left-hand side of the table and the df
Between
running across the top of the
table. F
cv
is found where the row and column corresponding to these two
numbers intersect. You might be wondering what df
Between
is in the present
example. It always represents the degrees of freedom between groups.
However, in a two-way ANOVA, we have three values of df
Between
: one for
TABLE 11.8 ANOVA Summary Table: Definitional Formulas
SOURCE df SS MS F
Factor A (word type) A 1 [(
X
A
X
G
)
2
n
A
]
SS
A
___
df
A
MS
A
_____
MS
Error
Factor B (rehearsal type) B 1 [(
X
B
X
G
)
2
n
B
]
SS
B
___
df
B
MS
B
_____
MS
Error
A B (A 1)(B 1) [(
X
C
X
G
)
2
n
C
] SS
A
SS
B
SS
AB
_____
df
AB
MS
AB
_____
MS
Error
Error AB(n 1) (X
X
C
)
2
SS
Error
_____
df
Error
Total N 1 (X
X
G
)
2
TABLE 11.9 Two-Way ANOVA Summary Table
SOURCE df SS MS F
Factor A (word type) 1 18 18 13.95
Factor B (rehearsal type) 1 98 98 75.97
A B 1 50 50 38.76
Error 28 36 1.29
Total 31 202
10017_11_ch11_p290-315.indd 304 2/1/08 1:32:12 PM
Complex Experimental Designs
■ ■
305
factor A, one for factor B, and one for the interaction term. Therefore, we
need to determine three values of F
cv
, one for each of the three terms in
the study.
To determine F
cv
for factor A (word type), we look at the degrees of free-
dom for factor A (df
A
1). This represents the degrees of freedom between
groups for variable A, or the number running across the top of Table A.8. We
move down the left-hand column to the df
Error
(df
Within
), which is 28. Where 1
and 28 intersect, we find that F
cv
for the .05 level is 4.20, and F
cv
for the .01
level is 7.64. This means that for our F
obt
to be significant at either of these
levels, it has to exceed the F
cv
for that alpha level. Because our F
obt
for factor
A exceeds both these values of F
cv
, it is significant at the .01 level. In APA
publication format, this is written as F(1, 28) 13.95, p .01. This means
that there was a significant main effect of factor A (word type). If we look at
the column means from Table 11.2 for word type, we see that participants
did better (remembered more words) when concrete words were used than
when abstract words were used. I have initially interpreted the main effect of
factor A at face value, but we will see when we interpret the interaction that
participants did not remember concrete words better in both of the rehearsal
type conditions.
We also need to determine F
cv
for variable B and for the interaction term.
Because the degrees for freedom are the same for all of the terms in this
study (1, 28), we use the same values of F
cv
. In addition, because the values
of F
obt
also exceed the F
cv
of 7.64 for the .01 level, we know that the F
obt
for
factor B and for the interaction term are also significant at the .01 level. Thus,
for factor B (rehearsal type), F(1, 28) 75.97, p .01, indicating a significant
main effect of rehearsal type. Referring to the row means from Table 11.2, we
see that participants remembered substantially more words when imagery
rehearsal was used than when rote rehearsal was used. Once again, I have
interpreted the main effect of factor B at face value, but we will see that the
interaction qualifies this interpretation. In other words, imagery rehearsal
led to better performance overall, but not when we break it down by word
type. Last, for the interaction term, F(1, 28) 38.76, p .01, indicating that
there was a significant interaction effect. When rote rehearsal was used,
word type made no difference; however, when imagery rehearsal was used,
the performance for the two word types varied. With imagery rehearsal,
participants remembered significantly more concrete words than abstract
words. To aid in your interpretation, try to draw the graph representing the
cell means for this 2 2 design.
Assumptions of the Two-Way Randomized ANOVA. The two-way rand-
omized ANOVA is used when you have a factorial design. The remaining
assumptions are as follows:
• All conditions (cells) contain independent samples of participants (in
other words, there are different participants in each condition).
• Interval or ratio data are collected.
• The populations represented by the data are roughly normally distributed.
• The populations represented by the data all have homogeneous variances.
10017_11_ch11_p290-315.indd 305 2/1/08 1:32:13 PM
306
■ ■
CHAPTER 11
Post Hoc Tests and Effect Size. As with a one-way ANOVA, post hoc tests
such as Tukey’s HSD (honestly significant difference) test are recommended.
For the present example, a 2 2 design, post hoc tests are not necessary,
because any significant main effect indicates a significant difference between
the two groups that make up that variable. In other words, because each
independent variable in the present study has only two values, a significant
main effect of that variable indicates significant differences between the two
groups. If one or both of the independent variables in a factorial design have
more than two levels, and the main effect(s) is (are) significant, then Tukey’s
HSD test should be conducted to determine exactly which groups differ sig-
nificantly from each other. In addition, it is also possible to use a variation
of the Tukey HSD test to compare the means from a significant interaction
effect. These calculations are beyond the scope of this book but can be found
in a more advanced statistics text.
© 2005 Sidney Harris, Reprinted with permission.
As noted in previous chapters, when a significant relationship is
observed, we should also calculate the effect size—the proportion of vari-
ance in the dependent variable that is accounted for by the manipulation of
the independent variables(s). In Chapter 10, we used eta-squared (
2
) as a
10017_11_ch11_p290-315.indd 306 2/1/08 1:32:13 PM
Complex Experimental Designs
■ ■
307
measure of effect size with ANOVAs. You may remember that
2
SS
Between
/
SS
Total
. When using a two-way ANOVA, we have three values of SS
Between
: one
for variable A, one for variable B, and one for the interaction term. Referring
to Table 11.9, we can obtain the SS scores needed for these calculations. For
factor A (word type),
2
is calculated as
2
18
____
202
.089
This means that factor A (word type) can account for 8.9% of the total vari-
ance in the number of words recalled. We can also calculate the effect sizes
for factor B (rehearsal type) and the interaction term using the same formula.
For factor B,
2
is 98/202 .485. In other words, factor B (rehearsal type)
can account for 48.5% of the total variance in the number of words recalled.
Clearly, rehearsal type is very important in determining the number of
words recalled; according to Cohen (1988), this effect size is meaningful.
Last, for the interaction effect,
2
50/202 .25. Thus, the interaction of
factors A and B can account for 25% of the variance in the number of words
recalled. This means that knowing the individual cell or condition (in other
words, the factor A by factor B condition) that the participants were in can
account for 25% of the variance in the dependent variable. This is also a
meaningful effect size.
Two-Way Randomized ANOVA IN REVIEW
CONCEPT DESCRIPTION
Null hypothesis (H
0
) The independent variable had no effect—the samples all represent the same population.
In a two-way ANOVA, there are three null hypotheses: one for factor A, one for factor B,
and one for the interaction of factors A and B.
Alternative hypothesis (H
a
) The independent variable had an effect—at least one of the samples represents a
different population than the others. In a two-way ANOVA, there are three alternative
hypotheses: one for factor A, one for factor B, and one for the interaction of factors
A and B.
F-ratio The ratio formed when the between-groups variance is divided by the within-groups
variance. In a two-way ANOVA, there are three F-ratios: one for factor A, one for
factor B, and one for the interaction of factors A and B.
Between-groups variance An estimate of the variance of the group means about the grand mean. In a two-way
ANOVA, there are three types of between-groups variance: that attributable to factor A,
that attributable to factor B, and that attributable to the interaction of factors A and B.
Within-groups variance An estimate of the variance within each condition in the experiment—also known as
error variance, or variance due to chance.
Eta-squared (
2
)
A measure of effect size—the variability in the dependent variable attributable to the
independent variable. In a two-way ANOVA, eta-squared is calculated for factor A, for
factor B, and for the interaction of factors A and B.
Tukey’s post hoc test
A test conducted to determine which conditions from a variable with more than two
conditions differ significantly from each other.
10017_11_ch11_p290-315.indd 307 2/1/08 1:32:16 PM
308
■ ■
CHAPTER 11
CRITICAL
THINKING
CHECK
11.2
1. Assuming that there were two significant main effects in a hypo-
thetical 2 4 design, would Tukey’s HSD need to be calculated for
these main effects? Why or why not?
2. A researcher is attempting to determine the effects of practice and
gender on a timed task. Participants in the experiment are given a
computerized search task. They search a computer screen of various
characters and attempt to find a particular character on each trial.
When they find the designated character, they press a button to stop
a timer. Their reaction time (in milliseconds) on each trial is recorded.
Participants practice for 1, 2, or 3 hours and are either female or male.
The ANOVA summary table appears next, along with the means for
each condition and the number of participants in each condition.
Two-Way ANOVA Summary Table
Source df SS MS F
Factor A (gender) 684,264
Factor B (practice) 989,504
A B 489,104
Error 2,967,768
Total 5,130,640
Condition Mean n
Female/1 hour 1,778.125 8
Female/2 hours 1,512.375 8
Female/3 hours 1,182.75 8
Male/1 hour 1,763.375 8
Male/2 hours 1,764.25 8
Male/3 hours 1,662 8
a. Identify the factorial notation for the design.
b. Complete the ANOVA summary table.
c. Determine significance levels for any main or interaction effect(s).
d. Explain any significant main or interaction effect(s).
e. Calculate eta-squared for any significant effects.
f. Draw a graph representing the data.
Two-Way Repeated Measures ANOVA and Mixed ANOVAs
When a complex within-participants (the same participants are used in all
conditions) or matched-participants (participants are matched across condi-
tions) design is used, and the data collected are interval-ratio in scale, then the
appropriate statistic is a two-way repeated measures ANOVA. This ANOVA
is similar to the two-way randomized ANOVA in that it indicates whether
there is a significant main effect of either independent variable in the study
and whether the interaction effect is significant. However, a correlated-
groups design requires slight modifications in the formulas applied. If you
find yourself in a situation where it is necessary to use a two-way repeated
10017_11_ch11_p290-315.indd 308 2/1/08 1:32:16 PM
Complex Experimental Designs
■ ■
309
measures ANOVA, you can find the calculations in a more advanced sta-
tistics text. In addition, some complex designs are mixed—one variable is
manipulated between participants and one within participants. Calculations
for such designs can be found in a more advanced statistics text.
Beyond the Two-Way ANOVA
In this and the previous chapter, we have discussed one- and two-way
ANOVAs. It is possible to add more factors (independent variables) to a
study and to analyze the data with an ANOVA. For example, if a study used
three independent variables, then a three-way ANOVA would be used. In
this situation, there would be three main effects, three two-way interactions,
and one three-way interaction to interpret. This means that there would be
seven F-ratios to calculate. Obviously, this complicates the interpretation of
the data considerably. Because three-way interactions are so difficult to inter-
pret, most researchers try to design studies that are not quite so complex.
All of the studies discussed so far have had only one dependent variable.
Besides adding independent variables, it is also possible to add dependent
variables to a study. With one dependent variable, we use univariate statistics
to analyze the data. Thus, all of the statistics discussed thus far in this text have
been univariate statistics. When we have more than one dependent variable,
we must use multivariate statistics to analyze the data. Many types of multivar-
iate statistics are available, including the multivariate t test and the multivari-
ate ANOVA, referred to as a MANOVA. These advanced statistics are beyond
the scope of this book. If you encounter them in the literature, however, you
can interpret them in a similar fashion to those statistics that we have covered.
In other words, the larger the t-score or F-ratio, the more likely it is the samples
represent different populations and that the test statistic is significant.
Finally, a meta-analysis is a statistical procedure (also beyond the scope of
this book) that combines, tests, and describes the results from many different
studies. Before this technique was developed, researchers had to rely on more
subjective reviews of the literature to summarize the general findings from
many studies. By allowing researchers to assess the results from a large number
of studies through one statistical procedure, a meta-analysis enables us to draw
more objective conclusions about the generalizability of research findings.
Summary
In this chapter, we described experimental designs that use more than
one independent variable. We discussed several advantages of using such
designs and introduced the concepts of factorial notation, main effects, and
interaction effects. After reading the section on main and interaction effects,
you should be able to graph data from a factorial design and interpret what
the graph means. We then discussed the statistical analysis of such designs,
using a two-way randomized ANOVA. We presented the various calculations
necessary to compute a two-way randomized ANOVA, along with the
assumptions of the test and a description of how to interpret the results.
10017_11_ch11_p290-315.indd 309 2/1/08 1:32:16 PM
310
■ ■
CHAPTER 11
factorial design
factorial notation
main effect
interaction effect
sum of squares factor A
sum of squares factor B
sum of squares interaction
sum of squares error
KEY TERMS
(Answers to odd-numbered exercises appear in
Appendix C.)
1. What is the advantage of manipulating more
than one independent variable in an experiment?
2. How many independent variables are in a 4 6
factorial design? How many conditions (cells)
are in this design?
3. In a study, a researcher manipulated the number
of hours that participants studied (either 4, 6,
or 8), the type of study technique they used
(shallow processing versus deep processing),
and whether participants studied individually
or in groups. What is the factorial notation for
this design?
4. What is the difference between a cell (condition)
mean and the means used to interpret a main
effect?
5. How many main effects and interaction effects
are possible in a 2 6 factorial design?
6. What is the difference between a complete facto-
rial design and an incomplete factorial design?
7. The cell means for two experiments appear
next. Determine whether there are any effects
of factor A, factor B, and A B for each experi-
ment. In addition, draw a graph representing the
data from each experiment.
Experiment 1 Experiment 2
A
1
A
2
B
1
35
B
2
58
A
1
A
2
B
1
12 4
B
2
412
8. Explain the difference between a two-way
ANOVA and a three-way ANOVA.
9. If you find two significant main effects in a 2 6
factorial design, should you compute Tukey’s
post hoc comparisons for both main effects?
10. Complete each of the following ANOVA summary
tables. In addition, answer the following questions
for each of the ANOVA summary tables:
a. What is the factorial notation?
b. How many conditions were in the study?
c. How many participants were in the study?
d. Identify significant main effects and
interaction effects.
Source df SS MS F
A 1 60
B 2 40
A B 2 90
Error 30
Total 35 390
Source df SS MS F
A 2 40
B 3 60
A B 6 150
Error 72
Total 83 400
Source df SS MS F
A 1 10
B 1 60
A B 1 20
Error 36
Total 39 150
11. In a study, a researcher measures the preference
of men and women for two brands of frozen
pizza (one low-fat and one regular) based on the
number of 1-inch pieces of each type of pizza
eaten when both types are available to the par-
ticipants. The following table shows the number
of pieces of each type of pizza eaten for each of
the 24 participants in the study.
Women Men
Brand 1 3 9
(low fat) 4 7
2 6
2 8
5 9
3 7
CHAPTER EXERCISES
10017_11_ch11_p290-315.indd 310 2/1/08 1:32:17 PM
Complex Experimental Designs
■ ■
311
Women Men
Brand 2 8 4
(regular) 9 2
7 5
10 6
9 2
10 5
Source df SS MS F
Gender 0.167
Pizza brand 6.00
Gender Pizza 130.67
Error 35.00
Total 171.83
a. Complete the ANOVA summary table. (If your
instructor wants you to calculate the sums of
squares, use the preceding data to do so.)
b. Are the values of F
obt
significant at .05?
c. What conclusions can be drawn from the
F-ratios?
d. What is the effect size, and what does this
mean?
e. Graph the means.
12. A researcher is attempting to determine the
effects of practice and gender on a timed task.
Participants in an experiment are given a com-
puterized search task. They search a computer
screen of various characters and attempt to find a
particular character on each trial. When they find
the designated character, they press a button to
stop a timer. Their reaction time (in seconds) on
each trial is recorded. Participants practice for 2, 4,
or 6 hours and are either female or male. The reac-
tion time data for the 30 participants appear here.
Women Men
2 Hours 12 11
13 12
12 13
11 12
11 11
4 Hours 10 8
10 8
10 10
8 10
7 9
6 Hours 7 5
5 6
7 8
6 6
7 8
Source df SS MS F
Gender 0.027
Practice 140.60
Gender Practice 0.073
Error 28.00
Total 168.70
a. Complete the ANOVA summary table. (If your
instructor wants you to calculate the sums of
squares, use the preceding data to do so.)
b. Are the values of F
obt
significant at .05?
c. What conclusions can be drawn from the
F-ratios?
d. What is the effect size, and what does this
mean?
e. Graph the means.
11.1
1. This would be a 3 5 design. There are three
types of crust and five types of toppings, so
there are 15 conditions in this study.
2. A 4 6 factorial design has two independent
variables. Thus, there is the possibility of two
main effects (one for each independent vari-
able) and one interaction effect (the interaction
between the two independent variables).
3. There appears to be a main effect of word type,
with concrete words recalled better than
abstract words. There also appears to be a main
effect of rehearsal type, with those who used
imagery rehearsal remembering more words
than those who used rote rehearsal. In addition,
there appears to be an interaction effect. When
imagery rehearsal is used, word type makes
no difference; recall is very high for both types
CRITICAL THINKING CHECK ANSWERS
10017_11_ch11_p290-315.indd 311 2/1/08 1:32:17 PM