Factorial Analysis of Variance

Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) In the context of ANOVA, an independent variable (or a quasiindependent variable) is called a factor, and research studies with multiple factors, in which every level of one factor is paired with every level of the other factors, are called factorial designs. Example: the Eysenck (1974) memory study, in which type-ofprocessing was one factor and age was another factor. Younger Older Counting Rhyming Adjective Imagery Intentional

Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) A design with m factors (with m>1) is called an m-way factorial design The Eysenck study described in the previous slide has two factors and is therefore a two-way factorial design We can design factorial ANOVAs with an arbitrary number of factors. For example, we could add gender as another factor in the Eysenck memory study However, for simplicity, we will deal only with two-way factorial designs in this course. We will also assume that each of our factorial samples (cells) contains the same number of scores n

Overview of the Factorial ANOVA Why might we want to use factorial designs? Factorial ANOVA (Two-Way) With a one-way ANOVA, we can examine the effect of different levels of a single factor: E.g., How does age affect word recall? Or, How does type of processing affect word recall? We need two different experiments to determine the effects of these two factors on memory if we use a one-way design. Moreover, using a factorial design allows us to detect interactions between factors

Overview of the Factorial ANOVA Example: Factorial ANOVA (Two-Way) We have developed a new drug for treating migraines, but suspect that it affects women differently than men The scores represent the number of weekly migraines reported following administration of the drug Low High Total Women M LW M HW M W Men M LM M HM M M Total M L M H

Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) Notice that the study involves two dosage conditions and two gender conditions, creating a two-by-two matrix with a total of 4 different treatment conditions. Each treatment condition is represented by a cell in the matrix. For an independent-measures research study (which is the only kind of factorial design that we will consider), a separate sample would be used for each of the four conditions.

Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) As with one-way ANOVAs the goal for the two-factor ANOVA is to determine whether the mean differences that are observed for the sample data are significant differences and not simply the result of sampling error. For the example we are considering, the goal is to determine whether different dosages of a drug and differences in gender produce significant differences in the number of migraines reported.

Factorial ANOVA Main Effects Factorial ANOVA (Two-Way) To evaluate the sample mean differences, a two-factor ANOVA conducts three separate and independent hypothesis tests. The three tests evaluate: 1. The Main Effect for Factor A: The mean differences between the levels of factor A are obtained by computing the overall mean for each row in the matrix. In this example, the main effect of factor A would compare the overall mean number of migraines reported by women versus the overall mean number of migraines reported by men. 2. The Main Effect for Factor B: The mean differences between the levels of factor B are obtained by computing the overall mean for each column in the matrix. In this example, the ANOVA would compare the overall mean number of migraines reported for the low and high dosage conditions.

Example: Main Effects Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Main effect A Total M L =16 M H =10.5 Main effect B Main effect A: main effect of gender. Women report more migraines than men Main effect B: main effect of drug. Administering a higher of the drug leads to fewer reported migraines

Interactions The A x B Interaction: Often two factors will "interact" so that specific combinations of the two factors produce results (mean differences) that are not explained by the overall effects of either factor. For example, a particular drug may have different efficacies for men vs. women. Different s of the drug might produce very small changes in men, but dramatic, or even opposite, effects in women. This dependence on the effect of one factor (drug dosage) on another (sex or gender) is called an interaction.

Example: Interaction Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Total M L =16 M H =10.5 Main effect B In this case, the drug seems to be much more effective for women than for men. We would say that there is an interaction between the effect of gender and the effect of drug dosage

Example: No Interaction Low High Total Women M LW =20 M HW =14.5 M W =17.25 Men M LM =12 M HM =6.5 M M =9.25 Total M L =16 M H =10.5 Main effect B In this case, the main effect of the drug dosage is the same as in the previous case, but there is no longer a difference between the effect of drug dosage on women versus men. There is no interaction.

Interactions This is the primary advantage of combining two factors (independent variables) in a single research study: it allows you to examine how the two factors interact with each other. That is, the results will not only show the overall main effects of each factor, but also how unique combinations of the two variables may produce unique results.

More Examples

Simple Effects To interpret significant interactions, researchers often conduct a fourth type of hypothesis test for simple effects Simple effects (or simple main effects) involve testing the effect of one factor at a particular value of the second factor In our example, testing the effect of the drug for women only or for men only are examples of simple effects, as are testing the effect of gender in low only or high- only conditions Testing for simple effects essentially consists of running a separate one way ANOVA across all levels of one factor at a fixed level of the second factor For example, in the Eysenck memory study, we could run a one-way ANOVA to determine the effect of processing condition on word recall for young subjects only

Example: Interaction Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Total M L =16 M H =10.5 Main effect B

Structure of the One-Way ANOVA (Independent- Measures) Total Variance Between Groups Variance Within Groups Variance

Structure of the Repeated-Measures ANOVA Total Variance Between Groups Variance Within Groups Variance Between Subjects Variance Error (residual) Variance

Structure of the Two-Way (Factorial) ANOVA Total Variance Between Groups Variance Within Groups Variance Factor A Variance Factor B Variance Interaction Variance

The Two-Way ANOVA Each of the three hypothesis tests in a two-factor ANOVA will have its own F-ratio and each F-ratio has the same basic structure: F MS MS between within Each MS value equals SS/df, and the individual SS and df values are computed in a two-stage analysis. The first stage of the analysis is identical to the single-factor (one-way) ANOVA and separates the total variability (SS and df) into two basic components: between treatments and within treatments.

The Two-Way ANOVA The second stage of the analysis separates the betweentreatments variability into the three components that will form the numerators for the three F-ratios: 1. Variance due to factor A 2. Variance due to factor B 3. Variance due to the interaction. Each of the three variances (MS) measures the differences for a specific set of sample means. The main effect for factor A, for example, will measure the mean differences between rows of the data matrix.

The Two-Way ANOVA: Possible Outcomes 1. All 3 hypothesis tests are not significant 2. Only main effect of Factor A is significant 3. Only main effect of Factor B is significant 4. Both main effects (for A and B) are significant 5. Only the interaction (A B) is significant 6. Main effect of Factor A and A B interaction are significant 7. Main effect of Factor B and A B interaction are significant 8. All 3 hypothesis tests are significant

Factorial ANOVA (Two-Way)

The Two-Way ANOVA: Steps 1. State Hypotheses Factorial ANOVA (Two-Way) whatever 2. Compute F-ratio statistic: F dfwhatever, dferror for each main effect and their interaction MSerror For data in which I give you cell means and SS s, you will have to compute: marginal means SS total, SS between, SS within, SS factor A, SS factor B, & SS A B df total, df between, df within, df factor A, df factor B, & df A B MS 3. Use F-ratio distribution table to find critical F-value(s) representing rejection region(s) 4. For each F-test, make a decision: does the F-statistic for your test fall into the rejection region?

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender Within (error) Total 1. Compute degrees of freedom df df df df df total between N 1 19 # cells 1 3 within total between gender df df # levels 1 1 # gender levels 16 11 df df df df 1 gender between gender

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 Gender 1 Dose 1 Dose x gender 1 Within (error) 16 Total 19 2. Compute SS within (SS error ) SSwithin SS 20 18 32 24 94

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 Gender 1 Dose 1 Dose x gender 1 Within (error) 16 94 Total 19 3. Compute SS between (SS cells ) SS n M M between cell 2 5 20 13.25 10 13.25 12 13.25 1113.25 5 45.563 10.5631.563 5.063 5 62. 75 313.75 T 2 2 2 2

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 Dose 1 Dose x gender 1 Within (error) 16 94 Total 19 407.75 4. Compute SS gender (SS Factor A ) SS n M M gender row gender T 10 15 13.25 11.5 13.25 10 3.0625 3.0625 10 6. 125 61.25 2 2 2

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 Dose 1 Dose x gender 1 Within (error) 16 94 Total 19 407.75 5. Compute SS (SS Factor B ) SS n M M col T 10 16 13.25 10.5 13.25 10 7.5625 7.5625 10 15.125 2 2 2 151.25

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 Dose 1 151.25 Dose x gender 1 Within (error) 16 94 Total 19 407.75 6. Compute SS gender (SS A B ) SSd ose SS SS SS gender between gender 313.75 151.25 61.25 313.75 21. 25101. 25

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 Dose 1 151.25 Dose x gender 1 101.25 Within (error) 16 94 Total 19 407.75 7. Compute the MS values needed to compute the 3 required F ratios: MS gender SSgender 61.25 61.25 df 1 gender MS gender SS gender 101.25 101.25 df 1 gender MS SS 151.25 151.25 df 1 MS error SSerror 94 5.875 df 16 error

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 61.25 Dose 1 151.25 151.25 Dose x gender 1 101.25 101.25 Within (error) 16 94 5.875 Total 19 407.75 8. Compute F ratios for each of the two main effects (gender and ) and their interaction: F whatever df MS whatever whatever, dfe rror MSerror F F F gender 1,16 1,16 MSgender 61.25 10.43 MS 5.875 error MS 151.25 25.74 MS 5.875 error MS gender 101.25 gender 1,1 6 17. 23 MS 5. 875 error

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 61.25 10.43 Dose 1 151.25 151.25 25.74 Dose x gender 1 101.25 101.25 17.23 Within (error) 16 94 5.875 Total 19 407.75 9. Finally, look up F crit for each of your obtained F values In this case, we happen to be lucky that they all have the same degrees of freedom (1,16), so we only have to look up one F crit

F table for α=0.05 reject H 0 df error df numerator 1 2 3 4 5 6 7 8 9 10 1 161.45 199.50 215.71 224.58 230.16 233.99 236.77 238.88 240.54 241.88 2 18.51 19.00 19.16 19.25 19.30 19.33 19.35 19.37 19.38 19.40 3 10.13 9.55 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.79 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.96 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.74 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 4.10 4.06 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.64 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 3.18 3.14 10 4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.98 11 4.84 3.98 3.59 3.36 3.20 3.09 3.01 2.95 2.90 2.85 12 4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85 2.80 2.75 13 4.67 3.81 3.41 3.18 3.03 2.92 2.83 2.77 2.71 2.67 14 4.60 3.74 3.34 3.11 2.96 2.85 2.76 2.70 2.65 2.60 15 4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.54 16 4.49 3.63 3.24 3.01 2.85 2.74 2.66 2.59 2.54 2.49 17 4.45 3.59 3.20 2.96 2.81 2.70 2.61 2.55 2.49 2.45 18 4.41 3.55 3.16 2.93 2.77 2.66 2.58 2.51 2.46 2.41 19 4.38 3.52 3.13 2.90 2.74 2.63 2.54 2.48 2.42 2.38 20 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 2.39 2.35 22 4.30 3.44 3.05 2.82 2.66 2.55 2.46 2.40 2.34 2.30 24 4.26 3.40 3.01 2.78 2.62 2.51 2.42 2.36 2.30 2.25 26 4.23 3.37 2.98 2.74 2.59 2.47 2.39 2.32 2.27 2.22 28 4.20 3.34 2.95 2.71 2.56 2.45 2.36 2.29 2.24 2.19 30 4.17 3.32 2.92 2.69 2.53 2.42 2.33 2.27 2.21 2.16 40 4.08 3.23 2.84 2.61 2.45 2.34 2.25 2.18 2.12 2.08 50 4.03 3.18 2.79 2.56 2.40 2.29 2.20 2.13 2.07 2.03 60 4.00 3.15 2.76 2.53 2.37 2.25 2.17 2.10 2.04 1.99 120 3.92 3.07 2.68 2.45 2.29 2.18 2.09 2.02 1.96 1.91 200 3.89 3.04 2.65 2.42 2.26 2.14 2.06 1.98 1.93 1.88 500 3.86 3.01 2.62 2.39 2.23 2.12 2.03 1.96 1.90 1.85 1000 3.85 3.00 2.61 2.38 2.22 2.11 2.02 1.95 1.89 1.84

F table for α=0.05 reject H 0 df error df between 1 2 3 4 5 6 7 8 9 10 1 161.45 199.50 215.71 224.58 230.16 233.99 236.77 238.88 240.54 241.88 2 18.51 19.00 19.16 19.25 19.30 19.33 19.35 19.37 19.38 19.40 3 10.13 9.55 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.79 4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.96 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.74 6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 4.10 4.06 7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.64 8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 3.18 3.14 10 4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.98 11 4.84 3.98 3.59 3.36 3.20 3.09 3.01 2.95 2.90 2.85 12 4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85 2.80 2.75 13 4.67 3.81 3.41 3.18 3.03 2.92 2.83 2.77 2.71 2.67 14 4.60 3.74 3.34 3.11 2.96 2.85 2.76 2.70 2.65 2.60 15 4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.54 16 4.49 3.63 3.24 3.01 2.85 2.74 2.66 2.59 2.54 2.49 17 4.45 3.59 3.20 2.96 2.81 2.70 2.61 2.55 2.49 2.45 18 4.41 3.55 3.16 2.93 2.77 2.66 2.58 2.51 2.46 2.41 19 4.38 3.52 3.13 2.90 2.74 2.63 2.54 2.48 2.42 2.38 20 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 2.39 2.35 22 4.30 3.44 3.05 2.82 2.66 2.55 2.46 2.40 2.34 2.30 24 4.26 3.40 3.01 2.78 2.62 2.51 2.42 2.36 2.30 2.25 26 4.23 3.37 2.98 2.74 2.59 2.47 2.39 2.32 2.27 2.22 28 4.20 3.34 2.95 2.71 2.56 2.45 2.36 2.29 2.24 2.19 30 4.17 3.32 2.92 2.69 2.53 2.42 2.33 2.27 2.21 2.16 40 4.08 3.23 2.84 2.61 2.45 2.34 2.25 2.18 2.12 2.08 50 4.03 3.18 2.79 2.56 2.40 2.29 2.20 2.13 2.07 2.03 60 4.00 3.15 2.76 2.53 2.37 2.25 2.17 2.10 2.04 1.99 120 3.92 3.07 2.68 2.45 2.29 2.18 2.09 2.02 1.96 1.91 200 3.89 3.04 2.65 2.42 2.26 2.14 2.06 1.98 1.93 1.88 500 3.86 3.01 2.62 2.39 2.23 2.12 2.03 1.96 1.90 1.85 1000 3.85 3.00 2.61 2.38 2.22 2.11 2.02 1.95 1.89 1.84

Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H =10.5 13.25, N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 313.75 Gender 1 61.25 61.25 10.43* <0.05 Dose 1 151.25 151.25 25.74* <0.05 Dose x gender 1 101.25 101.25 17.23* <0.05 Within (error) 16 94 5.875 Total 19 407.75 Both main effects are significant, as is their interaction. This suggests that: 1. The number of reported migraines differs between men and women Women report more migraines than men 2. The number of reported migraines is affected by the drug Fewer migraines are reported in the high condition 3. The effect of the drug differs across genders Women are more sensitive to the drug than are men

M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN =15.65 11.61, SS 2667. 8, N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups 9 1945.5 Age 1 240 240 Study 4 1514.8 378.7 Age x study 4 190.7 47.67 Within (error) 90 722.3 8.03 Total 99 2667.8 8. Compute F ratios for each of the two main effects (age and study condition) and their interaction: F whatever df MS whatever whatever, dfe rror MSerror F F F age study MS 240 8.03 age 1,90 29.89 MS error MSagestudy 190.7 4,90 23.75 MS 8.03 age study 4,90 MSstudy 378.7 47.16 MS 8.03 error error

M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C = 6.75 M R = 7.25 M A = 12.9 M IM = 15.5 M IN = 15.65 11.61, SS 2667. 8, N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups 9 1945.5 Age 1 240 240 29.89* <0.05 Study 4 1514.8 378.7 47.16* <0.05 Age x study 4 190.7 47.67 23.75* <0.05 Within (error) 90 722.3 8.03 Total 99 2667.8 Both main effects are significant, as is their interaction. This suggests that: 1. The number of remembered words differs between old and young subjects Younger subjects remember more words 2. The number of remembered words differs across study conditions More words are remembered in the deeper processing conditions, though we would need post-hoc tests to see which of these differences are significant 3. The effect of the study condition differs as a function of age Younger subjects get more of an advantage from deep processing

Effect Size for the Two-Way ANOVA Factorial ANOVA (Two-Way) Effect sizes for two-way ANOVAs are usually indicated using the R 2 -family measure eta-squared (η 2 ) R 2 -family measures indicate the effect size in terms of proportion of variance accounted for by the treatment effect(s) 2 variability explained by treatment effect R total variability We compute this measure for each of our effects. E.g.: SS 2 age 240 age 0. 09, SS 2667. 8 total SS 2 study 1514.8 study 0.57, SS 2667.8 total SSagestudy 190.7 0.07 SS 2667. 8 2 age study total