Two-way analysis of variance

In statistics, the two-way analysis of variance (ANOVA) is an extension of the one-way ANOVA that examines the influence of two different categorical independent variables on one continuous dependent variable. The two-way ANOVA not only aims at assessing the main effect of each independent variable but also if there is any interaction between them.

History

In 1925, Ronald Fisher mentions the two-way ANOVA in his celebrated book from 1925, Statistical Methods for Research Workers (chapters 7 and 8). In 1934, Frank Yates published procedures for the unbalanced case.^[1] Since then, an extensive literature has been produced, reviewed in 1993 by Yasunori Fujikoshi.^[2] In 2005, Andrew Gelman proposed a different approach of ANOVA, viewed as a multilevel model.^[3]

Data set

Let us imagine a data set for which a dependent variable may be influenced by two factors which are potential sources of variation. The first factor has $I$ levels ( $i \in \{1,\ldots,I\}$ ) and the second has $J$ levels ( $j \in \{1,\ldots,J\}$ ). Each combination $(i,j)$ defines a treatment, for a total of $I \times J$ treatments. We represent the number of replicates for treatment $(i,j)$ by $n_{ij}$ , and let $k$ be the index of the replicate in this treatment ( $k \in \{1,\ldots,n_{ij}\}$ ).

From these data, we can build a contingency table, where $n_{i+} = \sum_{j=1}^J n_{ij}$ and $n_{+j} = \sum_{i=1}^I n_{ij}$ , and the total number of replicates is equal to $n = \sum_{i,j} n_{ij} = \sum_i n_{i+} = \sum_j n_{+j}$ .

The experimental design is balanced if each treatment has the same number of replicates, $K$ . In such a case, the design is also said to be orthogonal, allowing to fully distinguish the effects of both factors. We hence can write $\forall i,j \; n_{ij} = K$ , and $\forall i,j \; n_{ij} = \frac{n_{i+} \cdot n_{+j}}{n}$ .

Model

Upon observing variation among all $n$ data points, for instance via a histogram, "probability may be used to describe such variation".^[4] Let us hence denote by $Y_{ijk}$ the random variable which observed value $y_{ijk}$ is the $k$ -th measure for treatment $(i,j)$ . The two-way ANOVA models all these variables as varying independently and normally around a mean, $\mu_{ij}$ , with a constant variance, $\sigma^2$ (homoscedasticity):

$Y_{ijk} \, | \, \mu_{ij}, \sigma^2 \; \overset{i.i.d.}{\sim} \; \mathcal{N}(\mu_{ij}, \sigma^2)$ .

Specifically, the mean of the response variable is modeled as a linear combination of the explanatory variables:

$\mu_{ij} = \mu + \alpha_i + \beta_j + \gamma_{ij}$ ,

where $\mu$ is the grand mean, $\alpha_i$ is the additive main effect of level $i$ from the first factor (i-th row in the contigency table), $\beta_j$ is the additive main effect of level $j$ from the second factor (j-th column in the contigency table) and $\gamma_{ij}$ is the non-additive interaction effect of treatment $(i,j)$ from both factors (cell at row i and column j in the contigency table).

An other, equivalent way of describing the two-way ANOVA is by mentioning that, besides the variation explained by the factors, there remains some statistical noise. This amount of unexplained variation is handled via the introduction of one random variable per data point, $\epsilon_{ijk}$ , called error. These $n$ random variables are seen as deviations from the means, and are assumed to be independent and normally distributed:

$Y_{ijk} = \mu_{ij} + \epsilon_{ijk} \text{ with } \epsilon_{ijk} \overset{i.i.d.}{\sim} \mathcal{N}(0, \sigma^2)$ .

Assumptions

Following Gelman and Hill, the assumptions of the ANOVA, and more generally the general linear model, are, in decreasing order of importance:^[5]

the data points are relevant with respect to the scientific question under investigation;
the mean of the response variable is influenced additively (if not interaction term) and linearly by the factors;
the errors are independent;
the errors have the same variance;
the errors are normally distributed.

Parameter estimation

To ensure identifiability of parameters, we can add the following "sum-to-zero" constraints:

$\sum_i \alpha_i = \sum_j \beta_j = \sum_i \sum_j \gamma_{ij} = 0$

Hypothesis testing

In the classical approach, testing null hypotheses (that the factors have no effect) is achieved via their significance which requires calculating sums of squares.

Testing if the interaction term is significant can be difficult because of the potentially-large number of degrees of freedom.^[6]

References

George Casella (18 April 2008). Statistical design. Springer. ISBN 978-0-387-75965-4.

Notes

↑ Yates, Frank (March 1934). "The analysis of multiple classifications with unequal numbers in the different classes". Journal of the American Statistical Association (American Statistical Association) 29 (185): 51–66. doi:10.1080/01621459.1934.10502686. Retrieved 19 June 2014.
↑ Fujikoshi, Yasunori (1993). "Two-way ANOVA models with unbalanced data". Discrete Mathematics (Elsevier) 116 (1): 315–334. doi:10.1016/0012-365X(93)90410-U. Retrieved 19 June 2014.
↑ Gelman, Andrew (February 2005). "Analysis of variance? why it is more important than ever". The Annals of Statistics 33 (1): 1–53. doi:10.1214/009053604000001048. Retrieved 19 June 2014.
↑ Kass, Robert E (1 February 2011). "Statistical inference: The big picture". Statistical Science (Institute of Mathematical Statistics) 26 (1): 1–9. doi:10.1214/10-sts337.
↑ Gelman, Andrew; Hill, Jennifer (18 December 2006). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press. pp. 45–46. ISBN 0521867061.
↑ Yi-An Ko; et al. (September 2013). "Novel Likelihood Ratio Tests for Screening Gene-Gene and Gene-Environment Interactions with Unbalanced Repeated-Measures Data". Genetic epidemiology 37 (6): 581–591. doi:10.1002/gepi.21744. Retrieved 19 June 2014.

This article is issued from Wikipedia - version of the Saturday, November 28, 2015. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.