2.1 The Pseudo F Statistic

Let Y be a matrix of N rows (sampling units) by p columns (variables). Let D = {d_ij}, i = 1,…, N; j = 1,…, N consist of the distances or dissimilarities between every pair (i, j) of sampling units. The first major milestone in the development of PERMANOVA was the basic achievement of direct partitioning of dissimilarity-based multivariate spaces in response to multiway ANOVA designs^3-9. Most similarities S = {s_ij} can also be reexpressed¹⁰ as dissimilarities d_ij = 1.0 − s_ij. A direct partitioning of distance matrices having Euclidean metric properties^10-12 was extended to semimetric dissimilarities^{6, 9}, having only the properties¹³ of symmetry (i.e., d_ij = d_ji), and that d_ij ≥ 0 and d_ii = 0 ∀ i, j.

Consider matrix $\mathbf{A}=\left\{a_{ij}\right\}=\left\{-\frac{1}{2}{d}_{ij}^2\right\}$. Centering the elements of this matrix by row-means, ${\bar{a}}_{i·}$, column-means, ${\bar{a}}_{·j}$, and the overall mean, ${\bar{a}}_{··}$, gives Gower's matrix G = {g_ij} = {$a_{\mathrm{ij}}-{\bar{a}}_{i·}-{\bar{a}}_{·j}+{\bar{a}}_{··}$}. Let X be an (N × g) design matrix whose columns are indicators coding for groups (levels) of an ANOVA factor in a one-way design (e.g., an intercept and orthogonal contrasts⁷). We can then obtain a projection (hat) matrix H = X[X′X]^{− 1}X′, derived from the normal equations used in linear regression^{14, 15}.

When Euclidean distances are used, then the PERMANOVA sums-of-squares are each equal to the sum of the classical univariate sums of squares across the original variables^{7, 16}; that is, $S{S}_{\mathrm{A}}={\sum }_{k=1}^{p}S{S}_{\mathrm{A}}^{\left[k\right]}$, where $S{S}_{\mathrm{A}}^{\left[k\right]}$ is the univariate among-group sum-of-squares for variable k = 1,…, p (and similarly for each of SS_R and SS_T). Hence, pseudo F on Euclidean distances is the same as the F statistic used in classical redundancy analysis (RDA)^{17, 18}. Distance-based redundancy analysis (dbRDA^{7, 8}), equivalent to PERMANOVA, is an RDA on the orthogonal principal coordinates (PCOs) of matrix G based on a chosen (potentially non-Euclidean) dissimilarity measure. However, direct partitioning of matrix G obviates the need for any eigenvector decomposition into PCOs or any corrections for negative eigenvalues that might arise for semimetric (or non-Euclidean-embeddable) dissimilarities⁸. Furthermore, PERMANOVA on one response variable using Euclidean distance yields the classical univariate F statistic¹⁹. So, PERMANOVA can also be used to do univariate ANOVA, but where p values are obtained by permutation²⁰, thus avoiding the assumption of normality.

2.2 Inference

The second major milestone, achieved more-or-less simultaneously, was the development of appropriate permutation algorithms^{21, 22} for achieving rigorous tests of individual terms in complex ANOVA designs, specifically by conditioning on other terms in the model^{23, 24} and using expectations of mean squares to identify correct permutable units^{25, 26}. In some cases (e.g., tests of interactions), only asymptotically exact p-values are readily obtainable, for example, via permutation of residuals under a reduced model²⁷, although synchronized permutations²⁸ might be possible.

For cases where there are too few possible permutations to achieve a precise p-value for inferences at a suitably small level of significance, approximate p-values can be obtained using Monte Carlo random draws from the asymptotic permutation distribution²⁹. Specifically, each of the numerator and denominator of pseudo F is asymptotically distributed as a linear form in chi‐square random variables, with coefficients being the eigenvalues from a PCO of matrix G²⁹.

3.1 Exchangeability and the Linear Model

PERMANOVA makes no explicit assumptions regarding either the distributions of original variables in Y or the distributions of dissimilarities in D. For a given test, PERMANOVA assumes only exchangeability^{30, 31} of permutable units under a true null hypothesis. PERMANOVA is only “nonparametric”⁹ for the one-way case. Inferences remain distribution-free, but PERMANOVA applies a linear model to the dissimilarity space; interactions are defined by reference to additive main effects. Indeed, a key motivation for the development of PERMANOVA was to perform tests for interaction^{7, 8}; PERMANOVA is a classical partitioning in the Euclidean space defined by the full set of PCO axes^{7, 8}. Consequently, the dissimilarity measure will have an important bearing on results, much more so than for rank-based tests, such as ANOSIM². Hence, in practice, values in matrix D should be neither over-ridden with repeated values at either an upper or lower bound nor dominated by undefined or erratic values, as can occur for sparse data^{32, 33}. Judicious choice of dissimilarity measure^{32, 34} or pooling of small-scale replicates³³ is often well advised.

3.2 Homogeneity of Multivariate Dispersions

Both ANOSIM and the Mantel test are extremely sensitive to differences in dispersions among groups; however, PERMANOVA (like ANOVA) is very robust to heterogeneity for balanced designs but not unbalanced designs³⁵. Furthermore, PERMANOVA, unlike ANOSIM or traditional MANOVA, is not sensitive to differences in correlation structure (shape) among groups^35-37.

A test for homogeneity of multivariate dispersions (PERMDISP) in the space of the chosen dissimilarity measure can be done, either to accompany PERMANOVA or in its own right³⁸. This test compares within-group spread among groups using the average value of the distances from individual observations to their own group centroid^38-41. The specific directions of distances are not taken into account, so PERMDISP, like PERMANOVA, does not identify differences in the shapes of data clouds among groups, only their relative spread.

5.1 Distances among Centroids

The number of individual sampling units in complex ANOVA designs may become quite large. Thus, visualizing patterns across factor levels can be difficult; stress (see shepard diagram) in metric or nMDS plots of replicates often exceeds the rule of thumb for interpretability (i.e., stress >0.2). Typically, interest lies in examining the relative positions of group centroids in the space of the dissimilarity measure; this is precisely as in univariate analysis, where plots of means are generally more useful than are plots of raw data. This is achieved by ordination of distances among centroids.

If D contains Euclidean distances, then the distances among centroids are equivalent to Euclidean distances among the arithmetic averages calculated separately for each variable. This equivalence does not hold, however, for non-Euclidean dissimilarities. Distances among centroids based on some other chosen dissimilarity measure are calculated as follows: (i) obtain the full set of PCO axes from matrix G, with each axis having been standardized by the absolute value of its respective eigenvalue; (ii) calculate arithmetic averages for each group separately along each PCO axis; (iii) for every pair of centroids (ℓ, ℓ′), (ℓ = 1,…, g) and (ℓ′ = 1,…, g), calculate Euclidean distances separately in each of the two sets: one based on PCO axes corresponding to non-negative eigenvalues $\left(d_{\ell {\ell }^{\prime }}^{+}\right)$ and one based on those corresponding to negative eigenvalues $\left(d_{\ell {\ell }^{\prime }}^{-}\right)$, if any; and (iv) the (g × g) matrix of distances among centroids in the space of the dissimilarity measure is then ${\mathbf{D}}^{\left[C\right]}=\left\{d_{\ell {\ell }^{\prime }}^{\left[C\right]}\right\}$, where $d_{\ell {\ell }^{\prime }}^{\left[C\right]}=\sqrt{{\left(d_{\ell {\ell }^{\prime }}^{+}\right)}^2-{\left(d_{\ell {\ell }^{\prime }}^{-}\right)}^2}$.

Distances among centroids can also be calculated directly from matrix G. For example, consider a one-way ANOVA model with n_ℓ sampling units in group ℓ and $N={\sum }_{\ell =1}^g{n}_{\ell }$. Let (i ∈ ℓ) denote an indicator for the subset of the i = 1,…, N observations that occur in group ℓ. An “averaged” (g × g) Gower matrix can be obtained as $\bar{\mathbf{G}}=\left\{{\bar{g}}_{\ell {\ell }^{\prime }}\right\}$, where ${\bar{g}}_{\ell {\ell }^{\prime }}=\frac{1}{n_{\ell }{n}_{{\ell }^{\prime }}}{\sum }_{\left(i\in \ell \right)}{\sum }_{\left(j\in {\ell }^{\prime }\right)}{g}_{ij}$. A suitable back-transformation yields the desired distances among every pair of centroids: $d_{\ell {\ell }^{\prime }}^{\left[C\right]}=\sqrt{{\bar{g}}_{\ell \ell }-2{\bar{g}}_{\ell {\ell }^{\prime }}+{\bar{g}}_{{\ell }^{\prime }{\ell }^{\prime }}}$, obviating the need for PCO axes.

5.2 “Main Effects” Plots, “Interaction” Plots, and “Residual” Plots

In multifactor ANOVA, interest lies in visualizing differences among factor-level centroids as well as the amount of variation attributable to different factors. One may calculate distances among centroids corresponding to the main-effect levels for each factor in turn and place all of these in a single ordination. Such a plot of “main effects,” particularly using metric MDS to preserve the original dissimilarity scale, shows the relative importance of factors, generally reflecting the relative sizes of estimated components of variation from the PERMANOVA partitioning.

Another natural ordination plot of interest is the multivariate direct analogue to an “interaction” plot in the space of the dissimilarity measure. One can do metric or nonmetric MDS (or PCO) on distances among centroids that correspond to individual cells defined by combinations of factor levels (averaging replicates within cells). Such a plot will not only show effects of individual factors but also (potentially) provide insights into the nature of any interactions detected between the factors.

A factor of primary interest may be statistically significant in a PERMANOVA partitioning of the full model, but its effects may be totally obscured by some other dominant factor(s) or covariate(s) in an ordination. Although plots of distances among centroids may shed some light on minor effects, a dissimilarity-based multivariate analogue to a residual plot, from which the variation due to dominant (but perhaps nuisance) factors or covariates has been removed, is desirable.

An (N × N) “residualized” Gower matrix ${\mathbf{G}}^{\left[R\right]}=\left\{g_{ij}^{\left[R\right]}\right\}$ is attainable directly as G^[R] = (I − H)G(I − H), where H is the usual “hat” matrix calculated on some model matrix X containing all terms (factors, covariates, etc.) the effects of which one wishes to remove. A “residualized” distance matrix is then obtained as ${\mathbf{D}}^{\left[R\right]}=\left\{d_{ij}^{\left[R\right]}\right\}$, where $d_{ij}^{\left[R\right]}=\sqrt{g_{ii}^{\left[R\right]}-2g_{ij}^{\left[R\right]}+g_{jj}^{\left[R\right]}}$. Any of the usual ordination methods (PCO, mMDS, or nMDS) may then be constructed in the usual way from this matrix of residualized distances.

6.1 Four-Factor Mixed Model: Okura Estuary

Consider a study of potential effects of sedimentation on intertidal soft-sediment fauna in the Okura estuary, near Auckland, New Zealand⁵⁸. Sites were classified a priori from earlier hydrological studies as having a high, medium, or low probability of sediment deposition. Thus, n = 6 sediment cores (13 cm diameter × 15 cm deep) were sampled randomly from each of the 15 sites, 5 within each of these 3 depositional types of environments. Sampling was repeated a total of six times in 2001–2002: once after a relatively dry period and once 7–10 days after a heavy rainfall event in each of the three seasons: winter, spring, and summer. Abundances of p = 73 taxa were recorded from a total of N = 540 cores in a four-factor design: Season (fixed with three levels: winter, spring, or summer), Rainfall (fixed with two levels: rain or dry), Deposition (fixed with three levels: high, medium, or low), and Site (random with 15 levels, nested within Deposition).

Ordination (nMDS) done directly on the full dissimilarity matrix (Bray–Curtis on fourth-root transformed data) has high stress (∼0.20), and although one may discern a pattern of differences between assemblages from different depositional environments, there is a lot of overlap (Figure 2), and any labeling scheme attempting to show season or rainfall effects is simply uninterpretable, due to high residual variation.

PERMANOVA partitioning shows that spatial effects are the strongest, with Deposition, Site, and the Residual contributing the largest components of variation to the overall model (Table 1). Small-scale spatiotemporal variation, identifiable from statistically significant high-order interactions of temporal factors with Site (Table 1, p < 0.001), was also apparent. Although there was no evidence for a three-way interaction (Season × Rainfall × Deposition; p > 0.48), effects of depositional environments varied across seasons and in different weather conditions (Season × Deposition and Season × Rainfall; p < 0.01).

A main-effects plot clearly shows that depositional effects are the strongest, especially contrasting high versus either medium or low depositional areas along mMDS axis 1 (Figure 3a). Seasonal effects are apparent along mMDS axis 2 (from winter to spring to summer), while the contrast of rain versus dry was relatively much smaller (Figure 3a). This mirrors the relative sizes of these main effects in the PERMANOVA partitioning: Deposition, followed by Season, then Rainfall (Table 1). An interaction plot of the 18 Season × Rainfall × Deposition cell centroids not only shows the same pattern but also shows that seasonal effects (along mMDS axis 2) are much larger in high depositional environments (right-hand side of the plot) than for either the medium or low depositional environments. Pairwise comparisons (not shown here) further support these general conclusions.

6.2 Hierarchical Design with a Quantitative Covariate: Kelp Holdfasts

Anderson et al.⁵⁰ studied organisms colonizing holdfasts of the kelp, Ecklonia radiata; samples were collected according to a spatially structured hierarchical sampling design along the northeast coast of New Zealand. There were four locations (separated by hundreds of kilometers from north to south: Berghan Point, Home Point, Leigh, and Hahei); two sites within each location (separated by hundreds of meters to kilometers); two areas within each site (separated by tens of meters); and n = 5 replicate holdfasts (separated by meters) within each area. A total of 351 taxa from 15 different phyla were enumerated/quantified from these N = 80 holdfasts.

Interest lies in quantifying multivariate variation in assemblage structure at each spatial scale (beta diversity^{34, 53}) while taking into account natural variation in the sizes of holdfasts; the volume of each holdfast was measured using water displacement. Bray–Curtis dissimilarities were calculated on fourth-root transformed values.

PERMANOVA partitioning using a sequential (Type I) sum of squares showed the greatest component of variation was the residual (smallest scale), followed by locations, areas, and then sites; the latter two scales were comparable in their effect sizes (Table 2). Volume, although statistically significant (p < 0.001), was less important than any of the spatial factors. For this example, stress is relatively high even for a “main-effects” plot of distances among centroids (Figure 4), reflecting the high dimensionality of this multivariate system. Nevertheless, the distinctiveness of the four locations is quite apparent, the centroids for sites within them appear like satellites around each one, while area-level centroids appear, in turn, as satellites around each site centroid (Figure 4). Similar patterns were shown in a nMDS plot (stress = 0.12) and in a three-dimensional mMDS plot (stress = 0.13) of these distances (not shown). The “balance” of the two satellites (i.e., being opposite and equidistant) around any particular centroid in this fully nested design was seen more perfectly in the 3D mMDS plot.

6.3 Randomized Block Design: Plankton

A study by Winsor and Clarke^{60, 61} investigated the catch of several types of plankton using two nets hauled horizontally, one being 2 m below the other. Ten hauls were made with the pair of nets at depth positions of 29 and 31 m, respectively. Data were transformed to logarithms of the catch numbers for each of p = 5 plankton types.

This yielded a randomized block design. Ordination of individual sampling units showed no strong effect of position on these plankton assemblages – the two different symbols are quite well mixed (Figure 5a). However, PERMANOVA clearly detected a significant effect of “Position” (Table 3a). The key is PERMANOVA's ability to partition, hence partial out, the (random) effects due to different hauls (equivalently, to identify the “paired” nature of the nets in this design), thereby diminishing the residual and yielding greater power for the factor of interest: the position of the nets (upper vs lower). A useful plot here is the nMDS plot of residualized distances, having removed the (dominating) effects of hauls, in which the contrast (separation) between plankton assemblages in upper versus lower nets is now quite obvious (Figure 5b). Not surprisingly (given Figure 5a), no statistically significant effects of “position” would be detected if variation among hauls were to be ignored (Table 3b).