Significance level 0.05 in comparison of two groups (A and B) indicates that the possibility of taking true hypothesis is 95% and taking wrong hypothesis is 5%. When we do paired comparison of three groups (A with B, B with C and A with C), the possibility of such mistake would be:
1-(0.95×0.95×0.95) = 14.3%.
If we have four groups (A, B, C and D), to compare all them we would do six comparisons: A with B, A with C, A with D, B with C, B with D and C with D. In this case, the probability of accepting wrong hypothesis would be:
1-(0.95×0.95×0.95×0.95×0.95×0.95) = 26.5%.
With increasing the number of paired comparison in multiple groups the probability of accepting wrong statistical hypothesis dramatically increases.
Multiple t-tests should never be performed. When we have several groups to compare, ANalysis Of VAriance (ANOVA) should be used instead of paired comparison of all groups.
ANOVA has specific terminology:
Way – an independent variable: if we have one grouping variable (species of bacterium including at least three different values) and one independent variable (zone inhibition around the disk with essential oil), then it will be one-way ANOVA. Student’s t-test actually is a variant of one-way ANOVA when grouping variable has only two values and, therefore, two groups are compared.
Factor – a test or measurement. Single-factor ANOVA tests differences between means of one factor (zone of inhibition around the disk with antimicrobial agent) in compared groups. Two-factor ANOVA simultaneously tests two different factors in compared groups, for example, zone of inhibition and minimal inhibitory concentration. Actually, a way and a factor are alternative terms for the same thing (independent variable).
Repeated measures – when one variable is measured several times under different conditions. For example, when inhibition zones are measured in triplicates during three consecutive days. Application of standard ANOVA in this case is not appropriate because it does not take into account correlation between the repeated measures, and the assumption of independence is, therefore, violated. Repeated measures ANOVA is comparable to a paired-samples t-test but with several groups instead of two.
Like other statistics, ANOVA tests also include parametric and nonparametric tests. However, nonparametric tests are rather scarce. First, we will discuss parametric variants of ANOVA tests, such as one-way (single-factor) ANOVA, two-factor ANOVA and repeated measures ANOVA. Then we will discuss nonparametric alternatives of ANOVA.
Parametric ANOVA tests the null hypothesis about equality of the means of all the groups which are compared. It produces a statistic called F which represents an equivalent to the t-statistic from a t-test. However, this statistic only demonstrates presence or absence of difference between groups, it answers the question, whether we should accept or reject the null hypothesis. If the null hypothesis is rejected (differences are statistically significant), then we should search answer to a question, which of the groups is actually different from others. Parametric variants of ANOVA calculate for this purpose so called “post hoc” (after the event) tests, which are aimed to point the group being different from others.
