An alternative approach to overcome shortcomings with multiple testing of binary data in ecotoxicology

Binary data such as survival, hatching and mortality are assumed to be best described by a binomial distribution. This article provides a simple and straight forward approach for derivation of a no/lowest observed effect level (NOEL/LOEL) in a one-to-many control versus treatments setup. Practically, NOEL and LOEL values can be derived by means of different procedures, e.g. using Fisher’s exact test in coherence with adjusted p values. However, using adjusted p values heavily decreases statistical power. Alternatively, multiple t tests (e.g. Dunnett test procedure) together with arcsin-square-root transformations can be applied in order to account for variance heterogeneity of binomial data. Arcsin-square-root transformation, however, violates normal distribution because transformed data are constrained, while normal distribution provides data in the range ((-infty ,infty )). Furthermore, results of statistical tests relying on an approximate normal distribution are approximate too. When testing for trends in probabilities of success (probs), the step down Cochran–Armitage trend test (CA) can be applied. The test statistic used is approximately normal. However, if probs approach 0 or 1, normal approximation of the null-distribution is suboptimal. Thus, critical values and p values lack statistical accuracy. We propose applying the closure principle (CP) and Fisher–Freeman–Halton test (FISH). The resulting CPFISH can solve the problems mentioned above. CP is used to overcome (alpha)-inflation while FISH is applied to test for differences in probs between the control and any subset of treatment groups. Its applicability is presented by means of real data sets. Additionally, we performed a simulation study of 81 different setups (differing numbers of control replicates, numbers of treatments etc.), and compared the results of CPFISH to CA allowing us to point out the advantages and disadvantages of the CPFISH.