1.1 Problems with Assuming Normality

To begin, distributions are never normal. For some this seems obvious, hardly worth mentioning. But an aphorism given by Cramér (1946) and attributed to the mathematician Poincaré remains relevant: “Everyone believes in the [normal] law of errors, the experimenters because they think it is a mathematical theorem, the mathematicians because they think it is an experimental fact”. Granted, the normal distribution is the most important distribution in all of statistics. But in terms of approximating the distribution of any continuous distribution, it can fail to the point that practical problems arise, as will become evident at numerous points in this book. To believe in the normal distribution implies that only two numbers are required to tell us everything about the probabilities associated with a random variable: the population mean μ and population variance σ2. Moreover, assuming normality implies that distributions must be symmetric.

Of course, nonnormality is not, by itself, a disaster. Perhaps a normal distribution provides a good approximation of most distributions that arise in practice, and of course there is the central limit theorem, which tells us that under random sampling, as the sample size gets large, the limiting distribution of the sample mean is normal. Unfortunately, even when a normal distribution provides a good approximation to the actual distribution being studied (as measured by the Kolmogorov distance function, described later), practical problems arise. Also, empirical investigations indicate that departures from normality that have practical importance are rather common in applied work (e.g., M. Hill and Dixon, 1982; Micceri, 1989; Wilcox, 1990a). Even over a century ago, Karl Pearson and other researchers were concerned about the assumption that observations follow a normal distribution (e.g., Hand, 1998, p. 649). In particular, distributions can be highly skewed, they can have heavy tails (tails that are thicker than a normal distribution), and random samples often have outliers (unusually large or small values among a sample of observations). Outliers and heavy-tailed distributions are a serious practical problem because they inflate the standard error of the sample mean, so power can be relatively low when comparing groups. Modern robust methods provide an effective way of dealing with this problem. Fisher (1922), for example, was aware that the sample mean could be inefficient under slight departures from normality.

A classic way of illustrating the effects of slight departures from normality is with the contaminated, or mixed, normal distribution (Tukey, 1960). Let X be a standard normal random variable having distribution Φ(x) = P(X ≤ x). Then for any constant K > 0, Φ(x/K) is a normal distribution with standard deviation K. Let ε be any constant, 0 ≤ ε ≤ 1. The contaminated normal distribution is

     (1.1)

which has mean 0 and variance 1 − ε + εK2. (Stigler, 1973, finds that the use of the contaminated normal dates back at least to Newcomb, 1896.) In other words, the contaminated normal arises by sampling from a standard normal distribution with probability 1 − ε; otherwise sampling is from a normal distribution with mean 0 and standard deviation K.

To provide a more concrete example, consider the population of all adults, and suppose that 10% of all adults are at least 70 years old. Of course, individuals at least 70 years old might have a different distribution from the rest of the population. For instance, individuals under 70 might have a standard normal distribution, but individuals at least 70 years old might have a normal distribution with mean 0 and standard deviation 10. Then the entire population of adults has a contaminated normal distribution with ε = 0.1 and K = 10. In symbols, the resulting distribution is

     (1.2)

which has mean 0 and variance 10.9. Moreover, Eq. (1.2) is not a normal distribution, verification of which is left as an exercise.

To illustrate problems that arise under slight departures from normality, we first examine Eq. (1.2) more closely. Figure 1.1 shows the standard normal and the contaminated normal probability density function corresponding to Eq. (1.2). Notice that the tails of the contaminated normal are above the tails of the normal, so the contaminated normal is said to have heavy tails. It might seem that the normal distribution provides a good approximation of the contaminated normal, but there is an important difference. The standard normal has variance 1, but the contaminated normal has variance 10.9. The reason for the seemingly large difference between the variances is that σ2 is very sensitive to the tails of a distribution. In essence, a small proportion of the population of subjects can have an inordinately large effect on its value. Put another way, even when the variance is known, if sampling is from the contaminated normal, the length of the standard confidence interval for the population mean, μ, will be over three times longer than it would be when sampling from the standard normal distribution instead. What is important from a practical point of view is that there are location estimators other than the sample mean that have standard errors that are substantially less affected by heavy-tailed distributions. By “measure of location” is meant some measure intended to represent the typical subject or object, the two best-known examples being the mean and the median. (A more formal definition is given in Chapter 2.) Some of these measures have relatively short confidence intervals when distributions have a heavy tail, yet the length of the confidence interval remains reasonably short when sampling from a normal distribution instead. Put another way, there are methods for testing hypotheses that have good power under normality but that continue to have good power when distributions are nonnormal, in contrast to methods based on means. For example, when sampling from the contaminated normal given by Eq. (1.2), both Welch’s and Student’s method for comparing the means of two independent groups have power approximately 0.278 when testing at the .05 level with equal sample sizes of 25 and when the difference between the means is 1. In contrast, several other methods, described in Chapter 5, have power exceeding 0.7.

In an attempt to salvage the sample mean, it might be argued that in some sense the contaminated normal represents an extreme departure from normality. The extreme quantiles of the two distributions do differ substantially, but based on various measures of the difference between two distributions they are very similar, as suggested by Figure 1.1. For example, the Kolmogorov distance between any two distributions, F and G, is the maximum value of

the maximum being taken over all possible values of x. (If the maximum does not exist, the supremum, or least upper bound, is used.) If distributions are identical, the Kolmogorov distance is 0, and its maximum possible value is 1, as is evident. Now consider the Kolmogorov distance between the contaminated...