Fat tails and outliers: The water crisis

Recently, the occasional severe droughts that afflict the drier parts of the US have given rise to a predictable whine of "correct" journalism that wrongly attributes the "water crisis" to human-caused changes in water supplies, instead of correctly to a combination of rising demand and misguided scientific and policy assumptions. Like the fallacies of extreme weather and quantitative finance, the water panic is ultimately rooted in an essentially wrong picture of statistical fluctuations in open systems. Amplified by "correct" herdthink, it has turned into a perfect storm of bad science and bad journalism.

Statistics and stationarity. A statistical distribution is a substitute for full knowledge of a collection (possibly infinite in size) of particular instances. It's a way of stating limited knowledge. Stationarity means that, while individual instances can differ and limited samples might show time dependence as the distribution is sampled, the statistical distribution itself doesn't change. It's not always right, but for most cases, it is a good starting assumption in the absence of compelling evidence to the contrary.

The problem with stationarity is that it is often combined with another, much more questionable assumption, Gaussianity of the distribution: the distribution is assumed to be a bell curve, arising by the classical Central Limit Theorem from more primitive proto-distributions with finite moments. For open systems not closed with respect to exchange of flows with the outside world, it's well-established that Gaussianity is usually wrong. Better to assume the more general case, a Lévy distribution with some of its moments infinite. Recall such distributions have "fat tails" and support large deviations from the mean ("black swans").

Wrong conclusions prompted by wrong assumptions. For example, a stationary but non-Gaussian distribution, if it is sampled periodically, will produce moments that look as if they're increasing in time, perhaps implying a time-dependent distribution. In fact, this is a well-known fallacy in statistical reasoning. The moments are simply diverging. The distribution is stationary, just not bell-curve. There are other, better ways of analyzing samples from statistical distributions in this case.*

It's been known for over a century that rainfall and other water-related flows in our environment do not follow Gaussian random-walk behavior. Their fluctuations exhibit violations of this assumption, of broadly two types: memory or time correlations (so that each step in the walk is not independent of previous ones); and divergent moments (infinite variance or standard deviation, say). Either one of these should be enough to signal that Gaussian assumptions should be loosened. Unfortunately, a combination of theoretical prejudice and convenience has led policy-makers to stick to wrong assumptions anyway, just as financial traders and regulators (until recently) have been under the spell of the classical Gaussian random walk to explain price changes.

And now to bad journalism. There has been a recent water crisis in the American West. Actually, it happens roughly once a decade - the current one is in the midst of disappearing after an exceptionally snowy winter. Such crises are increasingly attributed to humans changing the available water supply and destroying "stationarity." In fact, it's the accompanying Gaussian or bell-curve assumption that's wrong.** The "throughput" of annual water flow available for human use changes year to year. But the underlying statistical distribution doesn't have to change.

And then to bad policies. Policies built on wrong Gaussian assumptions will lead to the same characteristic mistake over and over: the conclusion that fluctuations should be frequent but small deviations from a well-defined mean. In reality, the distribution has much larger fluctuations, which hit regulators, policy-makers, and ordinary folks again and again as "surprises" or "crises." But there's no crisis, just wrong assumptions. Gaussian-stationarity was never a reality, only just an assumption, and a bogus one at that.

In the case of water flow, increasing demand raises the chance that, in any given year, the the water system will be flowing below the threshold needed to meet that demand. There are three solutions.

Lower demand. Much water use in the American West is heavily subsidized by state and federal governments. Much of it goes into marginal and inefficient agriculture (for example, growing alfalfa and rice - both monsoon crops - in the California Central Valley). Reduce those subsidies, and you'll reduce demand.

Boost reserves. Just as banks and other financial institutions should be required to hold on to larger reserves to meet "black swan" crises, so a hedge against a large downward fluctuation in water flow is to build more and larger reservoirs, and to manage them more conservatively.

Change expectations. If scientists, policy makers, politicians, and voters carry around in their heads a false concept that water flow is basically steady and predictable, they'll treat the inevitably different reality as a "crisis." If everyone involved understands that water flow is subject to large changes year to year and not predictable beyond outlining a rough range, the policies will be different and more oriented around hedging better against drought by storing water during good years.

The Earth's water flow is deterministic, like all aspects of climate. But it's also chaotic. Fat-tailed statistical distributions are characteristic residues of chaos, as is the typical pattern ("intermittency") of clusters of good years and bad. Caveat emptor.
---
* Use the absolute linear range, for instance.

** More technically, it's the assumption that the distribution can be estimated by estimating moments from past water flow data. If some or all of the distribution moments are infinite, this technique doesn't work and never worked. Scientists, engineers, and policy-makers who thought otherwise were simply fooling themselves.