Why weather is hard to predict

If my climate scenarios seem lifeless and idealized, they are. They're not real climates, but caricatures. Even the most complicated computer models of climate are caricatures, albeit caricatures with more details.

There is no sharp dividing line between "weather" and "climate." We all know weather is hard to predict, yet has repetitive features. But we tend to think that there is a long-term average of "climate" that is stable - but such a thing doesn't exist. "Climate" really just means "weather in the long run" - and it shares the same duality that we see in weather - repetitive, yet unpredictable.

It was the 19th century's physics that defined thermodynamic equilibrium and gave exact definition to now-familiar concepts like temperature and pressure and more esoteric notions like entropy. The atmosphere as a whole is not in thermodynamic equilibrium, because it has no one temperature or pressure, and its phase equilibrium of liquid and evaporated water shifts depending on temperature. The first half of the 20th century gave physics broader concepts that take us closer to what the atmosphere is really like. A local thermodynamic equilibrium (LTE) allows temperature, pressure, and phase equilibrium to change from one place and time to another, but still have exact definition at each point. It's a broader definition of thermodynamic state than equilibrium. Because of the variation of pressure, temperature, and phase, the LTE distributions of those fields must be supplemented by a list of the flowing matter, heat, radiation, and evaporation-condensation-precipitation. To really understand this steady state of nonequilibrium is to understand that the flows are causes and the local equilibria effects, not a cause.

But we're still not at the real atmosphere. In real life, equilibrium is always local, temporary, and approximate. Scientific education still biases the thinking even of scientists who should know better into an unconscious prejudice that equilibrium is more fundamental, instead of seeing the world as it actually is: equilibrium is a condition that will pass.

The flows of air, water vapor, and energy are not steady. Different components of the atmosphere, even at the same point in space and time, have slightly different temperatures. Phase equilibrium is frequently violated in a pretty serious way, at least outside clouds. The dynamics of the atmosphere is represented by a complicated set of nonlinear partial differential equations that describe how the thermodynamic and hydrodynamic fields change continuously and how cause and effect are linked on the stage of spacetime. They have no exact solution and even the most powerful computers are strained to the limit by attempts to solve them using approximate computation methods.

Buried in these equations is one of the greatest discoveries of 20th century science - why the world as we know it is both repetitive and yet shot through with a stream of unique, never-to-be-repeated events. In the last third of the 20th century, physicists, mathematicians, and others discovered chaos. It's a topic that deserves and will get its own full discussion. In fluids like the air, the chief manifestation of chaos is turbulence, of which our familiar heat convection is one part.

Chaos is what makes it impossible to predict weather much beyond two weeks. After trying naively and futilely in the 1950s and 60s to make exact predictions of weather with ever more computing power, meteorologists in the 70s and 80s abandoned the goal of a completely self-contained weather theory and instead invented sophisticated ways of combining present and past meteorological data (synoptic meteorology) with general physical principles (dynamic meteorology) into a fruitful symbiosis that draws on the strengths of each. Synoptic analysis uses the atmosphere itself as an analog computer - present and past observations suggest patterns and probable future evolution. Dynamic meteorology introduces physics into the picture, to enforce general laws of nature and veto predictions that would violate them - say, energy or mass conservation, or an approximate condition like hydrostatic equilibrium known to be satisfied to high accuracy. Computers are essential for this symbiosis, since they make it possible to store and compare vast numbers of atmospheric observations. In the old days, meteorologists had to rely on limited human memory and ability to compare similar situations.

The symbiosis of synoptic and dynamic meteorology began in the 1930s, but until the 1970s, the hope remained that the synoptic part could be eliminated and meteorology could be all dynamical, based on complete, self-contained theoretical predictions made from scratch (ab initio) using the complete theory of the atmopshere: fluid dynamics, geophysics, and radiation. The discovery of chaos dashed those hopes and made it clear that the symbiosis wasn't just practical - it was a deep necessity. Although the dream of complete theoretical weather predictions came to an end and the two-week limit had to be accepted, scientists have since learned to ask a different set of questions about the long-term behavior of chaotic systems. Chaos includes most of the non-trivial dynamical systems around you: the rushing stream, the human heart, the power grid, the stock market - and the weather.