In the face of chaos or, Chaos casts a shadow

Chaos is the score upon which reality is written.

- Henry Miller

Any natural system, unless proven otherwise, should be assumed to be nonlinear and possibly chaotic. If we can prove it's not linear, we then need to ask, is it chaotic? Under what conditions? If not, under what conditions? Then to move on: can basic degrees of freedom and distinct subsystems be identified? Can their trajectories in time be plotted?

The weather, the state of the Earth's atmosphere, is known to be chaotic, with a Lyapunov time of about two weeks. That is, given anything like our current knowledge of the Earth's weather at any instant of time, weather predictions can be projected out to no more than two weeks, before the forecasts become no better than random guesses.

Financial markets are also known to be chaotic, but unlike the weather, we have no precise, laboratory-controlled fundamental principles to start with. There are many different kinds of financial markets as well. They do seem to lose any distinctive predictability after periods ranging from months to a few years.

What to do? We can't make long-term predictions of chaotic behavior. But chaos is bounded aperiodicity, not unbounded. Over time, it traces out increasingly and ultimately infinitely complex trajectories of temperature, pressure, precipitation, or prices and commodity flows. They're bounded, however, which suggests the notion of a "box" or a "range."

Mathematicians have given us a more subtle and precise version of a "chaos box," called a stable manifold.* Once the periodic and transient behaviors of a system are analyzed and removed from consideration, what's left is the untameable but still boundable meanderings of chaotic motion as it traces out an attractor. The actual record of chaotic trajectories is an infinitely complex fractal, but a stable manifold shadows that fractal in such a way that the manifold does not change over time and the chaotic trajectories don't cross it.**

Such stable manifolds are "chaos captured," to the extent that it can be. They give us a way to take a for-all-time snapshot of chaos without attempting the impossible task of tracing the actual chaotic trajectory for an infinite period.

An earlier posting linked to the stable manifold for the Lorenz attractor, the first modern model of chaotic turbulence arising in the atmosphere. The stable manifold shadows the Lorenz attractor and, being two dimensional, can be converted into a crochet pattern.
---
* "Manifold" is mathematics jargon for a space, like a two-dimensional surface embedded in three-dimensional space, that looks Euclidean locally, but maybe not globally. "Euclidean" means its geometry is the one you learned in high school.

** Mathematicians and physicists say that this object (the stable manifold) is invariant.

William F. Buckley, Jr.: An appreciation

Where to start? A giant of the postwar world, one of the founders of modern conservatism, and a leading light who remade American politics, died just over a month ago. Many tributes have appeared, and his positive influence is everywhere you look.

When World War Two ended, American conservatism seemed to be finished. Although opposition to the increasingly statist bent of liberalism came from many and often conflicting points of view (individualism, religion, nativism, etc.), the two large forms of opposition were isolationism (opposition to American entry into the war - although that was not in any way purely a cause of the Right) and hostility to the dramatic expansion and pretensions of the federal government. Both oppositions appeared discredited, although disillusionment with the New Deal and federal intervention in general was widespread by the time of the 1940 election. And a significant part of the Republican party - its East Coast, Anglophile wing, led by Wendell Wilkie - did support intervention, in fact preventing foreign policy from becoming a major partisan issue in the 1940 and 1944 elections and marginalizing isolationism to a great degree. Nonetheless, while much of the New Deal had been repudiated after the 1938 midterm elections, everyone in 1940 knew what the great question was: to intervene or not, how fast and how deep. Of course, the Germans' undeclared submarine war of 1940-41 and the attack on Pearl Harbor at the end of 1941 rendered that debate moot. The US did intervene in World War Two, on the Allied side, and few regretted it afterwards, in spite of the immense cost and the dubiousness of having the Soviet Union as an ally.

Other factors were at work in the fragmented state of the American right in 1945. It had some of the same components familiar today. There were conservatives, people who deferred to tradition because it was tradition. The fact that the American political tradition was largely a liberal one (with one major exception) created something of a paradox.* That tradition was one of limited government, one overturned by the socialist-tinged "new" or "reform" liberalism of the Progressive and New Deal eras. (The change was large enough that it is questionable whether the adjective "liberal" should be used at all.) Classical liberals (later called libertarians) never bought into tradition-for-tradition's-sake and based their views on straightforward, historically informed ideas (abundantly confirmed by recent experience in the world wars and totalitarian dictatorships) about the negative effect on human liberty and prosperity by too-large and too-powerful a government. Some went further and re-developed the familiar 18th-century theories of natural rights and consensual government in terms understandable to a 20th century audience. To that they added a more modern and deeper appreciation of the limits of social knowledge and the impossibility of social planning.

A striking fact astute readers might notice is that I make no mention of the "religious right" in this potted history. That's because, after the liberal-fundamentalist battles of the late 19th and early 20s centuries, the fundamentalists retreated to their own, anti-political, subculture. To the extend that they were political, they largely voted Democratic, because they were largely southern or urban-ethnic Catholic - and in those days, virtually all white southern and Catholic voters voted Democratic. The "religious right" as we now know it, an alliance between conservative Catholics and conservative Protestants, was 30+ years in the future at the end of the war. In 1945, a few feeble attempts at ecumenism apart, the relation between Protestants and Catholics was one of mutual suspicion, laced with ignorance and prejudice - although the war itself had had a strong positive impact on this state of affairs. The full scope of this impact is in large part the complete transformation of religious and ethnic identity in America in the postwar era. This transformation made black civil rights easier to swallow (for whites), gave rise to a new level of interreligious understanding, and made Buckley's type of "big tent" conservatism possible. All of these were out of reach and almost unthinkable before 1945. In his relationship with American Jews, Buckley's approach was even more radical, given the level of anti-Jewish prejudice still prevalent in 1945. By the late 1950s, his nascent movement and its allies counted a significant number of Jewish thinkers, many of them refugees from Europe and carrying first-hand experience of life under the totalitarian dictatorships. Well before the Republican party, under Nixon, fully embraced support of Israel, Buckley adopted such support as a matter of course, because he understood the basic issues at stake. When the Soviet Jewry movement started in the mid-1960s, Buckley and his followers were among its early non-Jewish allies and supporters.

Buckley was a wealthy liberal Catholic, a man from a social background (including a father who was an American ambassador) that, on the surface, seemed conventionally WASP. It was his quirky Catholicism that initially made the difference. It made him have to think - really think - about political issues in a deeper way than most people have to, people who take their political views without much thought from their parents, teachers, and peers. This led him initially to oppose modern American liberalism from a traditionalist point of view, but also made him open to libertarian approaches. From these experiences, he formed a clear conception of how to rebuild the American right around a few simple and broadly appealing stands that any rational person could assent to, without regard to ethnic or religious background: opposition to the growth of government at home and opposition to the dominant form of collectivist tyranny abroad, communism. Buckley and his allies founded National Review, still the flagship of American conservatism, and began a campaign that stretched from the early 1950s to the late 70s to absorb all opposition to modern liberalism in this way. By doing this, Buckley sidestepped the leftover isolationist-interventionist debate and ended the American right's parochialism. By the late 1950s, all sorts of people, many Protestant, but a large minority of Catholics, and a surprising number of Jewish thinkers had been pulled into Buckley's movement, so much so that by the 1960s, the movement had become a broad political tendency in its own right and no longer a "movement." Buckley concentrated on the crucial common ground, dispensed with the provinciality and ethnocentrism of the prewar right, and even proved ecumenical enough to accept individualist and libertarian followers. (They often rejected him anyway, but that was their doing, not Buckley's.) Buckley even flirted with libertarianism himself, in his 1965 run for Mayor of New York and his famous, offshore 1960s pot parties on his yacht.

The era of Buckley's and conservatism's success was the late 70s and 1980s. In the 1960s, American liberals had obtained a blank check, under Kennedy and Johnson, and led themselves and the country, both domestically and internationally, into a gratuitous mess: instead of triumphing, liberalism went haywire and, after 1965, began a free-fall from which it has never recovered. The fact that the president who brought the Kennedy-Johnson tendencies to their logical conclusion - Richard Nixon - was a Republican seemed at first to set back the conservative cause. But Nixon was not a conservative. He was nominated in 1968 in part because he wasn't Goldwater and couldn't be tarred as a "crazy radical." The Watergate crisis (and the war that led to that crisis) were largely, and correctly, perceived as failures of the new, expensive, obtrusive, and frequently dysfunctional expanded government introduced by liberals. Nixon went along with it ("we are all Keynesians now," he famous said in 1972) and did big government a few times better - wage and price controls, entitlements and affirmative action as we now know them, and so on. His fall opened up a singular opportunity for conservatives, one taken by Reagan. The rest is history: a halt in the growth of domestic spending in the 1980s and 1990s, the successful and peaceful end of the Cold War, the end of "stagflation" and a generation-long success of low-inflation economic expansion and moderating the growth of the federal government. It was a stunning achievement, especially for a political movement - conservatism - that never gained majority acceptance. Buckley's approach saw to it that what we now call "identity politics" would not wreck his coalition and that, even when not having a strong influence over the Presidency, conservative ideas and policies shaped by those ideas would have their continued impact. Certainly, the rise of Clinton in 1992 was a direct response - "neoliberalism" - to the prior failure of liberalism and the success of conservatism. Clinton went on, chastened by the 1994 mid-term elections, to become the most conservative Democrat since - well, Grover Cleveland.

Buckley, like his movement, spent more recent years in ill-health. Certainly, the decline of conservatism after 1994 troubled him. Even more troubling was the return of "Nixonism," in the form of Bush Jr. and Karl Rove. This tendency started at the end of the 90s with a grand throwing-up-of-the-hands by conservatives, a giving-up on ever implementing a conservative program. Being the majority party in Congress put them into the position of actually having to decide whether to go ahead with serious reform, and they couldn't do it. (Many thought conservative reform politically impossible.) Another disturbing trend was the return of authoritarian, big-government populism, as signaled by Perot in 1992 and 1996, with its powerful tendency to split the right-leaning vote. (Indeed, Perot made Clinton's election and re-election possible.) To capture these voters for the Republican party meant dropping much, if not all, of conservatism as Buckley and others had defined it. The prize was permanently incorporating white southern and Catholic voters into the Republican coalition. The price: alienating everyone else - and proving multiple times over that big government fails for an inherent reason, not just when it's done by liberals. It also led, on an intellectual level, to nasty disputes between conservatives and libertarians on one side and neoconservatives on the other, those disaffected liberals who dropped out of the Democratic party in the 1970s and 80s. The opposition of many conservatives and libertarians to the Iraq war sharpened the nastiness.

Buckely's synthesis of traditionalism and "classical" liberalism (to use the L-word in its original and best sense) seemed unstable and incoherent to many. But it has proved enduring, like his very Catholic mix of faith and philosophical reason. It is both seriously intellectual, answering to that part of us that is consciously rational, and attractive because it recognizes the limits of individual reason. It combines with a classical liberal understanding of government a solid understanding of the limits of politics. Conservatives view society as more important than government, culture as more important than politics, and custom as more important than law. Buckley made his own contribution to culture as a prolific author and as host of public television's Firing Line for more than 30 years.**

He will be missed.
---
* That exception was the semi-medieval views that had taken hold in the antebellum South regarding slavery, which then influenced the dominant American thinking about race from the end of Reconstruction, in the 1870s, until the 1950s.

** When Binah was a youngster, he was confused by Buckley's sometimes notoriously Latin-inflected discussions on Firing Line, reflecting Buckley's high-level Catholic education. Latin came later to Binah and, with it, some understanding of what Buckley was talking about. Buckley himself adopted a less erudite and more straightforward style in later years.

Normal women and the men who love them

From the Wall Street Journal comes this silly story about the alleged "backlash" against Hillary Clinton leading to an alleged "backlash" against women in general. (Hat tip to Instapundit.)

Unlike blacks and whites, who are still too separated from one another, men and women spend enough time together and understand each other well enough (sometimes too well) to make such worries seem pathetic. And listen to the absurd assumptions behind all such sloppy journalism: we need such candidacies to feel "empowered." What a joke: politicians almost always reduce our freedom - they rarely enhance it.

In our society, the position of women and the way men view them, and the way women view themselves, has little linkage to the vagaries of politics and the ups and downs of Hillary Clinton. It was always a mistake for her and her supporters to frame her candidacy this way. Fortunately, relations between the sexes are not much impacted by such things, and the influence that radical feminists once had, 20 or 30 years ago, on our thinking has mercifully faded to almost nil.

Too bad for the identity politics mongers - but nice for the rest of us.

The coming Obama implosion

I probably shouldn't call it "coming" - it's already here, and it's not even April yet. For example, Christopher Hitchens takes another hard look at Obama and doesn't like what he sees. In case anyone's wondering if the Obama candidacy is finished or not, see this and this. There's always a certain amount of BS going on with figures at the fringes of any presidential campaign. But the core of this material is published and publicly available. The problem remains, not so much Obama, as the people around him. If the Clinton people are retreads from a moderately successful administration 10 years ago, Obama's people are failed retreads and would-bes from 20, 30, even 40 years ago.

There's not a lot more to say about the angry Pastor Wright. Long-time Obama-watchers were not surprised by the recent flap. Wright, a well-to-do pastor and child of upper-middle-class privilege, even more than Obama himself, is undoubtedly a cult leader captivating hearts and minds with myth-spinning, half-truths, and "hate speech." But take away the race mongering, and you have a successful cult guru not unlike, say, the Maharishi - and not at all like, say, Jim Jones, who took his beliefs far too seriously. After all, Jones committed suicide. Pastor Wright would never, ever do that to himself. Think instead of all those other angry cultists of the New Left and of how privilege has its price: the Chomskys, the Patty Hearsts, the Weather Underground, etc. - all those children of success acting out some weird personal drama and sometimes inflicting it on the rest of us. Mostly, such people need to heed their own advice and start with themselves.

But back to Obama: while not a full-blown cult follower, he is clearly a cult fellow-traveller of many years. The cult beliefs don't at all fit at all his own biography. But joining the cult was a shrewd political move - it allows him to keep one foot in angry identity politics, even if it's personally irrelevant to him, while still selling himself as "post-racial" to credulous white middle-class boobs. If anyone was paying attention, this whole dynamic was brilliantly captured and dissected last year by Shelby Steele.

Unfortunately, the Obama craze will not go away completely until the April 22 Pennsylvania primary. Clinton will almost certainly win it, by a large margin, leaving the Democrats with two candidates: one with a less-than-even but still serious shot at winning in November, and the other with even more limited appeal and his primary vote lead all coming from his home turf, Cook County, Illinois. The question then becomes, how crazy are those Dems? They specialize in self-destruction, and slavery/segregation/race has been their albatross for more than a century and a half: can they shake it?

POSTSCRIPT: For another slant on what's wrong with the Democrats, see here. Hint: it's more or less the same thing that's been wrong with them for a long time.

And all I got was this lousy New York driver's license

What about the rest of us? To the extent that we root for strong politicians, join political cults, invest our hopes and desires in charismatic leaders, all of us are Spitzer wives.

- Arnold Kling

Let's take a break from all that fancy climate stuff and pay a visit to Albany, where New York's now-former governor has cleaned out his office.

Is there anyone - really, anyone - in American politics more sanctimonious than Eliot Spitzer? When you survey the damage he caused in New York's financial world and the careers he ruined - for no reason, mind you, except his own megalomania - what's just happened to him could not have happened to a more fitting public servant. I guess there was a reason: building a career on prosecutorial wreckage, like some of America's other notorious official lawyers (think Mike Nifong of the Duke non-rape scandal). All of these guys discovered the power of threatening prosecution, not actually prosecuting: it leaves its targets defenseless. With egomania at the top of his agenda, it's not surprising that Spitzer's public approval ratings dropped from about 80% to under 10% in his first year as governor, and that was before this, the final straw. He fancied himself, and was fancied by others, as a "reformer" or a "progressive" or an "enforcer." In fact, he was a pathetic and loud-mouthed bully, enabled by adoring flunky journalists. It's another case of how corrupt our news media has become. They abandoned their most important function vis-a-vis politicians: keeping the bastards honest.

The Wall Street Journal waged a long and brave campaign against Spitzer. See their comments here, in particular those by John Fund and Kimberley Strassel. It comes with the territory, a familiar recipe: lots of power and/or money leads to narcissism and a sense of entitlement, and even a sense of "doing well by doing good." The former scheduling girl for Spitzer's high-priced New York escort service shares her experiences here. Spitzer was also prosecuting both call-girl rings and their johns. He was evidently writing down some phone numbers on the side.

Basically, he's a schmuck.

Don't miss Alan Dershowitz's lame and unconsciously funny attempt to defend Spitzer, which every woman of my acquaintance just laughed herself to death over. The problem with Dershowitz is obvious: the man who helped defend O. J. has tried very hard, and not quite successfully, to remake himself as a respectable Harvard professor with his books, lectures, and involvement in Jewish causes. But when something like the fall of Spitzer happens, the old Dershowitz is back with his smokescreen of rationalizations. His attempts to portray what happened to Spitzer as entrapment are simply wrong. No government agent solicited anything from him or was solicited by him. The Feds were just checking up on what looked like curious campaign finance transactions, which is how they discovered the money-laundering ....

Spitzer has apparently decided to sue the government, unclear for what. Having abused his position as prosecutor, then the governor's office, he'll now be abusing the legal system as a pseudo-plaintiff. Time for a pity party, I suppose.

The decline of basic research

The true mystery of the world is the visible, not the invisible.

- Oscar Wilde

The "global warming" hysteria has contributed to and been enhanced by a worrisome and strengthening trend of the last 20 years, the decline in interest and support for basic research. Behind this trend, among other causes, is the feeling that hard and deep scientific questions are already answered or are unimportant.

Climate hysteria as a major culprit. Neither is true, certainly not in the case of weather and climate. But if conclusions about climate and "global warming" already predetermined, then why bother with fundamental research about the question? And with so much of the political and media establishment already with settled and closed minds on the issue - not on any rational basis, mind you, but with the righteous assurance that "something must be done!" - open-ended, curiosity-driven research suffers. After the 1992 Climate Summit at Rio de Janeiro, scientists, even ones with long-standing careers and impressive track records in climate research, suddenly found themselves having to couch their research grant pitches in language reminiscent of medieval theology: the conclusions were foreordained, and research questions had to take a back seat to "something must be done!" The demand certainly came from, not scientists, but politicians and later the media: from the point of view of scientists, it was top-down, not bottom-up, and outside-in, not inside-out.

Distinct but related is the redirection of educational resources - talent, time, and money - away from scientifically sound disciplines like climatology, meteorology, and geosciences, toward poorly-defined and overtly politicized programs in "environmental studies," ones with - again - poorly grounded but preordained conclusions narrowing the scope of what can be asked or investigated.

Stem cells: Another case. "Global warming" isn't the only instance of politicians and journalists doing more than prejudging scientific outcomes and actually attempting politicized science made-to-order. Another is stem-cell research, upon which subject the conventional news media can be safely assumed to misreport or outright lie about. Most of what they say about the subject is done in the service of - again - predetermined political "narratives" with only the most tenuous connection to reality.

Embryonic stem cell research has produced and continues to produce important basic research results. However, it has produced nothing of therapeutic value so far, a fact apparently lost in all the bruhaha. Therapies are coming out of adult and placental stem cell research.

The stem cell research episode did highlight a disturbing trend, the "porkification" of public science funding - the conversion of government research support into patronage and pet projects of politicians and their followers. There are many examples. Apart from its fanaticism, the "global warming" cult fits this pattern in many ways. The trend started in the late 1980s, but did not become fully evident until the late 1990s.

Climate scientists have, to a large extent, been similarly abused as dummies by powerful groups "throwing their voices." This is not to say that certain scientists haven't been in on the cause themselves. But the most important of them are not climate scientists, and their scientific arguments are often a joke: sloppy and demagogic, violating much of what's been learned about climate in the last two centuries, and sometimes violating basic physical principles. Furthermore, there is no conspiracy by the Bush administration to suppress good science, and its approach to the "global warming" cult has been inconsistent and confused rather than just negative. There are serious disagreements about policy, no doubt - stem cells was the most significant, not climate - but these are not disagreements about science proper, but rather about what's ethical and deserving of government funding.

What's worrisome here is the lack of recognition of the need for embryonic stem cell research as a basic research priority - understanding reproduction, genetics, and so on. Instead, the "hard sell" is nonexistent applications and therapies based on embryonic stem cells, often in pathetic language better fit to a revival meeting.

Knowledge versus authority. The long-term cost of all this is subtle but no less devastating for all that. It means we've entered an era in which, thanks to the pervasiveness of politicized news media chatter, scientific questions increasingly can no longer be investigated and debated scientifically. Instead, the authorities (politicians) and pseudo-authorities (journalists) will pre-decide the conclusions, and scientists will just have to "go along to get along" if they want public research funding and publication in reputable journals.* That's what did in Galileo: he thought he was debating science ("natural philosophy" in those days); unwittingly, he had been sucked into an authority struggle.

Science will never be fully cut-and-dried, a cookbook. It requires a certain level of taste and a good intuition for good problems, ones that are interesting, exciting, and solvable. That's what distinguishes it from empty speculation. But a political culture unheedful of the non-authoritarian nature of science, its open-endedness, can't help but create a crisis of science once politically-determined conclusions are used as clubs to beat the non-compliant into submission. Meanwhile, policies are increasingly built on things known to be wrong or still unanswered, all because people don't want to look at hard questions or don't like the answers.

The essential conflict here is one between knowledge and political authority. Knowledge is not about political authority; the two are hard to reconcile with one another. Politicians, who carry political authority in our society, are in conflict with knowledge. Appointed bodies of "official" and "consensus" science - mostly not scientists, and not at all acting scientifically - establish the conclusions, then not-too-subtly hint to the news media that, with "consensus" science already pre-established, it's okay to vilify and smear dissenters and to spread the lie that there is a "consensus" of scientists on the issue. This Big Lie is then used as another club to keep dissenters quiet. Facts are turned into decontextualized factoids, misleading half-truths, or ignored altogether - when they don't fit the "narrative" the media is desperate to push, all so "something can be done." When the basis for the hysteria fizzles, the subject is changed and the lies, mistakes, and half-truths forgotten.

It's not all just about the money. The entire fake crusade of "global warming" is built on the falsehood that we have a climate theory that can answer the relevant questions. But, really, we don't. Spending more money for "climate change" research will actually make things worse if it's misdirected into "research" that takes such answers as known and given, instead of as unknowns. It will drown us in more thinly-disguised special pleading and propaganda, distracting everyone from important issues and pushing scientists into dead-ends.

Support for basic research doesn't necessarily mean an increase in total science funding. When one considers the money squandered on the Space Shuttle or the International Space Station, it's clear that total dollars is not the best metric. The real problem is the creation of political bodies specially empowered to decree pre-ordained conclusions that are are then relentlessly repeated by the media and start to shape scientific and educational agendas.**

People at large can believe whatever they want to, although they also have to accept whatever the consequences of those beliefs are. But the political dictation of science itself is a distressing and potentially catastrophic trend. It's happened before: in the 17th century, it nearly destroyed astronomy and related sciences in Italy during the Inquisition. Early modern natural science survived in certain Protestant countries and in Catholic countries (like France) where the Inquisition was barred from entering. It destroyed genetics and biology in the Soviet Union during the Lysenko episode. Biology survived in the West. Hitler and his allies among academic faculty and students destroyed what had been the leading scientific culture in the world. It took Italian science two centuries to recover. Russian science still hasn't recovered, and in many ways, neither has the German.

Will this fate overtake climate and geosciences in the West? In our case, it won't be a powerful Church or Party that will do the deed. It will be fanatical activist groups, demagogic politicians, and cynical ex-scientists, all under the spell of a sanctimonious crusade - with opportunists of all types and lackey scribblers tagging along, trying to ride a tiger they should never have gotten on in the first place.
---
* A striking case is the decline of Scientific American, once a highly literate and superbly written venue for the popularization of science. The articles were largely written by scientists or writers trained and practicing in science. But when it passed from Greatest Generation to Worst Generation (in the late 1980s, at the same time as America's universities), it entered an era of turmoil, sale and resale, and ultimately conversion into something like People magazine - and I don't mean that to insult People.

Now Scientific American is just another tiresome rag screaming misleading B.S. about, say, string theory, and (it practically doesn't need to be said) fully on-board with the "global warming" hysteria, smearing critics and misleading and brainwashing its readership. Of course, in our post-modern, post-literate age, does "readership" even make sense? Its writers consist of few scientists any more. That era seems to be gone for good.

** The 1986 Challenger Space Shuttle explosion was a very telling episode illustrating the difference between what the engineers knew from first-hand experience and what the higher, non-technical management at NASA found expedient to believe about their spaceship.

Hysteria, regression, and amnesia

A politically- and journalistically-induced craze like "global warming" has its costs. The most obvious is a poisoned politics overburdened with pseudocrises. The potential economic cost is also clear: implementing the Kyoto Accord reductions of CO2 emissions would mean dismantling a significant chunk of modern civilization.

A spiritual cost. But there's another cost as well, more elusive but very real, imposed on our minds, both the general public and the scientists who've been arm-twisted and panicked by the alliance of politicians, Official Science, environmentalists, and the media - not scientists now or ever - that drives this fake crisis. For the general public, the price is the decline of scientific literacy - the "global warming" hysteria is both of result and a further cause of this decline. As we live in a civilization based at every turn on advanced technology and scientific discoveries, this is worrisome. Another, related worrisome trend is the decaying influence of education, which more and more is challenged by the "junk-food" alternative: information in place of knowledge, tidy and conveniently televised moralistic and narrative fallacies left and right replacing cause-and-effect thinking, journalism in place of books, and misleading fabrications and half-truths in place of first-hand knowledge and personal experience.

In the scientific world, the price of "consensus" science officially imposed is, self-censorship among scientists who don't want the hassle and personal vilification that result from defying the false official thinking. Add to that the shifting of research funding and acceptance by scientific journals away from scientifically sound work toward "para-science" - politically-motivated hack work that doesn't ask questions and look for answers, but takes predetermined answers and policies - "something must be done!" - for granted, then fills in the "back story" to rationalize whatever policies have already been chosen anyway. This sort of thing is something, but it's not science or research.

The price extends into technical details as well, as the previous posting demonstrated. It's the use of bad techniques - methods inappropriate to the task, methods disproved, inadequate methods superseded by better ones - in a rush to a predetermined conclusion in support of "narratives" and policies already chosen by politicians and the media. From the point of view of a scientist, this is especially depressing; it puts the cart before the horse. It would be like doctors abandoning modern antisepsis and surgical techniques and taking up bloodletting again, if some fringe environmentalist group decided that rubber gloves and surgical scalpels were evil talismans of modern civilization and the media then decided, in unison, to scare everyone into accepting it.

In the case of the study of Earth's climate, the large looming discoveries of the last 50 years - nonlinearity, chaos, the non-Gaussian (non-normal or non-bell curve) statistics of events, the limits of prediction, the difficult of defining what we even mean by words like "climate," the fragmentary but still impressive advances made in filling in the details of paleoclimate and the Ice Ages - have been largely ignored, because (a) they're often hard to fully grasp, at least at first; and (b) they uniformly conflict with the "global warming" agenda, either by directly contradicting it or by clarifying how hard it is to predict the weather in the future. The "global warming" agenda can only be accepted by "unknowing" what modern science has learned about climate and dynamical systems and regressing to more naive and poorly informed positions.

We forget and forget and ... huh? So finally, we end with amnesia - a weird disconnect between what scientists and educated laypeople already know (the long-term weather can't be predicted, say) and the unfounded but aggressive confidence that everyone has been bullied into on this subject. But it also shows up in the purely political aspects of the situation. The Kyoto Accord on reducing CO2 emissions was agreed to in 1997 and initialed by President Clinton. The U.S. Senate then rejected the accord in a virtually unanimous vote. Their motives were largely about the economic cost, a perfectly legitimate concern, and one that Clinton himself was fully aware of - the White House expected the Senate to reject it. It was never formally submitted for ratification, in fact.

Ah, but not to listen to our allegedly omniscient, but in reality, amnesia-inducing news media. In their frequent but largely false retelling of the story, the Kyoto Accord was agreed to by "everyone" (that is, by themselves), until the evil W. undermined it by withdrawing from it. Except the U.S. was never in it. All W. did was to revoke the presidential initialing of it. Furthermore, no country that signed the Accord has come anywhere close to meeting its reduction targets, and, unless it wants to dismantled modern civilization within its borders, no country ever will. The Kyoto Treaty is an example of the corruption of our politics by manufactured crisis, accompanied by the 24/7 drumbeat of media hysteria. It is to the great credit of the U.S. Senate (a body whose seriousness I frequently wonder about) that they rejected it - they took it seriously enough to see how fantastically absurd the Accord was and rejected it. They could have, after all, cynically accepted it anyway, knowing full well that the U.S. would never meet the targets, but indulging in a moment of feel-good self-righteousness. That is evidently what happened in many other countries.

Recently, Glenn Reynolds of Instapundit laid out another case of journalistic malfeasance committed in connection with Kyoto by the Associated Press: read his comparison of history with the AP's garbled pseudo-history here. As Reynolds rightly says, "You have to wonder ... why people bother to listen to the Associated Press when it can't get basic bits of recent history right."

Climate science the right way: An example of the Sun-climate connection

Many people have noted and investigated the connection between the Sun and the Earth's climate. Since the late 19th century, most of the scrutiny has focused on the Sun's roughly 11-year magnetic cycle.* Its connection to the Earth's climate is small but significant, important enough to be one of the prime determinants of Earth climate during the current interglacial period. We'll return to this solar influence later, as it's not fully understood.

But there is another influence. The Sun "flickers," over its whole radiation frequency range, on time scales running from days up to a few years. (The technical name is total solar irradiance (TSI) fluctuations.) This flickering leaves an "imprint" on the Earth's climate which will show up in many measured time-series indexes. Because the Earth's climate is nonlinear, we should expect subharmonics to form, at lower frequencies or longer periods - months to decades. The IPCC and many scientists tend to dismiss such intermittency as noise - as if chaos had never been discovered and thrown into question whether "noise" even exists at all.

A newly published empirical study of the Sun-Earth climate connection demonstrates that indeed this does happen. Not only does it happen, this flickering alone explains roughly half of the variation of a "global temperature" statistical index over the last 60 years.** The authors, Scafetta and West, summarize their work here (PDF). They extended it back over the last four centuries by combining the shorter-term flickering with the 11-year solar magnetic cycle variations and explained roughly three-quarters of the variations of the "global temperature" index.

Scafetta and West's work is a beautiful piece of science done right. They identified the correlation between the flickering signal and the temperature index signal in an airtight way, checking their result by randomly scrambling the data's time ordering to see if their result would change. It didn't. As they point out, this result indicates that, while the coupling between the "space weather" caused by solar changes and the Earth weather is weak from the point of view of energy transferred, it does conserve information: the same structure of events and times in one signal shows up in the other, like a faint echo mimicking some distinctive signal.

"Consensus" science fails again. So what gives in the world of "official" climate science? While the scientific reports of the IPCC do acknowledge that the Sun-climate connection is important, they waffle on its exact nature and significance. The IPCC summary reports arrogantly dismiss it altogether. Previous attempts to nail it down quantitatively came up with ambiguous results. But Scafetta and West's methods show why: those negative results came from statistical techniques based on Gaussian "Mediocristan" methods and assume the central limit theorem. We've seen how it can and does break down; chaos and Extremistan behavior is all around us. It's taken science and mathematics several centuries to be able to cope with such phenomena, but modern methods up to the task are available.†

Why aren't they being used by the IPCC? Beats me. But one obvious result of the politically-driven, journalistically-obsessive climate hysteria is junk results based on unphysical concepts and bad methods - methods and concepts that have been proven wrong, don't make sense, or have been superseded. This cost of climate hysteria - the cost of bad theory, scientific regression, and intellectual corruption - is a serious topic for another day, one that deserves its own consideration.

My only real beef with Scafetta and West's summary of their technical results is that Physics Today published it as "opinion." It's not: it's real science. What can be said about the IPCC and its executive summaries is another matter.

---

In a short while, we'll see other, similar, but older results that seem to get up to 80% or so of the temperature index variation by comparing it to the Sun's variability. I'm willing to bet money on the proposition that, with all these solar influences taken into account and combined with the Earth's internal climate cycles, all or almost all (90% or more) of the variability can be correlated, not just back four centuries, but all the way back to the end of the last Ice Age.††

POSTSCRIPT: Here's another nail in the "global warming" coffin, from NPR: the case of the missing ocean heat that "should" be there. (Hat tip to Instapundit.)

As the coffin nails pile up in the case of "global warming," some are even talking about the pending collapse of the whole climate scare: see this from The Australian.
---
* Now you know enough to understand that "roughly periodic" means its Fourier spectrum isn't peaked at one frequency, but has a well-defined, but not infinitely narrow, peak at around 1/11 inverse years of frequency. The "flickering" is a broader, flatter part of the spectrum at higher frequencies running from inverse months to inverse days.

The solar irradiance is itself modulated by strongly non-linear processes near the Sun's surface, including the Sun's magnetism. Some of these processes themselves seem to be chaotic. So we have one chaotic system (the Sun's surface and atmosphere) coupled to another (the Earth's climate), the former "pumping" the latter with energy and leaving behind distinctive "information fingerprints."

** Such an average is physically meaningless in and of itself. But all points on the Earth's surface illuminated by the Sun experience this flickering, and the effect under discussion here necessarily has to show up in some way in any statistical index built up from local temperature measurements. The index used doesn't have to have a physical meaning.

† Scafetta and West state their result in terms of the statistical distribution P(t) of the time t between "events" (flickerings), which is different from, but equivalent to, the chaos and non-Gaussian discussions earlier. They find P(t) ~ (λ/t)^α, with α about 2.1 to 2.2.

In the Gaussian case - which would hold if, for example, the Earth were a closed thermodynamic system at a single temperature - this distribution would be exponential, P(t) ~ exp(-t/λ), with some relaxation time λ characteristic of the whole system. This is roughly the time that it takes a system to "settle down" after a single external disturbance.

Power-law, instead of exponential, decay in time of a disturbance is a different sort of "long tail" phenomenon. It means the disturbance takes much longer to really disappear than you might naively expect. Such behavior has been seen in laboratory measurements of controlled systems since the 1930s. Recall that a chaotic system is one that never settles down. The disturbances never stop; one leads to another, and so on.

†† I'm excluding the exactly periodic daily and annual cycles. It's sometimes good to state the obvious.

Fat tails and outliers: A closer look

No, it's not about the Fat Tonys of the world, Taleb's proverbial cabdrivers who know at least as much about events as so-called experts, not because they're so smart, but because the so-called experts know far less than they think. But Fat Tony might appreciate the world of "fat-tailed" probability distributions, since they provide the mathematical way of capturing, in part, the phenomenon of the black swan: why large deviations from the mean ("outliers") are less common than small ones, but still much more common than expected on the basis of the normal or Gaussian bell-curve distribution.

The Gaussian distribution is used so much because of an important mathematical result, the Central Limit Theorem (CLT). It states that, if we consider a large number of instances of a random process, the collective "distribution of distributions" is Gaussian, if certain conditions hold. These conditions are that:

• The individual instances making up the distribution must be independent of one another.
• The moments, or weighted averages, of the original probability distribution must be finite.

What happens in the "large numbers" limit, if these conditions hold, is that, of all the moments of the original distribution, only three matter after the dust settles - the total population size, the mean, and the variance (the zeroth, first, and second moments - see below). All the other moments either vanish or are controlled by the first three. These three are exactly the ones needed to define a Gaussian bell curve.

A simple example. Let's consider a population of particular instances of some property or attribute, quantified by a random variable x, allowed to range from -∞ to +∞. Its probability density is f(x); within an infinitesimal range dx, the total number of instances between x and x+dx is f(x) dx. The cumulative number of all instances of x < X is the integral of f(x) from -∞ to X. Define the nth moment (or weighted area under the curve) as M(n) = ∫ xⁿ f(x) dx. The non-negative integer n = 0, 1, 2, ... ∞.

The Gaussian with zero mean and variance of one is f(x) = exp(-x²/2)/√(2π). (The normalization is chosen such that M(0) = 1.) It is strongly peaked at x = 0 (the mean) and falls off rapidly for deviations from the mean.

The "fat tail" case occurs when, whatever f(x) is doing for small x, it decreases for large x as |x|^-a, a > 0, apart from overall multiplicative constants. f(x) falls off for large x, but far more slowly than the Gaussian does. Then M(n) ~ ∫ |x|^n-a dx. Replace the upper (lower) limit of the integral with +X (-X), X → +∞. Then M(n) ~ X^n-a+1. There are three possibilities:

• n - a + 1 < 0. The moment M(n) is defined (convergent or finite).
• n - a + 1 = 0. The moment M(n) is infinite, diverging logarithmically.
• n - a + 1 > 0. The moment M(n) is infinite, diverging as a positive power.

For a "fat-tailed" distribution behaving this way, while some moments (for lower n) might be defined, the remaining moments n > a - 1 are undefined. Therefore the CLT does not hold, and it is not correct to use Gaussian-based statistical methods for such populations.*

Long before Fat Tony.... Such distributions are called, in the mathematical literature, Lévy flights, after the French mathematician Paul Lévy, who first worked with them in the decade prior to the Second World War. Mandelbrot, the geometer of fractals, was a student of Lévy. Both Lévy and Mandelbrot went into hiding after the French defeat in 1940, avoiding the Nazi and Vichy dragnet of French Jews.

After the war, they were also intellectual refugees from a certain style of mathematics that swept over the French academic world and had a strong influence elsewhere. Collectively named the Bourbaki school, it drove applied and "heuristic" mathematics to the margins of the field and favored a lean, abstract approach of theorem-proof, with no pictures, diagrams, or applications. (It was the same period that the artistic avant-garde moved strongly in the same direction: away from sense perception, toward "pure" abstraction.) The situation relaxed in the 1970s and 1980s, followed by a strong revival of interest in applied mathematics both among mathematicians and scientists and engineers who use mathematics. While rigor and precision are essential to mathematics, it can't survive or even make sense without contact with applied problems and the world of the senses, and the Bourbaki revolution petered out.

Using Lévy's results, Russian mathematicians Gnedenko and Kolmogorov proved a generalization of the Central Limit Theorem that allows for systematic statistical methods to be applied even in such Extremistan cases. But the resulting "distribution of distributions" is not Gaussian. If we want to study the statistics of events in a chaotic system, like the climate or financial markets, say, we must use these generalized methods pioneered by Lévy, not the 19th-century methods of binomials, Poisson, and Gauss. Like 20th-century artistic palettes and musical styles, it's a 20th-century statistics cookbook of expanded possibilities and greater generality. In the next posting, we'll meet a recent climate case where appropriate statistical methods were applied, with striking results, to a situation where wrong methods were long used.
---
* Usually, a > 1 in practice. If 0 < a < 1, then even the zeroth moment M(0), the total number in the population, is infinite. (The mean and variance are undefined as well.) Mathematicians can still cope with cases where some or all the moments diverge, by using something called the generating function of the probability distribution.

A shaggy Ark story? Plus: some Mamet

Tudor Parfitt, professor at University of London's School of Oriental and African Studies, might have discovered, not the Ark (aron ha-qodesh), but a copy built about 700-800 years ago by a south-central African people called the Lemba, who claim to be one of the Ten Lost Tribes. I know, I know: they're everywhere: the Pashtuns, the North American Indians, etc. But follow Parfitt's story and see for yourself.

His two-hour program aired on the History Channel on March 2 and will re-air later this month. The Lemba do share a genetic marker distinctive to the Israelite priestly clan, the kohanim, descendants of Aaron, Moses' older bro.

And here's a political thought or two from playwright David Mamet at - oy! - the Village Voice. It's stirred up eye-rubbing across the blogosphere and conventional media. Thus Daniel Henninger of the Wall Street Journal provides commentary upon Mamet - yea, verily.

"When oy! meets girl"

Come on, admit it: you rushed out to get your copy of this year's Sports Illustrated special swimsuit issue. It features Israel's top super model, Bar Refaeli. So don't miss it :)

Meet the thinkers: The curious aviary of Dr. Taleb

We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns - the ones we don't know we don't know.

- Donald Rumsfeld

Some of us been waiting for something like this book for a long time, and its Levantine author has come a long way - all the way from the hills of northern Lebanon and the Syro-Greek Orthodox town of Amyoun. The book is The Black Swan: The Impact of the Highly Improbable, and the author, Nassim Nicholas Taleb, former financial trader and now extraordinary professor of the inexact sciences at the University of Massachusetts, Amherst, etc., etc. - essentially, the Dean's pet, and they don't know where to put him. The Black Swan is one of the most important science books for a non-science audience in many years. Like the best chaos and complexity books of a decade or two ago, The Black Swan deals with scientific questions arising from the stuff of everyday life, not far-off galaxies and times long ago.

The core of Taleb's point is the impact of what we don't know, the improbable, and how "randomness" is really another name for our ignorance. But Taleb has a larger target: a whole book was needed to attack and dismantle the legitimacy of bell curve statistics, "Mediocristan" methods wrongly applied to "Extremistan," and explain why so much of the world doesn't follow the "middle of the road" behavior prescribed by the Gaussian-normal distribution and its cousins, such as the binomial or Poisson distributions.

The book is rich with fallacies exploded:

• The Ludic Fallacy. This is the fallacy we pick up when we learn probability based on tightly constrained assumptions, "rule of the game," that make understanding statistical methods based on them as easy as an elementary cookbook. (Ludus is Latin for "game" or "fun.") Real life often presents us with situations of limited knowledge, where probabilistic thinking is appropriate, but where we don't know the "rules of the game," at least not all of them. Many trained in probability and statistics apply the cookbook methods anyway, for the lack of anything better. They capture risk - the known unknowns - but not true uncertainty - the unknown unknowns.
  
  
• The Narrative Fallacy. This is a biggie, practiced on an industrial scale by the news media, every day. We draw connections between dots where the real connections are different, or don't exist, or there are no dots to be found. The news media does it to keep our attention with frequently made-up stories, or "narratives," to use the post-modern jargon, that seem better than no story, or a different one.
  
  
• The Narrative Fallacy supports a related fallacy, one of Misattributed or Reified Intentionality, the fallacy that human society is collectively a result of human intentions or consciousness. In fact, most of it is not, and attempts to force it to be so have led to one disaster after another. Our minds are too limited and possess too narrow a scope of awareness to make this possible. Human society is mostly made behind our backs, so to speak. Taleb's developed views on this question end up very close to the views of the famous Austrian school of economics and sociology.

Taleb has an outrageously funny time explaining what went wrong with statistics and the social sciences in the 19th century, when they were invaded by the concept of the Average Man, and everything was reduced to bell curves, means, and small variations.* All this would hold if our world were Mediocristan. But much of our world is not.

Who's Stan, and what's the difference? Mediocristan is tightly constrained by "fixed totals," or what physicists call "conservation laws." We've already met these and seen what they do. They force the collective behavior into highly restricted patterns, with "equipartitions" of energy, or number, or volume. This certainly is an aspect of our world, and not just in thermodynamics. Heights and weight, for example, both of them strongly limited by gravity and metabolic limits, are distributed in a way close to the bell curve. But then again, consider the distribution of weights in aquatic animals, and you can already see: without gravity, the maximum size is much bigger (think of whales and octopi).

The key to Mediocristan is the Central Limit Theorem. If a population's distribution (of whatever attribute) is made up of independent instances and has well-defined moments (weightings), then the distribution approaches the bell curve in the limit of "large numbers." The presence of "fixed totals" guarantees well-defined distribution weights (moments).

But in many, perhaps the majority of, cases, it fails. The instances are not independent of one another, not distributed with well-defined weights, or neither. The distribution then has much less reason to clump near the mean. In fact, in such cases, many of our usual statistical clichés (means, variances, medians, etc.) fail to capture what's going on.

This is the world Taleb calls Extremistan.** If there's no "fixed total" of something being distributed (like economic wealth, or the total number of books sold by a single author, say), there's no reason to think that the total will be broken up in a roughly even way among instances. Here is the key to understanding much of our world - economic markets, wealth, and income in particular. Many days on markets are boring. Some are interesting. A few are extraordinary - and it these days, the black swans of the financial world, that end up dominating the cumulative history of the market. Just look at the last few months' newspapers.

We encounter similar truths in biological evolution, in contrast to the anodyne but wrong gradualism still dominantly taught. Most of the cumulative change in biological evolution is due to a small number of extraordinary turns of events that have outsized impacts echoing through the millennia. Ditto for human history.

And of course, on Taleb's home ground of financial markets, the reality of black swans, and fractal or fat-tailed distributions, is of intense interest. The disastrous application of bell curve-based statistical methods to quantitative finance in the last generation has not made markets better-behaved or investment strategies sounder. On the contrary: the 1998 Long Term Capital Management and 2008 mortgage crises make clear just how wrong these methods are. They're "state-of-the-art" in some sociological sense, but it's a mistake to call them an art, much less a science.

We've met these strange birds already: Taleb's black swans are the stream of unique events of chaos. His grey swans are those occasional, semi-tamable events at the low frequency end of the spectrum.

Plato in Nerdistan. As the book develops in its middle, Taleb wanders through the thickets of epistemology, how we know what we know. This part is somewhat weaker than the book's earlier and last parts, because the argument goes too far afield and loses a bit of focus. Taleb over-blurs the distinction between event (his specialty) and entity. Before European explorers reached Australia, they believed that all swans are white. The whiteness was not an essential part of the definition of "swan," nor was the belief obviously false. It was a contingent statement about two different properties of things: "swanness" and "whiteness." This supposed connection met its end when the explorers encountered the black swans of Australia. A deeper lesson took a longer to sink in, and some still resist it: disproving something is much easier than proving it. Proving something requires understanding its nature more deeply and thoroughly than our knowledge often runs.

Even this middle part is rich with deserving targets. Taleb calls them "Platonified abstractions," the stuff of academic knowledge. They're thrown around confidently by people who don't know what they don't know. This might almost be a definition of nerdity: what you know fits into cut-and-dried abstractions, and you confuse these with the actual world only known to us very imperfectly. Nerds stand in counterpoise to Taleb's foil, the Fat Tonys, the proverbial cabdrivers of the world who know better and who understand that when it comes to Platonicity, you can take it or leave it.

What do you know, and how do you know it? Exact human knowledge is coined under laboratory control or by precise logic. Most of the knowledge we use in everyday life is approximate knowledge in well-defined, if not controlled, conditions. At the edges of what we know is amorphous knowledge, often mixed in with a lot of prejudice and guessing. And if we want more and better knowledge, we face the reality of trade-offs. I can be sure something will happen today, but I don't know its significance. I can also be sure something significant will happen in the next year, but I don't know when.

Modern science is not based on induction, contrary to common belief. It's based on a mixture of hypothesis, deduction, controlled experiment, and controlled mathematics. It's not because scientists are dogmatists that they live by deduction. It's because deduction allows one's reasoning to be kept under precise control, with all the assumptions on the table and the steps clear. Induction (like statistical correlation) can certainly be strongly suggestive of hypotheses, and it's essential for developing logical definitions. But you can't prove anything with it. One counterexample - the black swan - destroys it. Silent evidence is always lurking to upset the induction cart.

The essence of probability. Coping with limited knowledge means falling back on probabilistic arguments, and this is in fact the origin of statistics. Its modern founders (Pascal, Bayes, Laplace, Gauss) all identified probability with a greater or lesser sense of certainty about something, not its frequency. This distinction fueled a great 19th-century debate between Bayesians and frequentists. Until the 1920s, the frequentists had the upper hand. But modern mathematics has abandoned frequentism, except as an approximation in carefully circumscribed situations where the Ludus isn't a Fallacy (like sports or gambling, for example). With frequentism came many long-unexamined false assumptions; for example, that "noise" and "randomness" are "theory-free" concepts. In fact, few things are more loaded down with theoretical assumptions than "randomness," if taken as a metaphysical category. Taking it as a statement about the limits of human knowledge, OTOH, makes it almost a truism. In most cases, the Ludic Fallacy will come back to bite us: we often don't know all of the rules of the game.

Unfortunately, the frequentist approach to statistics is still taught because it's cookbook. Even in situations where a canned approach is not appropriate, a recipe feels comforting, relieving people of having to think. I might even call this the Cookbook Fallacy: having a wrong recipe is better than no recipe. Actually, no recipe is better than a bad one - at least it's honest and doesn't force us into wrong assumptions.

Taleb in his garden. Along with his skeptical empiricism, Taleb exhibits other exquisitely refined scientific tastes, paralleling his capacious gourmand tastes in literature and food. This might seem an affectation, but it points to an important truth.

Richard Feyman, another man of powerful scientific intuition, said: to do good science, you gotta have taste! Science, like the arts, has its forms of kitsch: rules mechanically applied without the imagination and drive for the fully worked-out development, but without indulging in useless repetition. Science is still and will always remain partly an art. To have taste is to avoid weak arguments and rationalizing, and to avoid applying methods and concepts where and when they are not valid. It is to think that, if there's no deep fundamental principle that prevents something, then why not? What would the world look like if it were so? Maybe you lack the imagination to see that that is our world. Taste is seeing that not taking obvious things for granted is a true royal road to discovery. It is paying attention to the silent evidence, to the dog that didn't bark, and to the pious who prayed and drowned anyway: survivor bias.

Taste in science also requires revisiting fundamental issues, ones never completely resolved. The progress of science has solved many problems defined more narrowly. But deep issues remain, even if transformed. Science has its classics and its literature, history, and philosophy; progress doesn't erase their importance. Read them and avoid being a cultural philistine.†

Taleb reminds us that what we don't know can hurt us, and that what we don't know is often more important than what we do. The Black Swan is a fine book. Buy, read, and enjoy it, patiently and slowly. And if nothing else, be charmed by the bittersweet tale of Yevgenia and her unknown masterpiece.
---
* Hayek attacked much the same in his Counterrevolution of Science, laying out the 19th-century origins of Platonified pseudo-knowledge in the social sciences and the pretensions of social planning that often went with it. Plenty of perceptive people, like our old friend Poincaré, resisted this development, this misapplication of inappropriate mathematical methods to society. But the Tyranny of the Cookbook is an unrelenting one.

** Not to be confused with Wackistan. That's where Ahmadinejad lives.

† Taleb uses the German term, Bildungsphilister, just to show, I suppose, that he isn't one.

Be alert to a real affectation, indulging in philosophical problems isolated from anything real. As Taleb points out, most philosophical issues worth bothering with are suggested by something outside philosophy.

Chaos and the problem of long periods

And so, back to the strangely soothing subject of chaos. If it's happened before, it might happen again.

Nonlinear systems exhibit subharmonics of low frequency, so low that the associated behavior happens only occasionally. If we don't know the fundamental dynamics of a system and can't work out its deterministic logic from first principles, then we are left with staring its empirical behavior, over a necessarily finite time. If we see something happen, how do we know it's a unique event or if it will repeat? After all, its period might just be really long. Is the system not just nonlinear, but chaotic as well?

Stated in a more technical way, empirical study of chaotic systems in always limited by the longest observed time scale. The reciprocal of that time scale gives the lowest frequency we can know about. Without being able to observe the system for an infinitely long time means we can never know for sure what's a unique event and what just takes a long time to repeat.

There's no general solution to this problem, given the restrictions on our knowledge just stated. We might try to ascertain the system's fundamental dynamics and so try to answer the question through mathematical deduction. We might try to extend our observing period through paleo-knowledge, indirect use of old proxy evidence that allows us to infer something about the system over time scales longer than human history. Evolutionary biology and paleoclimate are reconstructed through such techniques. All of these play a large role in the study of chaos in real life.

The final odyssey

People are just dying too damn fast :(

Now it's Arthur C. Clarke, science and science fiction writer, and the inventor of geosynchronous orbits and communications satellites. Hal will open the pod bay doors no more.

See here for the February and October 1945 issues of Wireless World, with Clarke's articles on space flight and satellites.

POSTSCRIPT: Glenn Reynolds of Instapundit has a fine tribute to Clarke here. Clarke was the last of the Big Three (Asimov and Heinlein being the other two) whose work filled our young minds with curiosity and big questions all those decades ago. "The world is better for his having lived, and worse for his having died."

Obama the cultist?

PRE-POSTSCRIPT: Obama has now made his speech (see here and here). Mickey Kaus parses and arrives at some conclusions.
-----
More disturbing news about Obama and how he's let other people define him. It just reinforces the point that he's not ready for prime time. There's no indication that Obama shares the inflammatory views of his pastor and the fringe causes associated with his church. But if so, a large question mark hangs over him and his future as a national political figure: why is he associated with such people?*

Is he black enough? Obama remains what he was a few months ago: a questionably "black" candidate - although he wasn't anointed as such until Oprah endorsed him and all the white middle-class liberals rushed out for a look - a political portrait one-quarter or one-third filled in, the rest question marks and blank spaces.

You can see further comment here and here. It's a reminder that much of what passes for "progressive" politics in the West today is a thinly veiled mishmash of hate, conspiracy theories, and juvenile rage. The last thing it represents is progress. It's not about this or that policy; changing those merely leads the hate-mongers to shift their target to something else. Pay attention to the music, not the lyrics. It's the detritus of yesteryear's alliance of the New Left, radical Christian churches, and the mythical "Third World."

Obama has spent most of his adult career with one foot in this fringe swamp of the self- and America-hating left. It is this issue - not the silly rumors about his being a Muslim - that should have been out in front for discussion all along. What's striking is Obama attempts to rationalize his pastor's screeds; these are the feeble excuses of a cult member trying to defend the cult guru.

Off to the races. How serious is Obama about these cult beliefs? Many people have simply assumed that he's fully into it. But there is another, more likely, possibility. Obama's biography looks like a younger Colin Powell: a man of mixed racial background whose life circumvented the main traumas of segregation and desegregation and is comfortable around both white and black culture. Someone in this position has some powerful advantages, but also lives a painful dilemma. One way to resolve it is to play the game, "Are You Black Enough?", and pay your "race dues." That seems to be the case with Obama and the angry Pastor Wright. There's probably an element of cynical calculation involved, but this aspect of Obama's biography also reflects a genuine personal dilemma. For more along these lines, see this by Matthew Yglesias and this by David Bernstein.**

For American liberals, this fiasco is another case of getting burned by their left-leaning political slumming. They get entranced by the slogans, then wake up the next day in a place they didn't expect. The saving grace this time is that they have plenty of forewarning. What's happening now is a striking case of post-facto rationalizing by the supposedly ultrasmart, but in reality very provincial, people with fancy degrees who've been caught - again - with poor judgment and self-deception. It's something they seem good at. It certainly makes all the portentous hairsplitting about, say, Romney and Mormonism, look asinine.

If Clinton wins Pennsylvania (which she probably will), she will have won all the swing states, and all but one (Illinois) of the large states. The Democrats would be crazy not to nominate her. Clinton has a better than even chance at the nomination, one that is growing every day. She is certainly the candidate in a better position to take on McCain. Obama's candidacy is about to enter a terminal tainspin, and by May or June, everyone will be wondering what the fuss was about.†

Paging Senator Obama. Meanwhile, whether Obama will turn out to be a flash in the pan or a national politician of real significance remains open. This election will not decide it. Both he and the voters need another election cycle or two. Certainly, 20+ years of national politics lies open to him. The result hinges on his self-definition, something he hasn't yet taken a stand on. If Obama resolves his dilemma the right way, he can escape the self-ghettoization of identity politics and become a serious national politician. If not, then not.
---
* Mark Steyn asks the same question in a different way here. Curiously, Oprah left this church quite a while ago - which only makes the question sharper.

** If he experienced this dilemma, Colin Powell resolved it by becoming career military.

Probably the main motive for angry cults and cult leaders - like Pastor Wright and Louis Farrakhan - is just this dilemma faced by a formerly oppressed minority moving from ghetto to mainstream. The upside is obvious: the minority is no longer held down by legally restrictions and social prejudices and can participate in society like everyone else. The less-discussed downside is obvious from the emergence of the black underclass that started in the late 1950s and still lingers: it's the embarrassing problem of washing your dirty linen in public. American and Caribbean blacks have been from the start a semi-submerged distinctive subculture with only faint connections to Africa: they're far more New World than Old. The major step toward a more just society started when whites started acknowledging this fact. But when a semi-submerged subculture moves into the mainstream, everything is now on view, both strengths and warts.

A telling fact: angry identity politics cults emerge after the oppressed minority achieves some enhanced freedom and opportunity, not before. The origins of the Nation of Islam are not in the 19th-century South, but the mid-20th-century North - Detroit, to be exact, where founder Elijah Muhammad picked up, among other things, Henry Ford's nasty rantings about the Jews. Similarly, contrary to the fantasies peddled by Wright, he and Obama and his congregation are not poor and oppressed: quite the contrary, they're dizzy with freedom and opportunities even their parents could barely imagine. Living next door to the underclass neighborhoods of Chicago, ringing with drug pusher gunfire, makes clear every day that some of the newly freed are making poor use of their new freedom. Conspiracy theories are attractive to people in such situations - they appear to explain everything and distract everyone from harsh facts.

The case of American Jews demonstrates the point in a different way. Although there have been identity demagogues amongst us (Rabbi Kahane being the best-known), the success and acceptance American Jews have enjoyed in the last 60 years greatly limit their appeal.

† Keeping insufferable television journalists in our faces 24/7? Was that the point?

Happy Saint Patrick's Day!

Or, Evacuation Day here in Boston :)

Toppa the morning to you!

Long life to Sheherazade!

March is Women's History Month and, this year, also features your favorite holiday, Purim. Why not combine the celebrations with an appreciation of that sadly neglected heroine of the medieval Persians, Sheherazade? She was smart enough to survive a thousand nights plus one with that gynocidal king, Shahryar, so reminiscent of another Persian princess more than a millennium earlier.

Speaking of Persian princesses, even if you missed her graphic novels Persepolis and Persepolis 2, Marjane Satrapi's animated film version Persepolis is a must-see. And don't forget her older "sisters," Roya Hakakian and Azar Nafisi, respectively authors of Journey from the Land of No and Reading Lolita in Tehran.

Stuff I like

Here's a list of things that white people are supposed to like. I like # 1, 2, 3, 8 (when he's not being a politician), 38 (yes!), 57, and 78 (having been one myself).

I'm lukewarm to mildly negative about some of the other stuff on this list. So maybe I'm not white, and I've been fooling myself all these years.

(Hat tip to the Modest Pagemaster.)

It's Pi Day

You remember π, no? It's the ratio of a circle's circumference to its diameter. It's an irrational number, as well as transcendental, and thus never repeats its decimal expansion:

π = 3.14159265358979323846264338327950....

Today is 3/14, or Pi Day. Or perhaps, we should call it, National Good-Enough Day, a day for controlled approximations :)

POSTSCRIPT: This is really scary - familiar song, new lyrics (MP3). More here.

Umpteen ways to look at chaos

There are at least three distinct and powerful ways to look at chaos. So far, we've been looking at only one: the aperiodicity path. The other two are the sensitivity-to-initial-conditions ("butterfly effect") path and the statistical path. All three are more or less equivalent, but have greater or lesser degrees of flexibility and generality.

The common way chaos has been discussed, both by scientists and in popular culture, goes by the poetic name of the "butterfly effect," otherwise known technically as "exponential sensitivity to initial conditions." The idea is that a system whose dynamical rules (the rules by which it changes in time) are known is deterministic: if the system's state is known exactly at some time, it can be known exactly for all time, at least in principle.

The catch is that no real system's state is ever known exactly in real life. So determinism in practice means the following, by no means obvious, requirement: approximately knowing the system state means being able to approximately predict its future or reconstruct its past. Chaotic systems severely curtail this possibility, because a small error in knowing the system's state at one time gets blown up exponentially fast into a much larger uncertainty. The error ε(t) a time t later after starting is ε(t) ~ ε(0)·exp(λt). λ is the Lyapunov exponent. It has units of inverse time; its reciprocal 1/λ is the Lyapunov time. In that time after starting, roughly, you can no longer predict the system's behavior. The presence or absence of a butterfly's flapping wings in Brazil could lead or not lead to a hurricane in Florida, or so the saying goes. No mathematical-computational model mimicking the system can compute faster or better than the system itself.

A chaotic system is its own best computer.

Determinism is a powerful concept and fundamental to scientific knowledge. I know the system exactly at an instant. What happens next is fixed. Determinism differs from randomness or stochasticity, where what happens next is one of a set of possibilities, each with a probability attached. Chaos shows how an ordered, deterministic system can look random, and how apparent randomness is really a fantastic sort of order.

While this approach to chaos is valid, it's not as useful as the other two approaches. It requires that you know the system's complete dynamics (even if you can't solve it). If you don't, the frequency-event-Fourier approach, or the statistical way are much more helpful. These other two methods are useful for nonlinear, complex systems whose dynamics is too hard. In the case of weather, the basic equations are known, but are far too complicated to solve. In the case of biological evolution or the financial markets, no one knows more than a fragment of the dynamics in an exact mathematical form.

A statistical look at chaos. To analyze chaos statistically, forget about evolution in time. Just bin "changes" in the system by how often they occur, by size.* Some very interesting distributions emerge, power laws, not the Gaussian bell curve or one of its cousins (say, the Poisson or binomial distributions). Large changes are less probable than smaller ones, but their probability declines slowly, as a low power of the size, not at all the rapid falloff of the Gaussian. This phenomenon sometimes goes by the name of "fat" or "long tails."

The two alternative approaches can be applied in these cases, where the other one fails. As long as degrees of freedom and subsystems can be identified, and a few, quite general assumptions are made, chaos in many systems can be investigated with these methods. They're agnostic about what the fundamental dynamics is. They might even help to "reverse engineer" or reconstruct the dynamics when it is not initially known.**
---
* This is clearly related to the Fourier approach, just interchanging the axes. The "size" of change is related to the "power" (the latter is essentially the square of the former).

** In technical jargon, this is called "reconstructing the phase space" - we'll meet the concept again.