'Foreign Policy' Counts Liberal Terror Experts As 3/5 Of A Person

Foreign Policy magazine released its third "Terrorism Index" earlier this week, and hardly anybody else noticed. Did you?

An "index" in only the loosest sense of the term, the "Terrorism Index" is a report of a survey. So-called experts were asked questions and their responses were compiled and expressed as percentages.

Various fragments of the "index" are now being released as news items, so you may soon read that

35 percent of the experts believed that Pakistan “is most likely to become the next al-Qaida stronghold,”

or that

a majority (53 percent) said the United States should “selectively engage more moderate leaders [of Hamas] … in an effort to drive a wedge between moderates and extremists.”

Take all these numbers with more than the usual grain of salt.

The survey group consisted of 108 "experts", most of whom have worked for the federal government; the others are "respected" academics. And the questions were standard and naive. So all the general criticisms I leveled against the second Terrorism Index apply to its third incarnation, as well.

Briefly,

The Terror Index is nonpartisan presumably because it includes donkeys as well as elephants; but if 80 percent have served in the government and the rest are distinguished academics, then it can hardly be impartial, can it?

How many whistleblowers does it include? How many academics on the list would rather tell the truth than remain "distinguished"? How many politicians, generals, intelligence analysts, and so on -- whistleblower or distinguished -- come from outside the United States? How many come from outside the mainstream?

[...] Had Foreign Policy seen fit to include real outside experts on global terror [...] their "Terror Index" may have been more useful, because it may have been based on a more sophisticated understanding of the world.

The third Terrorism Index repeats all these mistakes and more. In addition, along with the "terrorism" questions, the experts were asked to position themselves on an "ideological" scale. In response to this question, 25 members of the group identified themselves as conservative, 39 said they were moderate, and the other 44 called themselves liberal.

Foreign Policy could have simply reported that the panel was 23% conservative, 36% moderate and 41% liberal. Instead they chose to "weight" the responses of the liberal experts so that there would appear to be an equal number of liberals and conservatives on the panel. In other words, each liberal expert's opinion was counted as 25/44 of an opinion. That's just a shade less than three-fifths.

No, I am not kidding. You have to read all the way to the bottom of the final page, but this is how the report explains its method:

To ensure balance, the survey was weighted according to ideology to make the number of weighted liberal respondents equal to the number of conservative respondents. Moderate and conservative respondents remained unweighted.

For the record, excerpts from the executive summary of the curiously weighted survey:

A majority of America’s foreign-policy experts now hold a negative view of the White House’s “troop surge” strategy in Iraq, and two thirds support a redeployment of troops in the next 18 months, according to a bipartisan survey produced by Foreign Policy magazine and the Center for American Progress.
...

Of the more than 100 foreign-policy experts (both liberals and conservatives) surveyed, 53 percent now say that the surge is having a negative impact—an increase of 22 percentage points in just the past six months. Nearly all of the experts (92 percent) believe that the war in Iraq is having a negative impact on U.S. national security.

A bipartisan majority (68 percent) now say that the United States should redeploy troops from Iraq in the next 18 months, though most oppose an immediate withdrawal. Surprisingly, more conservatives (25 percent) called for an immediate pullout than liberals or moderates.

Overall, nearly all of the experts (91 percent) say that the world is becoming more dangerous for Americans and report that the country is not winning the war on terror (84 percent). More than 80 percent predict a 9/11-scale terrorist attack on the United States in the next 10 years.

Pakistan was named as the country most likely to become the next al Qaeda stronghold, ahead of Iraq. Seventy-five percent also said that Pakistan—home to A.Q. Kahn’s now infamous nuclear black market ring—was the most likely to transfer nuclear technology to terrorists in the near future.

But when the experts were asked to name the ally that least serves U.S. security interests, Pakistan placed second to Russia, with Moscow’s consistent criticism of the United States, refusal to back tougher sanctions against Iran, and the increasingly authoritarian tendencies of President Vladimir Putin likely weighing on the experts’ minds.

Iran was ranked as the 4th most significant international threat.

I called the questions naive. Here's an example of what I mean:

Nearly all of the experts (92 percent) believe that the war in Iraq is having a negative impact on U.S. national security.

How many of them also believe that this effect is an intentional part of the policy?

Oh, right! They weren't asked.