The Conservative Cat

This afternoon Matt Drudge had a link to this article in the New York Times, entitled Some Politics May Be Etched in the Genes. It describes a study of political party affiliation and ideology, and concludes that ideology (conservative or progressive) is genetically influenced while party affiliation is influenced by one's upbringing. When Bruce read the article to me, the following sentence sent my ears back:

In the study, three political scientists - Dr. John Hibbing of the University of Nebraska, Dr. John R. Alford of Rice University and Dr. Carolyn L. Funk of Virginia Commonwealth - combed survey data from two large continuing studies including more than 8,000 sets of twins.

The fact that the three scientists extracted survey data from two ongoing twin studies means that they are using a technique called meta-analysis: in other words, they did not conduct a study, but a study of studies.

Meta-analysis was originally developed as a way to synthesize information from large sets of research results. In recent years, however, it has been used improperly to prove that ESP phenomena are real. Since only two studies were used, it should immediately sound an alarm bell in your brain. In actual fact, the paper's results were based entirely on a single study called the Virginia 30K Project; the second study, which took place 20 years ago in Australia, was used as a sort of control. Unfortunately, very few of the questions in the Australian study matched those in the Virginia project, which made it a lousy control. So, right off the bat, we should understand that this is a tested hypothesis but not a theory. The attempt to replicate the result (which is the heart of the scientific method) was hampered by the fact the researchers couldn't go out and do a real twin study. They had to make do with what they could dig up from the literature.

Analyzing people's political opinions is a tricky business. It is well known that opinion polls are easily-- sometimes even unintentionally-- biased. There is an excellent example of the pitfalls of basing political affiliations on surveys at the Political Compass Project. The graph points cluster around the diagonal in the liberal quadrant, showing a strong correlation between social and economic liberalism. On the economically conservative side, however, the points are all over the map. That's because the questions that determine social conservatism were designed by a liberal, and are more appropriate to the way conservatives are portrayed on television than on the way they think in the real world.

In this case, the study data was designed for a purpose completely unrelated to the analysis of political leanings, so none of the normal safeguards against survey bias could have been applied. This is perhaps acceptable when analyzing behavior patterns such as sexual orientation or affection for country music, which tend to remain constant over one's lifetime, but it is very different to use it for measuring attitudes, which can be influenced greatly by recent experiences (such as the last few questions asked during the survey).

But there's more: here's what the New York Times has to say about how the study was conducted.

From an extensive battery of surveys on personality traits, religious beliefs and other psychological factors, the researchers selected 28 questions most relevant to political behavior. The questions asked people "to please indicate whether or not you agree with each topic," or are uncertain on issues like property taxes, capitalism, unions and X-rated movies. Most of the twins had a mixture of conservative and progressive views. But over all, they leaned slightly one way or the other.

It is worth noting that in the actual journal paper the authors claim they used "28 items" for determining political orientation plus "a select set of additional items" to verify the methodology. The table in their paper, however, only shows 28 items (none of the additional ones). One of the items is Astrology, which doesn't seem to me to have very much to do with politics, and another is Segregation, a philosophy that only appeals to lunatics. The responses were totaled for each person, with a conservative response counting as +1, a liberal response as -1, and a neutral response as 0. The paper states that the median result was +3, but doesn't give us any idea as to whether the responses were spread evenly across the spectrum or lumped in the middle. If most people are clustered around the middle (as is implied by the New York Times article), then the assignment of ideological affiliation is mostly a matter of statistical error and the whole study is worthless.

Finally, twin studies never show that a behavior pattern is wholly genetic or wholly envrionmental. For example, opinions on Abortion show a 64% correlation among identical twins and a 52% correlation among fraternal twins: the opinions in the home are clearly having an effect, even if it is less than the effect of the DNA. To their credit, the authors of the paper are very up-front about this aspect of the study, even though the newspaper account ignores it entirely.

That last point is the important one: it's the difference between science news and science.

Respectfully submitted,

Ferdinand T. Cat