Morgan bias
I copped some flack yesterday for suggesting that the Morgan poll has a two percentage point left bias.
Now we all know there are commentators on this blog who routinely accuse Morgan of substantial biases. Those commentators typically go on to apply a correction formula derived from Colonel Sanders’ 11 secret herbs and spices and then they declare the Morgan poll guarantees the triumphant return of the Howard government for a fifth glorious term. So there can be no mistake: that is not my claim.
Nonetheless, I think there is a left-bias in Morgan that needs to be at least considered when interpreting Morgan polls. It would be reasonable for you to ask: What is the evidence for my claim of a Morgan bias? I have two lines of evidence.
The first is Simon Jackman’s academic research into the polls following the 2004 election. I previously blogged on Simon Jackman’s preliminary research. The preliminary paper no longer seems to be on the web — however, it looks like the research has been finalised and published as “Pooling the Polls Over an Election Campaign”, Australian Journal of Political Science, 2005, V40(4):499-517. There is a related slide presentation here. Jackman found that in the four months prior to the 2004 election Morgan typically under-estimated coalition support by 4.9 percentage points. To be fair to Morgan, the pollster changed its “two-party preferred” methodology following the 2004 election.
Second, notwithstanding Morgan’s changed methodology, since the 2004 election Morgan has typically predicted a Labor primary vote that was around two percentage points higher than ACNielsen and Newspoll for the same period. This tendency is clear in the next graph of the Labor primary vote moving averages for Morgan, Newspoll and ACNielsen.
With the Coalition primary vote predictions, a corresponding approximate two point bias is evident between Morgan and Newspoll. However, The relationship between ACNielsen and the other two is a little more complicated.
The two-party preferred (TPP) vote share predictions are a little more difficult to interpret, compared with the primary vote predictions for the major parties. There are two complicating factors.
First, the pollsters use different methodologies to calculate their TPP predictions. Newspoll and Morgan currently use the same methodology, which is based on the preference flows at the last election. ACNielsen calculates its TPP vote share prediction using responses to the poll questions.
The second problem is the pollsters get consistently divergent primary vote estimates for the non-Labor, non-Coalition parties. This divergence can be seen in the next two moving average graphs.
The consistent divergence in the Green primary vote predictions is of particular concern.
Notwithstanding the complexity of various inputs and methodologies, the outcome is that since 2004 both Morgan and ACNielsen have been tracking to the left of Newspoll with their published TPP vote share predictions. In the following graph I have plotted two moving averages for ACNielsen. The first — labelled “published” — is based on ACNielsen’s published TPP vote share predictions. The second — labelled “adjusted” — is calculated using the preference flows from the last election. When the ACNielsen TPP vote share is adjusted, it tracks closer to Newspoll than Morgan.
I accept it is arguable that the predicted TPP vote share divergence is a rightish bias in Newspoll rather than a leftish bias in Morgan. I prefer the left bias story on the basis of Jackman’s research into the 2004 election. I also prefer the left bias story because it is explainable. I suspect the leftish bias in Morgan is possibly an artifact of Morgan’s cluster sampling frame and the leftish bias in ACNielsen is possibly an artifact of ACNielsen’s TPP calculation methodology,
In conclusion, I think it not unreasonable to at least consider applying something like a two percentage point adjustment to Morgan’s TPP vote share and Labor primary vote predictions when interpreting Morgan poll results. This is consistent with Jackman’s observation that one might apply an a priori correction to polls based on their actual performance in the lead-up to the previous election. In Mogan’s case, that correction should recognise Morgan has changed the way in which it calculates its published TPP vote share prediction.
Note: I have not commented on what biases may or may not be evident in the Galaxy series, largely because I do not have enough polling data on which to form a view. Also note: I have used a three period moving average for ACNielsen, as it is published monthly, and a six period moving average for Newspoll and Morgan, which were being published on a fortnightly basis. This year, Morgan has been published weekly.
Update: Possums Pollytics has also looked at the Morgan bias question.