Systemic polling bias
For any given weekend since the last Federal election, it was not uncommon for ACNielsen and Newspoll to have estimates for the Greens that were statistically different. In plain English: at least one of the pollsters was wrong! This can be seen graphically.
This of course raises the question of systemic polling bias. To get a sense of the systemic differences between the pollsters, I have put together the following table. It gives the average poll results from the pollsters since the last election for each of the major party groups.
| Morgan | ACNielsen | Newspoll | Divergence | |
|---|---|---|---|---|
| Coalition | 41.92 | 42.83 | 43.45 | 1.53 |
| Labor | 40.03 | 38.00 | 38.29 | 2.03 |
| Green | 8.43 | 9.89 | 6.42 | 3.47 |
| Other/Minor | 9.61 | 9.56 | 11.84 | 2.23 |
| 2PP Coalition (calculated using 2004 preference flows) | 49.07 | 50.21 | 50.61 | 1.54 |
| 2PP Coalition (from polling) | 48.40 | 48.83 | - | 0.43 |
| 2PP Coalition Divergence | 0.67 | 1.38 | - |
I am not convinced these results are necessarily meaningful. Morgan, for example, tended to report weekly on reduced sample sizes when it got anomalous results, which could skew its results in this analysis. For ACNielsen, I have excluded the results from June 2006, as its minor party primary vote prediction has not been made public yet.
Nonetheless, the results are indicative of a systemic polling bias problem - particularly with the Greens. I suspect this bias comes about from two sources. First, the way in which the pollsters develop their sample frame can bias the result. Second, the actual questions the pollsters ask and the sequence in which they are asked can also bias the prediction.
It is a significant issue, as these Green and minor party results feed into national two-party preferred estimate from each of the pollsters. From 2007, I expect Newspoll will report preference allocations according to the way in which people say they will vote, rather than use the preference flows from the previous election. Similarly, while Morgan reports both, I expect the focus to move to actual reported preference flows.
It is another useful reminder that interpreting polling results is as much an art, as it is a science. And it legitimises the practice of adjusting results from some polling organisations by a percentage point or two; although it is arguable which pollster is accurate and which is biased. For the record, I cannot quite come at the seven-point adjustment one of my regular commentators applies.
Note: in undertaking this analysis, I found an error in my earlier minor party charts in respect of the Morgan series. It has now been corrected. You may need to hit the refresh button on your browser to get the latest chart.