Oz Politics Blog

I found yesterday’s program on opinion polls quite interesting. As a consequence, I thought I would try to capture my rules of thumb on voter intention polling.

1. Opinion polling has a poor record at predicting election outcomes, even six months out from an election (as in, the polls get it wrong almost as often as they get it right)
   • To be fair, voter intention polling does not seek to predict how the nation will vote at the next election, but how it would have voted were the election were held last weekend. A possible exception is Morgan’s question: Who do you think will win the next election?
   • We know from past election studies that roughly half the electorate reports that it decided how to vote during the election campaign
2. Polls are typically volatile from fortnight to fortnight, but in most cases this volatility is ‘noise’ not ’signal’
   • Much of the electorate is largely disconnected from political processes — particularly between elections — and I suspect those who are disconnected typically respond to voting intention polling questions in terms of their loosely evolving preferences and their likes and dislikes, rather than any clearly formed voting intention
   • Also as noted above, some of the electorate is undecided outside of the election campaign period, and in this context some respondents give close to random answers to the polling organisations
3. All polling organisations have systemic biases inherent in their methodologies — these might be attached to sampling design, engagement modality (eg. face-to-face or telephone), question design, question order, answer prompting, the way in which the two-party preferred result is derived (by direct question or preference flows at the last election), or some other aspect of the polling process.
   • For example, I have noted for some time that the Morgan polls are typically to the left of Newspolls conducted on the same date (or is it that Newspolls are typically to the right of Morgan)
4. Individual opinion polls are not an accurate measure of public opinion
   • The consequence of the above factors (and others) is that the stated statistical error margin for any poll overstates its precision by orders of magnitude — Andrew Leigh and Justin Wolfers have estimated that “the true standard error of the polls is equivalent to a poll of 25 voters that suffered only from sampling error.” This is much more error than would be expected from the 2000 voters in a typical Morgan Poll or the 1400 voters in a typical ACNielsen poll.
5. Therefore, polls must be interpreted
   • Individual opinion polls must be considered in context: within the time series, in comparison with polls from other pollsters, against the backdrop of political events, and against other information sources (eg. betting markets and qualitative polling)
   • I typically reject as noise any significant poll movement that can not be attributed to an event that one would reasonably expect to change voting intention
   • Even if I can point to a plausible explanatory variable, I usually suspend judgment on a single poll result that could mark a discontinuity
   • To put it another way: don’t read too much into a single poll result
   • I look to the medium term trends in the polling, and I usually give precedence to the the moving average over (say) two months, rather than the individual fortnight on fortnight poll results
   • When considering a two-party-preferred (TPP) prediction, I look at how the preference flows were calculated — were they based on a direct question or preference flows at the previous election. If based on a question, I test the plausibility of preferene flows against the range of preference flows from previous elections
   • While I generally accept the direction of an extreme poll trend (as an indicator of the current mood, not as a prediction of the next election result), I usually have doubts about the scale of TPP vote predictions that are outside of the range of election experiences since the second world war (ie. any Federal poll that predicts a winning margin wider than 57-43)
   • A TPP moving average in the 49 to 51 per cent range can be particularly difficult to interpret
6. The media has a vested interest in sensationalising the noise in polling sequences — a dramatic swing to a party one fortnight is headline news, and in the followed fortnight the dramatic swing back to the other party is also headline news
   • My pet aversion in this area is the commentators who judge the Federal Budget by the following weekend’s poll — it is simply crap analysis — most punters do not analyse a Federal Budget between Tuesday and Friday and then adjust their voting intention in time for the pollster on Saturday — for most people, if they do adjust their voting preference, it is when they are negatively impacted by a Budget measure once it has been implemented

Are there other rules of thumb I should mention? Any you disagree with?

Update: an observation from Andrew Norton.

This might be worth listening to. Australia Talks - ABC Radio National - 6-7pm tonight. The spiel from the website follows.

Opinion polls have been a part of political life since the 1960’s, but Prime Minister John Howard has addressed the latest AC Nielsen poll directly by saying he “vows to work harder”. So, do they make any difference to party leadership, and to policy itself? Are they a waste of time? Who do you think drives opinion polls - the public, the pollsters or the politicians?

Today’s Australian reported that a Newspoll over last weekend had the gap between Labor and the Liberals closing in NSW. The headline prediction was a state-wide two-party preferred vote of 56 per cent for labor and 44 per cent for the Coalition. The primary votes were 42 per cent for Labor and 37 for the Coalition. This result is an improvement for the Coalition on the poll conducted over the first two months of 2007. That poll predicted 59 to 41 per cent split in Labor’s favour.

I suspect the preference flows to the Coalition (at 33 per cent) are a little low, and a more accurate TPP prediction would be 55 to 45 per cent. But it is a moot point, as neither 56/44 nor 55/45 would see a significant change to the make-up of the NSW Legislative Assembly.

Were an election held between last Thursday and Saturday, Labor would have received 61 per cent of the national two-party preferred vote. The ACNielsen poll of some 1400 people also predicted that the Coalition would have received 39 per cent. [SMH, Age]

In respect of the primary votes, the poll predicted 35 per cent for the Coalition and 50 per cent for Labor.

I was critical of the recent Morgan poll that predicted 61.5 per cent for Labor. Then I wrote, “Bollocks! There is no way that Labor would get 61.5 per cent of the national two party preferred vote, either last weekend — were an election held then — or in October/November this year when the next election is likely to be held.” I have not changed my view. A 61 per cent two-party preferred vote is implausible come the election.

With today’s ACNielsen poll, it would appear that at least some of the result can be attributed to unusually low preference flows to the Coalition. Typically, the Coalition receives around around 40 per cent of the non-Labor-non-Coalition vote in preferences at an election. In today’s poll it got just 27 per cent. If we adjust preference flows in line with recent elections, today’s result looks more like 59 to 41 per cent in Labor’s favour. Still significantly above any election result since the second world war.

Will the 2007 election be the ‘Ruddslide’ that today’s papers predict? I have answered this question a number of times this year, but I will say it again. It is simply too early to tell. I cannot predict from the polling to date whether 2007 will play out like 2001 or 2004 or not (see below). In Rudd’s favour he is starting the year in a better polling position than either Beazley or Latham. It makes the challenge for Howard in 2007 all the more difficult — but not impossible.

Headlines proclaiming that Howard’s spoiling strategy had failed are a little premature. Howard is playing a medium term game. It is the standard two pronged game: pander to the punters and slam the opposition at every possible turn. The effectiveness of Howard’s medium term strategy cannot be judged after a few short weeks. If previous election years are any guide, It was not until the middle of the year that a recovery trend (from Howard’s perspective) was evident. Howard only achieved positive polling territory from the middle to late in the third quarter of the year.

While I think it too early to call the 2007 election, I would rather be in Rudd’s shoes than Howard’s with polls like these. If Rudd is stll polling high 50s to low 60s in June-July it may well indeed be a Ruddslide.

The usual opinion poll graphs are here. You may need to hit the refresh or reload button on your browser to see the latest graphs.

2007 Federal Election

The average probability from the five bookies for a Coalition government following the 2007 election is 52.0 per cent. It was 51.7 per cent on Monday.

Bennelong

Perhaps the most interesting odds have been Sportingbet’s on who will win the seat of Bennelong. In just under two weeks, Maxine McKew has been dropped by the punters.

Centrebet also has a book on Bennelong. However, rather than focus on the parties, it offers odds on the players: $1.18 for Howard, $4.50 for McKew and $14.00 for anyone else. Centrebet’s odds on anyone else are interesting, as they allow for the prospect of Howard or McKew withdrawing from the race.

NSW Election

The odds for the NSW Election on 24 March have not moved significantly over the campaign period. According to Centrebet, Labor has an 82 per cent probability of taking Government.

Centrebet has Labor the favourite in Wollondilly, Tweed, Port Stephens, Penrith, Murray-Darling (just), Monaro, Kiama, Gosford, Drummoyne, Camden and Balmain. The Coalition is the favourite in Terrigal, Tamworth (just), Pittwater and Bega. And an independent is the favourite in Manly (just) and Dubbo.

SportingBet is running a book on how many seats labor will win in the NSW Legislative Assembly. The favourite range is 45-50 seats, closely followed by 50-55 seats.

The usual graphs are here. You may need to hit the refresh or reload button on your browser to see the latest graphs.