Oz Politics Blog

If I have it right, Newspoll is out on Tuesday, there is every chance that Morgan will publish a one weekend poll on Friday, and ACNielsen is out next Monday. When the polls come out, the papers will either say there has been a post Budget bounce for the government or there has not. Both headlines would be fatuous. Both analyses would be flawed.

It is nigh on impossible to assess a Federal Budget by a single poll on the following weekend. Anybody who attempts it either does not understand voter intention polling, or is deliberately setting out to deceive you. My advice is: don’t get sucked in. Don’t fall for the pop-psephology that will attribute a move (or the absence of a move) from one poll to the next to this event or that. It is rarely true.

To begin with, most voters do not analyse a Federal Budget between Tuesday night and Friday and then adjust their voting intention in time for the pollster on Saturday. The reality is that most voters are disconnected from the political process most of the time. Budget week is no different.

The real problem with voting intention polls are the stochastic perturbations (random noise) associated with polling. In practice, voting intention polls have a much larger margin of error than that predicted by statistical theory. For error purposes, a sample of 1000 voters would be better treated as if it were only a sample of 100 people and perhaps smaller. Andrew Leigh and Justin Wolfers have estimated that “the true standard error of the polls is equivalent to a poll of 25 voters that suffered only from sampling error”. As a consequence, any single poll result is fairly likely to fall almost randomly within plus or minus three percentage points of the actual population parameter.

With a high noise factor and a low underlying trend, if you take all possible consecutive poll findings, according to their probability of occurrence, it is entirely possible that the random noise would suggest a trend that does not exist in the population. Take, for example, the following Newspoll data from 2001 from March to immediately before the Tampa episode at the end of August. The trend over the period was a fall of 0.4 percentage points per fortnight in Labor’s two party preferred vote prediction. Of the 13 data pairs during the period, only four pairs had a declining trend of one per cent. Four of the data pairs suggested a dramatic improvement in Labor’s fortunes and five suggested a dramatic decline. None of the data pairs suggested the most accurate fortnightly trend prediction of zero (when you consider that Newspoll publishes it results in integer percentage points).

In summary, none of the consecutive data pairs gave the most accurate trend result possible. Only 30 per cent of the data pairs for this period gave a reasonable prediction of Labor’s fortunes. But similarly, 30 per cent of the pairs yielded a trend line in the opposite direction to the underlying trend in the population. 40 per cent of pairs over-estimated Labor’s demise in 2001. All up. 70 per cent of the apparent trends from the consecutive data pairs were misleading.

Whatever the papers say about a post Budget bounce, don’t believe it. The media pundits cannot reliably interpret two consecutive polls when the underlying population trends are likely to be small, and the random noise range associated with the polls is a number of multiples of the underlying trend. No-one can.

Please note: I am not arguing that the post Budget polls would be meaningless. All I am arguing is that individual voter intention poll results must be considered in context: within the time series, in comparison with polls from other pollsters, against the backdrop of political events, and against other information sources (eg. betting markets and qualitative polling). You will need a number of polls following the Budget to come to an informed view about how the government is travelling post budget, and whether that is different from how it was traveling prior to the budget. Unfortunately, you only need one post-Budget poll to come to an uninformed view on the efficacy of the Budget.

To counter the influence of random noise, I tend to place more faith in the moving average, than any particular poll result. The six poll moving average I use with Newspoll and Morgan, and the three poll average I use for ACNielsen, are designed to cancel out some of the random noise. It is not a perfect solution, but it is far better than over interpreting an individual poll result.

Two of the three moving averages suggest that the trend to Rudd since December 2006 has come to an end. It is impossible to tell whether the polls will plateau at this point, guaranteeing a Labor landslide; move slowly back to the government, giving Labor a narrow to comfortable victory; or move decisively back to the government and a fifth Howard term.

Links: Previously I have posted on my rules of thumb for interpreting polls.