Oz Politics Blog

I found yesterday’s program on opinion polls quite interesting. As a consequence, I thought I would try to capture my rules of thumb on voter intention polling.

1. Opinion polling has a poor record at predicting election outcomes, even six months out from an election (as in, the polls get it wrong almost as often as they get it right)
   • To be fair, voter intention polling does not seek to predict how the nation will vote at the next election, but how it would have voted were the election were held last weekend. A possible exception is Morgan’s question: Who do you think will win the next election?
   • We know from past election studies that roughly half the electorate reports that it decided how to vote during the election campaign
2. Polls are typically volatile from fortnight to fortnight, but in most cases this volatility is ‘noise’ not ’signal’
   • Much of the electorate is largely disconnected from political processes — particularly between elections — and I suspect those who are disconnected typically respond to voting intention polling questions in terms of their loosely evolving preferences and their likes and dislikes, rather than any clearly formed voting intention
   • Also as noted above, some of the electorate is undecided outside of the election campaign period, and in this context some respondents give close to random answers to the polling organisations
3. All polling organisations have systemic biases inherent in their methodologies — these might be attached to sampling design, engagement modality (eg. face-to-face or telephone), question design, question order, answer prompting, the way in which the two-party preferred result is derived (by direct question or preference flows at the last election), or some other aspect of the polling process.
   • For example, I have noted for some time that the Morgan polls are typically to the left of Newspolls conducted on the same date (or is it that Newspolls are typically to the right of Morgan)
4. Individual opinion polls are not an accurate measure of public opinion
   • The consequence of the above factors (and others) is that the stated statistical error margin for any poll overstates its precision by orders of magnitude — Andrew Leigh and Justin Wolfers have estimated that “the true standard error of the polls is equivalent to a poll of 25 voters that suffered only from sampling error.” This is much more error than would be expected from the 2000 voters in a typical Morgan Poll or the 1400 voters in a typical ACNielsen poll.
5. Therefore, polls must be interpreted
   • Individual opinion polls must be considered in context: within the time series, in comparison with polls from other pollsters, against the backdrop of political events, and against other information sources (eg. betting markets and qualitative polling)
   • I typically reject as noise any significant poll movement that can not be attributed to an event that one would reasonably expect to change voting intention
   • Even if I can point to a plausible explanatory variable, I usually suspend judgment on a single poll result that could mark a discontinuity
   • To put it another way: don’t read too much into a single poll result
   • I look to the medium term trends in the polling, and I usually give precedence to the the moving average over (say) two months, rather than the individual fortnight on fortnight poll results
   • When considering a two-party-preferred (TPP) prediction, I look at how the preference flows were calculated — were they based on a direct question or preference flows at the previous election. If based on a question, I test the plausibility of preferene flows against the range of preference flows from previous elections
   • While I generally accept the direction of an extreme poll trend (as an indicator of the current mood, not as a prediction of the next election result), I usually have doubts about the scale of TPP vote predictions that are outside of the range of election experiences since the second world war (ie. any Federal poll that predicts a winning margin wider than 57-43)
   • A TPP moving average in the 49 to 51 per cent range can be particularly difficult to interpret
6. The media has a vested interest in sensationalising the noise in polling sequences — a dramatic swing to a party one fortnight is headline news, and in the followed fortnight the dramatic swing back to the other party is also headline news
   • My pet aversion in this area is the commentators who judge the Federal Budget by the following weekend’s poll — it is simply crap analysis — most punters do not analyse a Federal Budget between Tuesday and Friday and then adjust their voting intention in time for the pollster on Saturday — for most people, if they do adjust their voting preference, it is when they are negatively impacted by a Budget measure once it has been implemented

Are there other rules of thumb I should mention? Any you disagree with?

Update: an observation from Andrew Norton.

If there was a motto for Christmas, it would be: to give is better than to receive. Personally, I find it a lose-lose proposition. As my family well knows, I am absolutely crap at buying presents for people. And I am often stunned at what people buy me.

One of my brothers gave me a book that I would never have picked for myself. Nonetheless, I have just finished reading The Triumph of the Airheads: and the Retreat from Commonsense, by Shelley Gare. According to its dust jacket,

The airheads are winning. We live in an upside-down world where celebrity matters more than substance; correct spelling is considered less important than knowing how to do PowerPoint; bright maths and science students go into investment banking so they can make truckloads of money; and small girls seriously regard a trashy hotel heiress as a role model. We have an American president who gets Sweden and Switzerland mixed up and Australian politicians who spend millions on spin doctors while schools and hospitals go begging. The age of the airhead has one message. Commonsense doesn’t pay off. If you’re smart, be smarter: play dumb.

My brother told me he found the book insightful. It was one of the best books he had read in the last 12 months. If there was ever the need for evidence that siblings can be worlds-apart different; I found the book fatuous (according to chapter 5, this is the new F-word). For much of the book, I was wondering whether it was satire. Was it a clever send-up of airheads supposedly written by an airhead? Or was it just a long, angry stream of disconnected anecdotes purporting to be an argument against illogical thinking and the woes of (post) modernity? In the end, I settled for the latter explanation.

The book does not define an airhead. The reader is left with the task of triangulating a definition from the many, many examples in the book. Airheads appear to include economists; statisticians; economic rationalists; postmodern academics; people with executive MBAs; people who use management jargon-speak; rich people; people more interested in process than outcomes; people who focus on the immediate rather than the longer-term; people addicted to consumerism; people who don’t read much beyond business, lifestyle and gossip magazines; people who think more about themselves than they think of others; people with theories (go figure); people without ethics or morals; and people who want to be accepted by their peer group. If you can see the common thread through all these categories of people, let me know.

Not only is an airhead not defined, their ascent is not charted nor is the mechanism of their ascendancy identified clearly. The book argued that airheads have soared over the past twenty years. However, the case for this argument is not put. Almost all of the anecdotes come from the last ten years. Gare did not refute the possibility that airheads have always been ascendant (although she does acknowledge there have always been airheads). The best she does at explaining the mechanism of their ascent is to say that “there was something in the newly postmodern, economically-rationalist atmosphere that gave them lift”. It’s a circular argument.

The Triumph of the Airheads is a stream of anecdotes of stupid decisions, actions and statements organised loosely into thematic chapters. While these stories were funny, there was no thesis and little connecting story. The book read more like a series of opinion pieces from a magazine — light hearted and amusing, but not compelling. There were ironies here. It read like a montage from the glib lifestyle magazines it so deplored, and it was written by someone who cannot mount an argument (in the classical rather than the postmodern sense).

Beyond the rant, it was difficult to see a proposed course of action to address the triumph of the airheads. There was no remedy. There was no prescription to fix the problems Gare identified. The book is largely a lamentation for a lost — perhaps mythical — time when airheads did not rule the roost.

At the end of the book, I decided an airhead was anyone who disagreed with Shelley Gare’s eclectic world view; myself included. After all, I did some economics and statistics at university. I work in the public service. And (like Howard and Rudd), I am economically conservative.

Ultimately, this book is little more than succor for angry, middle aged, left-leaning, reasonably well-off but not wealthy, (small-c) conservatives. It confirms their prejudices, and names and shames the usual suspects: the rich and powerful, the economic rationalists who have ruined everything, the postmodern academics who have trashed the education system, and the senior bureaucrats and businessmen who run their organisations for their personal gain. It is little more than the latest incarnation of the age-old argument against change. It is a rant against the youth of today (especially the vacuous, selfish, rich under-30s), with their limited understanding of classical literary references (indeed, references to anything that was not on television last week).

Name-calling is an airhead way of dealing with those with whom we disagree. You can see why I wondered whether it was an attempt at satire.

Two stars for the amusing anecdotes.

The ACT Coroner, Maria Doogan, has released her report into the 2003 Canberra fires that killed four people, injured 435 people, and destroyed 487 houses.

While it is arguable that more should have been done to reduce fuel loads in the winter months of 2002, or to attack the fires more aggressively after the lightning strikes of 8 January 2003, the critical issue has always been the absence of compelling pre-fire warnings to the residents of Canberra on 15, 16 or 17 January. Given the intensity of the fires and the prevailing weather conditions, there was little that could have been done in the days immediately prior to 18 January 2003 to stop the fires. The failure to warn is addressed in Chapter 7 of the report.

From the morning of Wednesday 15 January 2003, it should have been clear to the Emergency Services Bureau that the predicted weather conditions for 17-20 January were such that the fire would burn into the Canberra suburbs. At that point, a number of clear warnings were necessary. Canberrans should have been warned: (1) a serious bush fire is approaching and it represents a significant risk to Canberra residents; (2) how to prepare their home and personal effects for the fire; and (3) they need to decide early whether to fight or flee the flames.

However, the Coroner observed,

Until the first Standard Emergency Warning Signal was sounded at about 2.40 pm on Saturday 18 January, there had been no official warnings to the people of Canberra. The SEWS message was too little, and it was delivered far too late.

More seriously, the Coroner noted,

Mr Stanhope either misunderstood or deliberately downplayed the seriousness of the situation in his comments on ABC 666 and 2CC radio station at about 3.00 pm [on 18 January], referring to the declaration of a state of emergency as ‘essentially an administrative measure’ and telling people who were obviously in danger not to be unduly anxious or alarmed.

There has always been a limited number of explanations for the failure to warn Canberra. Either the Emergency Services Bureau was incompetent in its assessment of the fire threat, and/or the ACT Government was negligent and/or incompetent in its failure to warn the citizenry of Canberra. The Coroner opted for the latter explanation. The fire threat was known and communicated in-confidence to the ACT Cabinet and to senior ACT Ambulance and Fire Brigade staff on 16 January 2003. The scale of the threat was clearly understood at the time. However, it was not communicated to the Australian Federal Police. Nor was action taken to warn Canberrans.

The Coroner’s conclusions are compelling,

My overall impression is that senior personnel at the Emergency Services Bureau lacked competence and professionalism and that the bureau was disorganised and was functioning in a chaotic, uncoordinated fashion, particularly during the most critical period of the fires. It seems the left hand did not know what the right hand was doing, and neither hand was actually doing very much to deal with a crisis that was escalating, day by day, hour by hour.

…

The Emergency Services Bureau had been specifically warned as long ago as 1991, again in 1994—in Mr Cheney’s assessment of the Hannan report and the later endorsement of Mr Cheney’s assessment in the McBeth report—and subsequently of the potential for a firestorm of the type experienced in 2003, but the bureau failed to implement procedures to take account of such an eventuality arising from a combination of drought, high winds, high temperatures, low humidity, an ignition source, and a heavy fuel load.

At all relevant times, as Chief Minister and Attorney-General, Mr Stanhope was responsible for the Department of Justice and Community Safety (which organisationally housed the Emergency Services Bureau). Mr Stanhope was also acting in the role of Minister for Emergency Services at the time of the conflagration on 18 January 2003 and on the previous day. As such, in accordance with the conventions of the Westminster model of responsible government, which apply in Australia, Mr Stanhope was the relevant Minister at the most critical time of the firestorm.

On Thursday 16 January, two days before the firestorm hit the suburbs, the Cabinet generally, including Mr Stanhope, knew a potential disaster was on Canberra’s doorstep but did nothing to ensure that the Canberra community was warned promptly and effectively.

To date, the ACT Government has reacted to any criticisms of it by saying it is inappropriate to criticise the hard working fire-fighters who did their best on the day. For a long time this diversionary tactic worked. The Coroner has now pierced the smoke-screen (pun vaguely intended) to apportion blame.

Much has been written about the Prime Minister’s embrace of things green following the release of the Stern Report in the United Kingdom. Clearly the debate has benefited the Greens in the opinion polling (as can be seen in the next graph).

Interestingly, though, it did not translate into additional votes for the Greens in yesterday’s Victorian election. It will be interesting to see if (a) the Green’s Federal polling boost is sustainable, and (b) if it translates into Green primary votes at the next Federal election.

Let’s speculate as to why the Prime Minister has taken on a green hue. I suspect Andrew Bolt had it right on the Insiders this morning. Bolt argued that the Prime Minister was on a hiding to nothing being seen to oppose the environment, so he sought to change the terms of the debate: At what price are you willing to go green and how green are you prepared to go?

In popular mindset, the green recipe is to introduce disincentives on high greenhouse gas energy sources (eg. impose a carbon tax), provide incentives for electricity generation from alternative sources (wind, solar, tidal, geothermal, etc.), and introduce overall demand reduction measures.

There is little argument from anyone on encouraging demand reduction through, for example, better housing design, or a policy of turning the lights off overnight in city buildings. However, there is no stomach among politicians to use rationing to deal with growing demand. Can you imagine the outrage if politicians handled power shortages in the same way they handle water shortages: to tell the people living in even numbered residences that they can only use electricity on the even days of the month.

A carbon tax and encouraging alternatives are just as vexed. For starters, a carbon tax hits the mum and dad punters with higher electricity and petrol costs, it is a tax on industry and would slow the economy, and it would hit the hip pocket with higher costs on all manufactured goods. It is not an electoral winning strategy.

Alternative power sources have their own problems. Typically, they are either not reliable over the 24 hour cycle, significantly more costly than the carboniferous coal, not suited to Australia’s conditions and/or they cannot be scaled to match demand over the next 50 years. Put bluntly, they are not competitive with Australia’s substantial cheap coal reserves, without taxing individuals and industry.

Rather than an unfocused debate around the problem, the Prime Minister is seeking to shift the debate to the difficulties of finding a solution if we wish to maintain our high energy lifestyle. The Prime Minister appears to be arguing that if the greenhouse scenario is a disaster, and a carbon tax is the appropriate response, then nuclear power generation is both economically feasible and a sensible response to our electricity needs.

It is a discussion that wedges the Greens and Labor in their opposition to nuclear, and their absence of fully developed, sustainable and costed alternatives that are competitive. It is a tighter wedge for Labor than the Greens. Unlike the Greens, Labor actually wants to win government and so it does not want to be tarred with higher taxes and crack-pot energy ideas. While John Howard was not in the green camp, all Labor had to do was repeat its broad green policy commitment. Now Howard is in the green camp, political differentiation requires Labor to articulate strategies and not just broad policy positions on the environment.

I suspect the Prime Minister will keep the nuclear option on the table without actually embracing it - the Switkowsky report certainly gives the Prime Minister a lot of wriggle room in this regard. At the same time he will seek to force Labor and the Greens beyond platitudes to articulate specific (non nuclear) strategies that can be costed and critiqued. This tactic is likely to end in a stalemate that sees the coal-fired status quo continue unchallenged.

In summary: The Prime Minister is cactus if he does not recognise the general concerns in the public about green issues. He expects Labor and the Greens to be cactus if they are forced to say what they will actually do. It should be an interesting debate.

With yesterday’s news that underlying or core inflation is above the Reserve Bank’s target range of two to three per cent, an interest rate rise is a safe bet when the Reserve Bank Board meets on the first Tuesday in November.

It will be the third increase this year, and the seventh straight rise since 2002. Some economists are predicting an eighth increase in February 2007.

In July I wrote that we are starting to see increased speculation that the 2007 election will see a change of government driven by a change in the economic fundamentals. I expect more of the same.

It will be interesting to see how the betting market responds in the next few weeks.

Update 11 November: Labor are keen to make this an election issue with the following campaign.