Friggin spammin bots

Bryan · Friday 31 December 2004 · 10:31 am

That internet poker mob are at it again. They spammed every post. Bastards!

I have decided to apply the authorisation code plug-in that Catallaxy uses. I know it’s a pain, but it stops my inbox from being flooded with hundreds of emails awaiting for moderation.

If it rejects your commentary hit the Refresh button to get another number. (I think it is always a good idea to draft your comments in a word processor and then cut and paste them into the blog comment box when you have finished - that way you can be sure not to lose them - and you can check the spelling).

Email me if you are having problems.

Election 2004: opinion polls - part 1

Bryan · Thursday 30 December 2004 · 11:19 pm

The 2004 election was a disaster for Australia’s major polling organisations. While Newspoll and Morgan did passably on predicting the primary vote (Coalition 46.70 to Labor 37.64 per cent), they got it completely wrong on the two party preferred splits. The final two-party preferred result (Coalition 52.74 to Labor 47.26 per cent) was out side of the margin of error for their predictions.

The other big error from Morgan and Newspoll was their prediction the Greens would get a 12 per cent vote in the Senate. As it turns out, the Greens only got 7.5 per cent. But we will leave the analysis of that one for another time.

ACNielsen was within the margin of error for its two party prediction, but it was outside on its Coalition primary vote.

Galaxy got the closest, even though it had a small sample size and therefore a wide margin of error. I am not sure if it was beginners luck or a better methodology.

Anyway, Morgan suggested it failed the two party preferred result because the way in which people said they would cast their preferences differed from what they actually voted. According to Morgan, more people followed the how to vote card rather than voting according to their stated preferences.

The following table presents the four opinion polls immediately prior to the 2004 election. For each, the way in which preferences were allocated to derive a two-party preferred prediction is noted in the rows % residual vote to Coalition, and % residual vote to Labor. For reference purposes the opinion poll results are compared with the last four Federal elections.

  ACNielsen Galaxy Newspoll Morgan 2004 Election 2001 Election 1998 Election 1996 Election
Poll date 5-7 Oct 2004 5-6 Oct 2004 6-7 Oct 2004 7-8 Oct 2004 9 Oct 2004 10 Nov 2001 3 Oct 1998 2 Mar 1996
% Primary Coalition vote 49 46 45 45.5 46.7 43.0 39.5 47.3
% Primary Labor vote 37 39 39 38.5 37.6 37.8 40.1 38.8
% Total major party primary vote 86 85 84 84 84.3 80.9 79.6 86.0
% Residual primary vote 14 15 16 16 15.7 19.2 20.4 14.0
% TPP Coalition vote 54 52 50 49 52.7 51.0 49.0 53.6
% TPP Labor vote 46 48 50 51 47.3 49.0 51.0 46.4
% residual vote to Coalition 36 40 31 22 38.6 41.6 46.7 45.4
% residual vote to Labor 64 60 69 78 61.4 58.4 53.3 54.6
Sample size 2029 1200 2500 1311 N/A N/A N/A N/A

Compared with the other pollsters, it appears that Galaxy had a better handle on how the preferences would flow.

Of note is the declining Coalition share of preferences from those people voting for minor parties and independents in the House of Representatives. It has fallen from a peak of 48 per cent in 1998 to a low of 39 per cent in 2004.

Nonetheless, alarm bells should have been ringing at Newspoll and Morgan with their predictions of 31 and 22 per cent. These figures were implausibly low, particularly Morgan’s. While Morgan did well on predicting the primary vote, it did poorest on the allocation of preferences from those people who did not vote for the Coalition or Labor. That is where it lost the plot.

Morgan has sought to address this problem in the four opinion polls it has released since the 2004 election (see next table). It now publishes two, two-party preferred predictions. The first is based on how people say they will allocate their preference votes. The second is based on the actual distribution of preferences from the 2004 election. In the next table you can see that those polled by Morgan continue to favour Labor with their preferences at a higher rate than at the last election.

Morgan Polls Preferences distributed by how electors say they will vote Preferences distributed by how electors voted at the 2004 election
Coalition Labor Coalition Labor
October 30/31 & November 6/7, 2004 50.5 49.5 50.5 49.5
November 13/14 & 20/21, 2004 50 50 50.5 49.5
November 27/28 & December 4/5, 2004 52.5 47.5 53 47
December 11/12 & 18/19, 2004 51 49 52.5 47.5

This prompted me to consider how Newspoll was performing in terms of its allocation of preferences since the Federal election. The results are set out in the next table. I have also included the four Morgan polls since the election, using Morgan’s two-party preferred result based on the distribution by how electors say they will vote.

Morgan Morgan Morgan Morgan Newspoll Newspoll Newspoll
Poll date 30-31 Oct, 6-7 Nov 2004 13-14, 20-21 Nov 2004 27-28 Nov, 4-5 Dec 2004 11-12, 18-19 Dec 2004 19-21 Nov 2004 3-5 Dec 2004 17-19 Dec 2004
% Primary Coalition vote 44.5 44 46.5 45.5 46 47 46
% Primary Labor vote 39.5 39 36 36.5 38 33 39
% Total major party primary vote 84 83 82.5 82 84 80 85
% Residual primary vote 16 17 17.5 18.0 16 20 15
% TPP Coalition vote 50.5 50 52.5 51 53 55 52
% TPP Labor vote 49.5 50 47.5 49 47 45 48
% residual vote to Coalition 38 35 34.3 30.6 44 40 40
% residual vote to Labor 63 65 65.7 69.4 56 60 60

The first thing to note is that while Morgan begins the period with a similar preference flow to that which occurred during the election, it develops a pro-Labor bias quite quickly. This prompted me to wonder whether this is an explanation for the apparent pro-Labor bias in Morgan polls over the past three years that we have discussed previously. If I get the time, I will investigate this further tomorrow.

Interestingly, Newspoll does not show this trend to the same degree as Morgan. Each of the three Newspolls have a stronger Coalition preference flow than occurred at the last Federal election. [Update: that Newspoll does not show this trend is not surprising. Newspoll allocates preferences for the first two years in the election cycle based on preference flows at the last election. It is only in the lead up to the next federal election that Newspoll asks its respondents questions about their preference intentions.]

In summary, the major opinion polls performed poorly last election because they did a poor job of predicating preference flows from those people who voted for a minor party or independents. My preliminary investigations suggest that Morgan has a greater problem in this regard compared with Newspoll. If this trend is evident over the past three years data, it may go some-way to explaining the pro-Labor bias in Morgan polls between the 2001 and 2004 elections.

Really Simple Syndication

Bryan · Wednesday 29 December 2004 · 9:12 pm

I have updated my email client to Thunderbird. I did it so that I can read my favourite blogs as they are updated. Like magic, as my favourite authors publish their latest insights they appear in my inbox when I check my email.

Some of the blogs I get served fresh every morning without having to hunt for them include Catallaxy, Imagining Australia, John Quiggin, South Sea Republic, Troppo Armadillo, Tim Blair, Darp, and Anonymous Lefty.

Unfortunately, not all of my favourite bloggers use software that supports the Really Simple Syndication (RSS) software or the earlier XML feed. Some of the Luddites I enjoy reading without an RSS or XML link include Mumble, The Poll Bludger, Darp, and Anonymous Lefty. Common guys, it is time to update your web sites.

Election 2004: Senate uplift factors

Bryan · 1:41 pm

Before it hit the self-destruct button, one of the secrets of success for the Australian Democrats was its capacity to attract a substantially increased vote in the Senate compared with the House of Representatives. The party’s psephologist, John Cherry, called this increased vote the Senate uplift factor. There are some interesting stories in the Senate uplift factors for 2004, especially when compared with 2001 and 1998.

2004 Federal Election

Party HOR votes Senate votes Uplift Factor
Coalition 5,471,588 5,390,114 0.99
Labor 4,409,117 4,186,715 0.95
The Greens 841,734 916,431 1.09
Au Democrats 144,832 250,373 1.73
Family First 235,315 210,567 0.89
One Nation 139,956 206,455 1.48
Xn Democrats (Nile) 72,241 140,674 1.95
liberals for forests 8,165 107,130 13.12

2001 Federal Election

Party HOR votes Senate votes Uplift Factor
Coalition 4,934,957 4,864,337 0.99
Labor 4,341,419 3,990,997 0.92
Au Democrats 620,248 843,130 1.36
Green 634,014 574,543 0.91
One Nation 498,028 644,364 1.29
Unity 24,653 30,193 1.22
Xn Democrats (Nile) 65,039 129,966 2.00

1998 Federal Election

Party HOR votes Senate votes Uplift Factor
Coalition 4,388,809 4,225,736 0.96
Labor 4,454,306 4,182,963 0.94
Au Democrats 569,935 947,940 1.66
Green 290,709 305,228 1.05
One Nation 936,621 1,007,439 1.08
Unity 87,252 93,968 1.08
Xn Democrats (Nile) 64,916 122,516 1.89

Liberals for Forests wins the uplift factor prize for 2004. For every primary vote Liberals for Forests got in the House of Representatives they multiplied it by just over 13 primary votes in the Senate. While some micro parties (not listed above) achieved a higher uplift factor, they did much poorer in their total number of Senate votes. Of the 107,130 Senate votes for Liberals for Forests in 2004, 55,170 came from Victoria. This was down on the more than 70,000 Victorian votes cast in 2001, but then the party held the donkey vote winning first column position. The other big supporters of Liberals for Forests in 2004 were New South Wales (21,179), Queensland (22,283) and Western Australia (5680). The Victorian success for Liberals for Forests in 2004 is particularly remarkable as the party had no House of Representatives candidates in Victoria. Liberals for Forests may be able to achieve a higher Senate vote in the next election by increasing its presence at the polling booth with more candidates in the House of Representatives.

Whereas the Liberals for Forests won the uplift factor prize, the loser was the new party for 2004: Family First. Family First only managed to translate 90 per cent of its House of Representatives vote into a Senate vote of 210,567. Family First is not likely to do as favourably in the exchange of preferences for the next election, so it will need to boost its uplift factor. It might learn how to do this from the other Christian party: the Christian Democrats (also known as Call to Australia and the Fred Nile Group). While Family First out polled the Christian Democrats in 2004, the Christian Democrats have increased their vote from 123 thousand in 1998, to 130 thousand in 2001, to 141 thousand in 2004. More importantly, the Christian Democrats have achieved a Senate uplift factor around 2 at each of these three elections. For every vote they got in the House of Representatives they received two votes in the Senate. If Family First could achieve an uplift factor of 2 it might become a big league player in future Senate elections.

The big vote improver over the past three Federal elections has been the Greens. In round terms, it has increased its Senate primary vote from 300 thousand in 1998, to 575 thousand in 2001, to 915 thousand in 2004. However, at each of these election it failed to achieve a significant uplift over its House of Representatives vote.

The big vote decliners over the period have been One Nation and the Australian Democrats.

One Nation’s Senate vote fell from 1 million in 1998, to 640 thousand in 2001 to just over 200 thousand in 2004 (although you could add a further 100 thousand if you count the votes Pauline Hanson got as an independent in Queensland). Surprisingly, as One Nation has lost its share of the Senate vote, it has managed to improve its uplift factor from 1.08 to 1.29 to 1.48.

The Australian Democrats had a similar experience in respect of its primary Senate vote. Its Senate vote fell from just under 1 million in 1998, to 840 thousand in 2001, to 250 thousand in 2004. However its uplift factor has oscillated from 1.66 to 1.36 to 1.73. The Australian Democrats may have a better uplift factor than the Greens because of the “keep the bastards honest” role it cast for itself.

Of the two major parties, the Coalition has managed to retain almost all of its House of Representatives vote in the Senate, particularly in 2001 and 2004. Labor, on the other hand, lost between 5 and 8 per cent of its primary vote at each election. With the state quota for three seats being 43 per cent, the Coalition is much better placed to win a majority of seats with a national Senate primary vote of 45 per cent compared with Labor’s 35 per cent.

New look and feel

Bryan · Tuesday 28 December 2004 · 10:32 pm

I have opted for a simpler style sheet for the blog. Some people had difficulty with the previous version. I hope this works for everyone.

Also, I have decided to take a break over January. I might get to post once or twice, but it will not be much more often than that.