ELECTION
2004 - Flawed Polling
Excellent sites to go to for Election 2004
coverage: Daily Kos,
DonkeyRising,
CJR Campaign
Desk, the Swing
State Project, Political
Animal, Atrios/Eschaton,
Talkingpointsmemo,
MyDD, Our
Congress, 2.004k.com,
Electoral
Vote, Race 2004
For the eRiposte Election 2004 home page, click
here.
ELECTION
2004: CAVEAT EMPTOR
Flawed Poll Results Tilt Significantly and Artificially in Favor of
Bush
Kerry is
not Out of the Woods Yet But He Appears to be About Even* with Bush
9/20/04
SUMMARY
In recent days, widely varying poll
results have been reported by different polling organizations. Some
polls show large or enormous leads for George W. Bush (over John Kerry),
while others show a tie or slight Kerry lead. However, analysis by seasoned poll watchers
show serious methodological flaws in the former polls - especially the
use of meaningless "Likely Voter" models and
unproven/incorrect Party ID weighting in Registered Voter models. Adjusting for
such flaws provides a picture of a race that is essentially close and
unpredictable - and essentially dead even* at this time. Other
factors such as the Margin of Error (MoE) in the polls, the Nader
effect, the impact of Absentee Ballots, and Undecided Voters, is also
not properly accounted for in some articles that provide
"analysis" on polling results. There are some broad lessons
here for Kerry
supporters - and some feedback for the Democratic Party and Senator
Kerry.
[P.S. A less noted but significant
aspect in the Gallup poll which skewed dramatically towards Bush is that
Kerry is "leading"
Bush amongst independents by 7% (MoE 4%)]
ACKNOWLEDGEMENTS The
following sites are acknowledged in particular since their analysis forms the main
basis for my summary:
Donkey
Rising (Ruy Teixeira), MyDD,
DailyKos, Left
Coaster OUTLINE
1. Brief summary of
some recent polls 2.
POLLING ASSUMPTIONS: What should we believe?
2.1
"Likely" Voter (LV) models 2.2
Registered Voter (RV) models 2.3
Margin of Error (MoE) 2.4
The Nader Effect 2.5
Overseas Absentee Ballots? 2.6
How do Undecideds Split Their Votes? 2.7
ASIDE: Telephone polling -- What about Cell Phone Owners? 2.8
WHAT MATTERS: The Electoral College!
3. Recent National
Poll results adjusted for LV and RV model flaws
3.1 Gallup 3.2
CBS/New York Times
3.3 Other Polls:
ABC, Fox, IDB, Newsweek
4. Conclusions and
Lessons for Kerry supporters
4.1 Some Election
2000 History
5. Still a close,
likely dead-even race: Feedback to Senator Kerry and the Democratic Party
6. APPENDIX: Why do
"leading" media outlets publish stories using highly
questionable results?
DETAILS
1. BRIEF SUMMARY OF RECENT
POLLS
The table below is extracted from the
useful site 2.004k.com,
showing the NATIONAL poll results reported in the past week. The third
column in the table is critical, and 2.004k.com provides important
guidance on this column (at the bottom of their page) - something
that is explored in greater detail in the next section.
RV = Registered Voters
(most desired group)
A = All Adults (next-most desired group)
LV = Likely Voters (arbitrary and subjective; least desired group)
As the table shows, many recent polls
show Kerry either tied with Bush or even slightly "ahead".
Others show Bush significantly "ahead". This is what we need
to explain and understand today.
CAUTION! I'm using terms like "ahead" and
"behind" loosely and this is NOT statistically accurate
terminology because it ignores the margin of error (MoE)! If you
included MoE, the vast majority of these results suggest a close and
unpredictable race. If you want to understand confidence intervals
(also important) and MoE, here are some useful links: StudyWorks,
CampaignDesk,
Secular
Blasphemy
Trend
in 2004 |
Bush
- Kerry -
Nader - (MoE) |
* |
Date |
Organization |
| Bush
by 9% |
50-41-3-(3) |
RV |
9/12
- 9/16 |
CBS
News/New York Times |
| Bush
by 8% |
50-42-n-(3) |
RV |
9/12
- 9/16 |
CBS
News/New York Times |
| Bush
by 1% |
47-46-2-(2) |
RV |
9/13
- 9/15 |
Economist/YouGov |
| Bush
by 1% |
46-45-1-(2) |
A |
9/13
- 9/15 |
Economist/YouGov |
| Bush
by 2% |
48-46-1-(2) |
LV |
9/13
- 9/15 |
Economist/YouGov |
| Bush
by 8% |
52-44-n-(4) |
RV |
9/13
- 9/15 |
CNN/USA
Today/Gallup |
| Bush
by 6% |
51-45-n-(4) |
A |
9/13
- 9/15 |
CNN/USA
Today/Gallup |
| Bush
by 13% |
55-42-n-(4) |
LV |
9/13
- 9/15 |
CNN/USA
Today/Gallup |
| Bush
by 2% |
47-45-3-(3.1) |
LV |
9/12
- 9/14 |
Greenberg
Quinlan Rosner |
| Bush
by 1% |
49-48-n-(3.1) |
LV |
9/12
- 9/14 |
Greenberg
Quinlan Rosner |
| Tie |
46-46-1-(3.5) |
RV |
9/11
- 9/14 |
Pew/PSRAI |
| Bush
by 6% |
49-43-1-(2.5) |
RV |
9/8
- 9/14 |
Pew/PSRAI |
| Kerry
by 1% |
47-48-3-(3) |
LV |
9/9
- 9/13 |
Harris |
| Bush
by 4% |
49-45-n-(3.5) |
LV |
9/9
- 9/12 |
Penn,
Schoen & Berland |
| Bush
by 3.5% |
47.6-44.1-3-(3.3) |
RV |
9/8
- 9/12 |
ICR |
| Kerry
by 2% |
43-45-3-(3.5) |
RV |
9/7
- 9/12 |
Investor's
Business Daily/TIPP |
| Kerry
by 2% |
44-46-n-(3.5) |
RV |
9/7
- 9/12 |
Investor's
Business Daily/TIPP |
| Tie |
46-46-3-(3.5) |
LV |
9/7
- 9/12 |
Investor's
Business Daily/TIPP |
| Tie |
47-47-n-(3.5) |
LV |
9/7
- 9/12 |
Investor's
Business Daily/TIPP |
2. POLLING ASSUMPTIONS: WHAT
SHOULD WE BELIEVE? 2.1
"Likely" Voter (LV) models The
main point I highlight in this section, as Ruy Teixeira has elegantly
argued for a while now, is that Likely Voter models create
significant errors in projecting the horse-race results. As a
clincher, he notes:
In 3 of the last 4 presidential
elections (including the last one), Gallup's final RV reading was
actually closer to the final result than their final LV reading!
Why
are LV models very likely (!) to be flawed? Teixeira in The
Gadflyer says:
The
other problem that is afflicting the polls and considerably
inflating perceptions of Bush's lead is the widespread, and highly
questionable, use of LVs, instead of RVs, to report horse race
results far in advance of the actual election. The reason why using
LVs instead of RVs is a bad idea is simple: the LV approach is
being asked to do a job—gauge voter sentiment and how it changes
from week-to-week (and even day-to-day)—that it was never designed
to do. What the LV approach was designed to do was measure
voter sentiment on the eve of an election and predict the outcome.
That was, and remains, an appropriate application of the LV
approach.
In
this post he summarizes the case against Gallup's LV data (bold
text is eRiposte emphasis):
Here is a summary of
the case against Gallup's LV data:
...Gallup
decides who likely voters are based on 7 questions about their
interest in voting, attention to the campaign and knowledge about
how to vote (e.g., where their polling place is located). The
interested/attentive/knowledgeable voters are designated
“likely” and the rest are thrown out of the sample. But as a
campaign progresses, the level of interest among voters tends to
change, particularly among those with partisan inclinations
whose interest level will rise when their party seems to be
mobilized and doing well and fall when it is not. Because
of this, partisans of the mobilized party (lately, Republicans)
tend to be screened into the likely voter sample and partisans of
the demobilized party (lately, Democrats) tend to get screened
out. But tomorrow, of course, the Democrats could surge, in which
case their partisans may be the ones over-represented in likely
voter samples.
That suggests
the uncomfortable possibility that observed changes in the
sentiments of “likely voters” represent not actual changes in
voter sentiment, but rather changes in the composition of likely
voter samples as political enthusiasm waxes and wanes among the
different parties’ supporters. And that is exactly what
political scientists Robert Erikson, Costas Panagopoulos, and
Christopher Wlezien find in their analysis of Gallup's 2000 RV/LV
data in their forthcoming paper, “Likely (and Unlikely) Voters
and the Assessment of Campaign Dynamics” in Public Opinion
Quarterly: “shifts in voter classification as likely or unlikely
account for more observed change in the preferences of likely
voters than do actual changes in voters’ candidate
preferences.”
That means that,
instead of giving you a better picture of voter sentiment and how
it is changing than conventional registered voter data, likely
voter data give you a worse one since true changes in voter
sentiment are swamped by changes in who is classified as a likely
voter.
...And then there's
this: the
LV data haven't been working so well lately even right before the
actual election. In 3 of the last 4 presidential elections
(including the last one), Gallup's final RV reading was actually
closer to the final result than their final LV reading!
Thus, as Teixeira says
(bold text is eRiposte emphasis):
Gallup
and its sponsoring organizations implicitly and explicitly
encouraged people to treat the LV finding as the real story and the
RV finding as an unreliable afterthought (after all, those voters
aren't "likely"!). The incredible irony, of course, is
that the real situation was exactly the reverse: as the Erikson
et al. findings suggest, it was the RV data that provided the best
gauge of voter sentiment and the LV data that should have been an
unreliable afterthought.
Finally, this
post from him (September 8th) illustrates the folly of using LV
models this far ahead, more quantitatively (bold
text is eRiposte emphasis):
Now
consider this excellent analysis along the same lines by Professor
Alan Abramowitz of Emory University, one of the leading academic
analysts of American politics. (He sent this to me in an email and
graciously agreed to allow me to share it with readers of this
blog.)
1. The
latest Gallup Poll has Bush ahead of Kerry by 52-45 percent among
likely voters but by only 49-48 percent among registered voters.
Based on the numbers of registered and likely voters in their
sample, this means that Gallup is projecting that 89 percent of
Bush supporters will vote but only 79 percent of Kerry supporters
will vote. That seems unrealistic. It is way out of line with data
from the American National Election Studies on turnout among
registered Dems and Republicans in recent elections. For the past
three presidential elections, the turnout gap between Republicans
and Democrats has averaged 3 percentage points and was never
larger than 4 percentage points. The smallest gap was in 1992 (1
point), the election with the highest overall turnout. Assuming
that 2004 will be another relatively high turnout election, we
should probably expect a relatively small turnout gap, similar to
1992.
The lesson? In
general, don't pay much attention to LV data. 2.2
Registered Voter (RV) models Having
dispensed with the LV model issue, we still have to address the
apparent discrepancy seen in the RV results shown in Section
1. It turns out, RV results from many organizations in this
election cycle also have flaws in the assumptions they have made on
party identification of voters. Using the documented party ID from
the 2000 election, or even in the 2004 study by Pew would reverse the
results in some cases (i.e., Kerry would lead Bush rather than the
other way around) and make them close-to-even in others. Here's
Teixeira again (bold text is my emphasis):
Lately,
and very suddenly, many polls have been turning up more
Republican identifiers than Democratic identifiers in their
samples—in some cases, many more (as high as a nine- to ten-point
Republican advantage).
How
realistic is it to be suddenly turning up a Republican lead on party
identification, much less a large one? Not very. The weight of the
academic evidence is that, while the distribution of party
identification among voters can and does change over time, it
changes slowly, not in big lurches from week to week.
And
the weight of the empirical evidence is that the distribution of
party identification among voters has favored and continues to favor
the Democrats. In 2000, the exit polls showed Democrats with a
four-point advantage over Republicans. In 1996, it was also five
points; in 1996 [eRiposte note:
presumably Teixeira meant 1992?],
it was three points and in 1988 it was also three points.
The
data also indicate that there were two shifts in party
identification over the 2001–2004 period which largely canceled
each other out. The first shift, in the period after September
11, shaved several points off the Democrats' lead and brought the
Republicans close to even (but never ahead) in party identification.
The second shift took place in late 2003 and 2004 and reconstituted
the Democrats' lead on party identification to about four points,
exactly where it was in the 2000 election according to the exit
polls (see this useful study "Democrats
Gain Edge in Party Identification" by the Pew Research
Center for more details).
But
if the party identification distribution is fairly stable and tends
to change rather slowly, why would polls suddenly be turning up
unrealistically high numbers of Republican identifiers? The best
explanation, in my view, is that when the political situation jazzes
up supporters of one party, they are more likely to want to
participate in a public opinion telephone poll and express their
views. An increased rate of interview acceptance by that party's
supporters would then skew the sample toward that party without the
underlying distribution having changed very much, if at all.
In this
case, the Republican convention, coming on the heels of the Swift
Boat controversy, may have helped raise political enthusiasm among
Republican partisans, leading to more interview acceptances and a
disproportionate number of Republicans in recent samples.
But
whatever the explanation for the disproportionate number of
Republicans in recent samples, if those numbers are unrealistic,
they are skewing reported horse race results toward Bush. What, if
anything, should be done about this?
One
possible solution is to weight poll results by a more reasonable
distribution of party identification. The issue of whether to
use this approach to the problem is well-summarized by Alan Reifman
in his invaluable essay "Weighting
Pre-Election Polls for Party Composition: Should Pollsters Do It or
Not?" on his website.
As
Reifman puts it:
One factor (among
many) that may contribute to discrepancies between different
outfits' polls in their Bush-Kerry margins . . . is polling firms'
different philosophies as to whether it's advisable to
mathematically adjust their samples—after all the interviews
have been completed—to make the percentages of D's and R's in
their survey sample match the partisan composition that is likely
to be evident at the polls on Election Day. The latter can be
estimated from exit polls from previous elections, party
registration figures (in states where citizens declare a party ID
when registering to vote), and surveys.
...
Given
that party identification does shift some over time, my instinct has
generally been to avoid party-weighting if possible and promote a
full-disclosure approach. This
is how I recently put it in Public Opinion Watch:
[B]ecause the
distribution of party identification does shift some over time . .
. polls should be able to capture this. What I do favor is release
and prominent display of sample compositions by party
identification, as well as basic demographics, whenever a poll
comes out. Consumers of poll data should not have to ferret out
this information from obscure places—it should be given
out-front by the polling organizations or sponsors themselves.
Then people can use this information to make judgements about
whether and to what extent they find the results of the poll
plausible.
But
this approach increasingly seems unrealistic to me. The polling
organizations and sponsors do not routinely release the data
I call for and certainly do not prominently display them. And
even if they did, the typical consumer of polling data lacks the
time and skills to use these data to re-weight or adjust reported
results. The fact of the matter is that people pay attention to
reported results period; therefore they are at the mercy of
whichever results are reported and emphasized (an issue that also
looms large in the LVs vs. RVs issue, discussed below).
This
suggests that weighting poll results by a reasonable distribution of
party identification may be necessary to avoid giving the public
distorted impressions of the state of the race.
What
is a reasonable distribution of party identification to use in such
weighting? One obvious candidate is the exit poll distribution from
2000: 39 percent Democrats, 35 percent Republicans, 26 percent
independents. Moreover, the Democratic advantage in this
distribution—four points—closely matches the average Democratic
advantage in 2004, as measured by the Pew Research Center (see
above) and other polling organizations, making it an even more
attractive option.
But
political analyst Charlie Cook probably has the best idea, even
though it can really only be implemented by the polling
organizations themselves: "dynamic party identification
weighting." Cook's idea is that polls should weight their
samples by a rolling average of their unweighted party
identification numbers taken over the previous several months. This
would allow the distribution of party identification to change some
over time, but eliminate the effects of sudden spikes in partisan
identifiers in samples (such as we are experiencing now).
Lacking
such a dynamic weighting, however, the best we can probably do at
this point is to use the exit poll distribution mentioned above. How
much difference would this make if we applied it to recent polls?
Quite
a bit. Here are Bush's leads in a number of recent polls,
ordered by size of his lead, once the horse race question is
weighted by the 2000 exit poll distribution (note: not all recent
polls can be included because you need the horse race figures among
Democrats, Republicans, and independents separately to do this
procedure and not all polls release these figures; in addition Zogby
and Rasmussen results are party-weighted to begin with and therefore
do not have to be re-weighted; RV results used unless only LV
results available):
CBS
News, September 6–8 RVs: +5
Zogby, September 8–9 LVs: +2
Rasmussen: September 10–12 LVs: +1
Fox News: September 7–8 LVs: +1
Washington Post, September 6–8 RVs: +1
Newsweek, September 9–10 RVs, –2
Gallup, September 3–5 RVs: –4
These
data present a clear picture of a tight race, with Bush likely
running a small lead, but not the solid—and even large—advantage
that has been conveyed to the public.
Separately, pollster John Zogby has
also indirectly criticized Newsweek for using highly unrealistic
weightings in their RV models (bold text is eRiposte emphasis):
Two new polls came
out immediately after mine (as of this writing) by the nation's
leading weekly news magazines. Both Time's 52% to 41% lead among
likely voters and Newsweek's 54% to 43% lead among registered voters
give the President a healthy 11 point lead. I have not yet been able
to get the details of Time's methodology but I have checked out
Newsweek's poll. Their sample of registered voters includes 38%
Republican, 31% Democrat and 31% Independent voters. If we look
at the three last Presidential elections, the spread was 34%
Democrats, 34% Republicans and 33% Independents (in 1992 with Ross
Perot in the race); 39% Democrats, 34% Republicans, and 27%
Independents in 1996; and 39% Democrats, 35% Republicans and 26%
Independents in 2000. While party identification can indeed
change within the electorate, there is no evidence anywhere to
suggest that Democrats will only represent 31% of the total vote
this year.
...
This is no small
consideration. Given the fact that each candidate receives anywhere
between eight in ten and nine in ten support from voters in his own
party, any change in party identification trades point for point in
the candidate's total support. My polls use a party weight of 39%
Democrat, 35% Republican and 26% Independent. Thus in examining
the Newsweek poll, add three points for Mr. Bush because of the
percentage of Republicans in their poll, then add another 8% for Mr.
Bush for the reduction in Democrats. It is not hard to see how we
move from my two-point lead to their eleven-point lead for the
President.
I will save the
detailed methodological discussion for another time. But I will
remind readers that my polling has come closest to the final results
in both 1996 and 2000.
UPDATE 10/7/04:
Chris Bowers has a significant
update on Party ID distribution.
Both Pew
and Mystery
Pollster, who is actually Democrat Mark Blumenthal, make the same
argument about Party ID. Specifically, they claim Party ID is an
attitude, not a demographic, and thus is not something a poll should
be weighted by.
...
Instead, after
reading through the data presented by both Pew and Blumenthal, and I
have come to agree with them, but only in part. Independent Party ID
is an attitude. By contrast, Party ID among Democrats and Republicans
is remarkably stable and does constitute a demographic. Almost all of
the shifts caused in Democratic and Republican Party ID is caused by
independents either shifting to one major party, or shifting away from
one major Party back toward independent. For example, here is the
chart Pew produced to argue that Party ID is unstable:
...
The change is not
taking place among Democrats and Republicans. Instead, the change is
taking place within two groups of voters, "Independent
Democrats" and "Independent Republicans," that a
three-way Party ID question is unable to measure. According to the
chart that Pew produces to argue that Party ID is not stable, what
they end up instead proving is that "independents" when
considered as a block, are unstable. Of the 16% they identify as
changing Party ID from October 23-26, 1988 until November 9-11, 1988,
15% were changing from independent to one of the two parties or from
one of the two parties to independent. Only 1% of the people
questioned shifted from one party to the other.
Pew's 1992 and 2000
graphs show similar lack of movement among partisans. From June 1992
until November 1992, only 4% of the population shifted from one
party to the other, while 22% shifted from independent to one major
party or from one major party to independent. In 2000, only 2%
shifted from one major party to the other, while 18% shifted either
from one major party to independent or from independent to one of
the major parties.
Mystery Pollster goes
on to cite the 2000
National Annenberg Election survey...The conclusion that NAES
made from this was that Independent Party ID "was not stable
over time."
Fine. Independent
Party ID is not stable over time. Pew demonstrates the same.
However, at the same time there is no evidence of any significant
shift from Democrats to Republicans or vice-versa. Democrats are not
becoming Republicans and Republicans are not becoming Democrats.
Even the data used to argue that Party ID is an attitude and not a
demographic demonstrates that almost every change in Democratic and
Republican Party ID occurred as a result of Independents shifting
one way or the other.
One could stop at
this point and assume that Party ID is stable among partisans, but
not among Independents. However, this would be wrong, since the
shift these polls are measuring among Independents is a mirage
generated by a three-way Party ID question in what is at least a
five-way Party ID country. Specifically, what most polls ignore is
that while around 40% of the population consider themselves to be
"Independents," around 75% of Independents lean toward one
party or the other and can be accurately considered
"Independent Democrats" or "Independent
Republicans". However, by forcing those two groups of leaners
to choose between their two allegiances, especially in a dataset of
1,500 or less, the three-way question, and possibly wavering base
energy, will create the appearance of a shift that has not taken
place.
...
Overall, what I
think this tells us about polling is, basically, that we lack
adequate information to know how accurate polls are in terms of
Party ID (unless, like Gallup, there is an extreme outlier).
...
More importantly,
what I think this shows is the folly of trying to appeal to
independent voters as a campaign tactic, instead of consolidating
and energizing your base. There just are not many "Independent
Independents" out there (maybe a little than 10% of the
registered voter population), and they
don't vote very often anyway. I wouldn't be surprised if
"Independent Independents" make up only 6-7% of the vote
in national elections. Why bother spending so much time appealing to
such a small group when keeping your base intact is far, far more
important?
Howard Dean may have
had some image problems, but at least he had the right strategy:
appeal to the base. Register the base. Energize the base.
Consolidate the base. This is especially true for Democrats, who
have a larger base than Republicans. Our problem has historically
not been with independents, but with not holding our own base
together and with turning out at a lower rate than Republicans. When
has Democratic turnout ever equaled Republican turnout? When has a
Democratic nominee ever held his base together as well as the
Republican nominee? 1964 is probably the last election that meets
both criteria.
Howard Dean had it
right, and Karl Rove has it right as well. Woe be unto us if we
continue to mock the way Bush is only going after his base, while
Kerry seeks "swing voters." Bush has a solid strategy, but
I worry that Kerry has been chasing a mythical beast.
Also see Dailykos.
The lesson? RV results are largely
meaningless unless weighted accurately by party ID. Democratic and
Republican Party ID appears to be far more stable that Independent
Party ID. This seems to be less due to people switching all over the
place than how the polls are conducted and questions are asked. 2.3
Margin of Error (MoE) I
already mentioned MoE briefly in Section 1 and
will simply reiterate here what I said there. Terms
like "leading"/"ahead" and
"trailing"/"behind" are OFTEN NOT statistically
accurate terminology because they ignore the margin of error (MoE). If
you included MoE, the vast majority of these results suggest a close
and unpredictable race. If you want to understand how to interpret
MoE (and also confidence intervals which are rarely mentioned even
though they are relevant), here are some useful links:
StudyWorks, CampaignDesk,
Secular
Blasphemy 2.4 The
Nader Effect Some polls
continue to report results with Republican, er Reform Party,
er Green Party, er Independent candidate Ralph Nader in the
mix. Of course, to their credit most of them seem to be reporting
results with and without Nader. The point though is that results
from national polls that include Nader need to be viewed with some
amount of skepticism because Nader is not going to make it to the
ballots in many states. GoKeever
at DailyKos did a systematic Nader survey on 8/5/04 which provides
a window into the Nader effect. Since then, Chris
Bowers at MyDD provided a simple summary of the Nader effect as of
8/30/04, which I reproduce here (the highlight is eRiposte emphasis).
Now
that Nader
is off the PA ballot, I recalculated the percentage of people
who can and cannot vote for him. Considering the success of legal
efforts either to have Nader removed from ballot or have his ballot
petitions rejected, and the abject failure of legal challenges to
have Nader placed on the ballot, Nader's ballot status looks bleak
indeed. Unless he has actually been certified on a state ballot and
no legal challenges remain, his status should be considered TBD. If
his petition has been rejected, then he should be considered off the
ballot.
|
Off |
On |
TBD |
| %
of US pop over 18 |
48.25 |
5.83 |
45.92 |
| %
of 2000 Nader Vote |
36.44 |
5.90 |
57.66 |
| Electoral
Votes |
239 |
37 |
262 |
Nader has been
verified on the ballot in six states where there no longer remain
legal hurdles for him to clear: AR, DE, IA, MT, NJ and SD. Nader is
off the ballot in 14 states: AZ, CA, GA, ID, IN, IL, MD, MI, MO, OK,
PA, NC, SC and TX. (source)
Quite a lot of
territory remains to be determined, but so far Greens and Democrats
have joined forces to rout Nader off the ballot. This is just one
wing of a huge Democratic effort this year--an effort that seems to
be dwarfing what took place in 2000--and I for one am enjoying
watching it succeed. The state with the final ballot deadline is
Minnesota, on 9/14. Much more will be known by then.
Incidentally,
[Libertarian Candidate] Badnarik will be on every single state
ballot, with the possible exception of Oklahoma.
The bottom line? Nader's impact will be
seen only in some states, and although that includes some battleground
states, there are other battleground states where he will be off the
ballot. Including his name in National Poll results (without Badnarik)
could provide an additional, inaccurate skew towards Bush. 2.5
Overseas Absentee Ballots The
website Electoral Vote suggests overseas voter registration may be
above average this election (see below). Considering how overseas absentee ballots
gave Bush an edge in Florida in 2000, and how close the races are this
year, we should not forget these ballots! They could swing some states
in a manner not captured by the national or state polls. Given the
significant support for Kerry (over Bush) outside the U.S., it is possible
that Kerry may benefit at least to a small extent from these votes in
2004, but it is hard to say right now. Here's Electoral
Vote:
I have it on good
authority that overseas voters are registering in huge numbers this
time, maybe double or triple 2000. I was told that the number of
people who showed up at the Democratic party caucus in England
earlier this year was 10 times what it was in 2000, ditto in other
countries. Americans overseas vote in the state they last lived in,
even if that was decades ago. There are about 7 million overseas
Americans and probably about 5 million are over 18. In Florida, it
was the overseas absentee ballots that swung the election. I believe
that something like 8% are military, but the rest are students,
teachers, artists, government workers, business executives, spouses
of foreign nationals, missionaries, retirees, and more. What is
significant here is that these people represent a lot of votes and
are not included in any of the polls. Nobody knows if they are
largely Democrats or Republicans, but their votes could be one of
the big surprises of this election.
Bottom line? Don't
forget absentee ballots and their ability to swing close races. If
you know people outside the U.S. ask them to vote!
2.6 How do Undecideds Split
Their Votes? No one I know on
the web has done more work to understand this aspect than Chris Bowers
at MyDD. Chris outlined his thinking and superb analysis in a number of recent
posts: here,
here
and here.
The last of these articles provides the rule that Chris is now using
(bold text is eRiposte emphasis):
Lately, I have been
spouting off about how undecideds [Und.] tend to break about 60% for
the challenger [Chal.] and 40% for the incumbent [Inc.] in the final
week of the campaign. I was basing this off of a quick estimate from
research I had done five weeks ago. However, I had never actually
sat down and crunched the numbers to truly determine the average
undecided swing in the final week of a campaign in elections
involving an incumbent. Now I have... Using
my own research as well as research sent to me by Nick Panagakis of
the National Council on Public Polls,
I have gone through 451 poll results since 1976. In all 451 cases,
the poll was in the field for at least one day that was within seven
days of the election. In every case, it was the final poll taken by
the polling firm for the campaign in question. Also, I do not
believe that any internal or partisan poll results were used.
Unfortunately, outside of the Presidential race, I was severely
lacking in data from 1996. ...The
results were as follows:
| Year |
Polls |
Und. |
Inc. |
Chal. |
| President |
28 |
2.4 |
14% |
86% |
| 1976-88 |
155 |
11.8 |
20% |
80% |
| 1994 |
101 |
11.2 |
35% |
65% |
| 1998 |
76 |
10.1 |
27% |
73% |
| 2000 |
31 |
8.6 |
40% |
60% |
| 2002-4 |
60 |
7.5 |
42% |
58% |
| 1992-04 |
283 |
8.9 |
34% |
66% |
| Total |
451 |
9.7 |
28% |
72% |
Here
are my first thoughts on this data:
- Is the incumbent
rule weakening? With the exception of 1998, where Democrats
kicked ass on GOTV, it has been a slow progression toward
parity. Then again, this chart has holes, such as a complete
lack of polls from 1996, which prevents one from drawing such a
conclusion with confidence.
- The number of
undecided voters is clearly going down. This year, I see no
reason not to expect more of the same. As the two parties slowly
become primarily ideological, rather than regional and ethnic
coalitions, the difference to voters is becoming starker. As a
result of this, more people have made up their minds going into
the booth. We really are becoming a polarized nation--that isn't
just pundit bullshit.
- The Presidential
sample stands out for its extremely small movement from final
polls until election night. Even though undecideds break
overwhelmingly--better than 6 to 1--in favor of the challenger
in a Presidential race, pollsters seem particularly adept at
national trial heats in Presidential races. Probably because of
the extreme amount of national attention given to the
Presidential race, far more people have made up their minds
going into the booth than in other elections. While we should
not expect significant movement from the final polls on November
1 to the final results on November 2, whatever small movement
there is will be almost entirely for Kerry.
- Considering how
accurate polling firms tend to be in national elections, in my
presidential projections I will now include likely voter models
from polling firms that do not include registered voter models.
Where available, I will still use the registered voter models
until the final week, however. Also, I will only include trial
heats that push leaners, and give 80% of the remaining undecided
to Kerry. There is simply no way that even 5% of the country
will be truly undecided between Bush and Kerry going into the
booth.
- Should the
1992-2004 average of 66 to 34 in favor of the challenger, or 2
to 1, be considered a benchmark at which point GOTV efforts make
the rest of the difference? At least for now, I'm going to run
with this possibility.
It seems like it would
be fine to use these numbers as a means of estimating how undecideds
will break in any given non-presidential election, as long as it is
sold purely as an estimation. These figures are averages and
invariably there will be results that break sharply from that
average. 66-34 is where the smart money, the house money, should be.
You will win over the long term betting on that sort of a split, but
in any given wager you might get burned.
UPDATE 10/7/04:
Chris Bowers turns in more
analysis! Bold text is my emphasis.
Last month I reported an
update to my research Incumbent Rule. Through an examination of
451 final pre-election polls in campaigns with an incumbent from
1976 until today, 72% of all undecideds broke toward the challenger
while 28% of all undecideds broke toward the incumbent. From
1992-2004, that percentage had dropped slightly. Over 283 polls, 66%
of the undecideds broke for the challenger while 34% broke for the
incumbent.
However, these figures are the average of a wide variety of
congressional, gubernatorial and Senatorial races. Even in these
campaigns, while fairly well known, the incumbents rarely suffered
from 100% name recognition and widespread crystallization of
opinion. When looking at only final pre-election polls in the
ultimate incumbent versus challenger campaigns--Presidential
campaigns--the percentage of undecideds to break for the challenger
increases sharply. In 28 final pre-election polls in Presidential
elections featuring an incumbent from 1976-1996, a whopping 86% of
undecideds broke for the challenger, while only 14% went for the
incumbent.
...
As Guy
Molyneux shows in the American prospect, incumbent Presidents
finish almost precisely where their final poll results project they
will finish:
There have been four incumbent presidential elections in the
past quarter-century. If we take an average of the final surveys
conducted by the three major networks and their partners, we
find that in three of these the incumbent fell short of or
merely matched his final poll number, while exceeding it only
once, and then by just a single point (Ronald Reagan). On
average, the incumbent comes in half a point below his final
poll result.
Year Incumbent Final Poll % Actual Vote %
1996 Clinton 51 49
1992 Bush 37 37
1984 Reagan 58 59
1980 Carter 42 41
The numbers for challengers look quite different. In every case,
the challenger(s) -- I include Ross Perot in 1992 and 1996 --
exceed their final poll result by at least 2 points, and the
average gain is 4 points. In 1980, Ronald Reagan received 51
percent, fully 6 percentage points above his final poll results.
Looking at just Gallup, Mystery
Pollster delivers even more bad news for incumbent Presidents:
[T]he final Gallup projections (sans undecided) show an
intriguing pattern: In the presidential elections since 1956
that featured an incumbent, Gallup's final projection of the
incumbent's vote exceeded the incumbent's actual vote six of
eight times. The only exceptions were Ronald Reagan in 1984 and
George H.W. Bush in 1992, and then by only 0.2% and 0.7%
respectively. On average, Gallup's projection of the incumbent's
vote has averaged 1.3 percentage points greater than the actual
result. Obviously, without seeing the raw results we can only
speculate, but this pattern suggests that Gallup has allocated
too many of the undecided over the years to incumbents.
It is quite clear that incumbent Presidents receive very few
undecideds from the final polls until the final result. Considering
this, Molyneux offers one way to track the Presidential race:
Think of it this way: The percentage that Bush receives in polls
represents his ceiling of support; he may get a little less, but
won't get more. In contrast, Kerry's percentage represents his
floor, and he will almost certainly do better on election day.
Assuming that Ralph Nader and other minor candidates will
receive about 2 percent -- which is what current surveys suggest
-- 49 percent becomes the critical line of demarcation in this
election. If Bush can get to 50 percent or above in the polls,
he should be able to win. At 49 percent -- where he is today --
we're probably looking at another photo finish, lots of
recounts, and narrow state-by-state victories dictating the
Electoral College outcome. And below 49 percent, Bush is almost
certain to lose.
The bottom line? It
appears pretty conservative to assume
that undecideds will probably split 2-1 in favor of the challenger. If
Kerry builds a good case with Americans, he can potentially get the
vast majority of the undecideds. 2.7
ASIDE: Telephone polling -- What about Cell Phone Owners? Recently,
a Jimmy
Breslin column in Newsday questioned the relevance of land-line
telephone polling today, given the profusion of cell phone use.
Electoral-Vote and John Zogby weigh in on this. Here's
Electoral
Vote:
How do pollsters
decide who to call? The simplest way to get a random sample is to
pick the area code and exchange to be sampled and have the computer
dial the last four digits at random. In the trade this is called RDD--Random
Digit Dialing. It is very easy to do but unfortunately has some
problems. Many numbers will be business phones, hospitals, police
stations, teenagers, and other undesirables. Also, with number
portability, the pollster may be getting Manhattan, KS instead of
Manhattan, NY. Consequently, RDD is not used much any more.
Instead, pollsters
buy lists of phone numbers. In some states, the government sells
voter lists. In other states, lists of residential customers can be
purchased from telephone companies. Commercial companies also sell
lists. There are many sources. These are commonly used. Pollsters do
not want to call cell phones for several reasons: First, cell phone
users may be in meetings, restaurants, cars, or other places where
they are not free to talk. Second, since cell phone users are
usually paying for air time, they don't want to waste it and might
break off half way, resulting in a useless interview. Third, that
number in Colorado the pollster called may actually be a student who
took her cell phone to college in California. Fourth, sound quality
may be poor and questions or answers may be misunderstood.
Fifth, and most
important, it might cost the pollster a lot of money: automated
calling of cell phones is illegal in the United States and there is
a substantial fine for breaking the law. US Code Title 47, Chapter
5, Subchapter II, Part I, Sec. 227 reads:
It shall be
unlawful for any person within the United States--
(A) to make any call (other than a call made for emergency purposes
or made with the prior express consent of the called party) using
any automatic telephone dialing system or an artificial or
prerecorded voice--
(iii) to any
telephone number assigned to a paging service, cellular telephone
service, specialized mobile radio service, or other radio common
carrier service, or service for which the called party is charged
for the call.
On the positive side,
pollsters are exempt from the "Do not call" list, but most
people do not know that.
But using land lines
is no bed of roses either. Many people use caller ID or their
answering machine to do call screening. Busy young professionals are
rarely home from work before 11 p.m. whereas lonely old people are
only too happy to talk to the nice young girl who seems to care
about what they think. Calls made at 2 p.m. are going to oversample
housewives, and so on. All these effects lead to biases. To correct
for them, pollsters conduct exit polls of voters leaving the polling
place on election day to get a good idea of the statistical make up
of the electorate for next time. These data are used to correct the
polls. For example, if the exit polls show that 10% of the voters in
some state are African Americans and in a state poll of 600 people
by accident only 30 are African Americans (5%), the pollster can
just count each African American twice. This process is called
normalization. It was proposed for use to correct undercounts in the
2000 census but was rejected by the courts on the gronds that the
constitution calls for an "enumeration" of the population,
not a statistical model of the population.
...
Now getting back
to Breslin's
column, in theory he is right that pollsters will miss people
who have only a cell phone, but of the 169 million cell phones, most
have a land line as well. It is estimated that 5% of the population
is cell only. And most of these don't live in battleground states.
And then an effect occurs only if cell-only users differ from land
line users in their political preference. Thus the error introduced
by missing the cell-only customers is probably smaller than the
error introduced by missing the overseas voters. But it is there and
in a close election, it could matter. Here is Zogby's response
to Breslin's column. Robert
Landauer wrote a good piece on the accuracy of polling is his
Aug. 31 column. Worth a look.
More from Zogby's response:
First of all, I still
conduct telephone polls. The reality is that polling on the
telephone is becoming more difficult; caller id and the widespread
use of cell phones are affecting response rates. That said, I
feel that representative samples can still be achieved on the phone.
Second, I stand by
both my telephone and interactive results. I have yet to see
evidence that the situation has gotten to the point where telephone
surveys are unusable, and I am equally confident that my interactive
surveys have reached a point where they are valid.
Third, cell phones do
pose a problem for the polling industry, but not to the level Mr.
Breslin feels. It is illegal for polling firms to call cell
phones, coupling that with the rapidly increasing rate of cell phone
use and the gradual decrease of land lines, the polling industry
will face a crisis within a decade. For now, the 170 million
cell phones are largely duplicates and triplicates of landlines.
Also, many of the people Mr. Breslin cites as missing because of
cell phones, are notoriously difficult to each, no matter the
circumstance.
To clarify, my
statements to Mr. Breslin were aimed at pointing to growing problems
in the industry. As an industry, we must adapt to the future
or face extinction, because the telephone will not always be a valid
method of conducting random samples.
The bottom line? None really (at least
right now). The polling industry has to keep pace with technology. 2.8
WHAT MATTERS: The Electoral College! Needless
to say, National Polls are useful more to identify broad trends and
situations where one or the other candidate has a commanding lead. In
close races such as this year's what really matters are the
state-by-state results which determine the electoral college
apportioning of the Presidential vote. Although
the website Electoral
Vote is relied upon by many to predict the race, I prefer
Chris Bowers' approach since his
methodology, I believe, is better sanitized to remove the kinds of
errors I have highlighted earlier in this page. Chris' basic
assumptions are outlined here:
The page I most often refer to to
assess the state of the Electoral Vote is Chris' General
Election Cattle Call on the Swing
State Project.
3. RECENT POLL RESULTS ADJUSTED
FOR FLAWS
3.1 Gallup
Let's start with Gallup
and Ruy Teixeira (9/17/04) (bold text is eRiposte emphasis):
Here are Bush's leads
in the three national polls released before Gallup's current poll
(no RV data available for DCorps and Harris; Pew and Harris matchups
include Nader):
Democracy
Corps, September 12-14 RVs: +1
Pew
Research Center, September 11-14 RVs: tied
Harris
Interactive: September 9-13 LVs: -1
Looks like a tie ball
game, right? But according to the
Gallup poll conducted September 13-15 and released today, Bush
is up......13???
Let's just say I'm
just a wee bit skeptical of this one. First, Gallup's poll only
includes one day (the 15th) these three other polls do not, so it
can't be Gallup's survey dates that explain the big Bush lead.
Second, this 13
point lead is an LV figure and, as I've repeatedly emphasized,
Gallup's LV screening procedure produces completely untrustworthy
measures of voter sentiment this far in advance of the election.
...
Throwing out the
Gallup LV data, then, let's move on to their RV result: an 8
point Bush lead. Obviously pretty far off the results of the
other contemporaneous polls summarized above, but....could be I
suppose.
But then there's
this: the Gallup internals show Kerry with a 7 point lead among
independent RVs. Huh? Kerry's losing by 8 points overall, yet
leading among indenpedents by 7. How is that possible? Only if there
are substantially more Republicans than Democrats in the sample.
That suggests that
reweighting the sample to reflect the 2000 exit poll distribution
(39D/35R/26I) would give a different result. It does: the race then
becomes dead-even, instead of an 8 point Bush lead. (Note: Steve
Soto of The Left Coaster got Gallup to give him their party ID
distributions for this poll and confirms a 5 point Republican party
ID advantage in their RV sample.)
One final note: I
mentioned the Pew Research Center poll had the race dead-even just
like the reweighted Gallup data. And what was Pew's party ID
distribution in their RV sample? You guessed it: a 4 point lead
(37-33) for the Democrats, just like in the 2000 exits.
I think we've finally
found out how to make these polls get along!
Chris Bowers has more
on "Gallup's Shame", where he points out:
It is pathetic and
unacceptable for a "non-partisan" polling firm to be
produce the outlying poll in favor of Bush in fourteen of its last
sixteen polls. The odds of this happening at random are around one
in 14,000. Considering those odds, the far more likely explanation
for all these outliers is that Gallup's polling methodology is
inherently structured in favor of Bush. Whether or not it is
intentional, I do not know. However, I do know that Gallup's polls
are connected to the largest news outlets in America of any poll,
both in terms of print (USA Today is the largest circulation
newspaper in the country) and cable news (CNN has more viewers than
Fox, they just watch for shorter periods of time). I also know that
sensational headlines sell. I further know that Gallup's
chairman is a Republican donor.
This is a shameful
state for the oldest and most respected polling organization in the
country. Shame on you Gallup.
UPDATE on
9/20/04:
Via Ruy
Teixeira, I see that John
Harwood has an article out in the Wall Street Journal trying to
understand the discrepancies in the polling results. Gallup's
Editor-in-Chief of Polling is quoted here defending Gallup's Likely
Voter model:
"We're open to any
scientific evidence that would point to our modifying our likely-voter
model," responds Frank Newport, editor-in-chief of the Gallup Poll.
Mr. Newport says so far he hasn't seen any.
Well, could there be more nonsense from
the head of a firm that once had a good reputation? I'm glad Harwood
points this out in the very next sentence (bold text), which elicited a
weak defense from Newport.
In 2000,
Gallup's election-eve sample of likely voters showed Mr. Bush leading by
two percentage points over Al Gore. Its registered-voter sample, showing
Messrs. Bush and Gore neck and neck, was closer to the actual Election
Day results. But Mr. Newport notes that in 1996 the likely-voter
model more accurately forecast the size of Bill Clinton's victory over
former Senate Republican Leader Bob Dole.
Ha ha! Newport gets the "scientific
evidence" he asked for and still clings to their discredited
methodology based on 1 data point. Why discredited? Well, as Teixeira
has pointed
out:
In 3 of the last 4 presidential
elections (including the last one), Gallup's final RV reading was
actually closer to the final result than their final LV reading!
3.2 CBS/New York Times
Here's Teixeira
on the CBS/New York Times poll (9/17/04):
CBS
News/New York Times Poll Has It Close to Even!
Well, that is if you
weight their data to conform to the 4 point Democratic party ID lead
which we have good reason to believe is the underlying distribution
in the voting electorate. As many have already heard, the
new CBS News/New York Times poll, conducted September
12-16, gives Bush an 8 point lead (50-42) among RVs--but also gives
the Republicans a 4 point edge on party ID. Reweight their data to
conform to an underlying Democratic 4 point edge (using the
39D/35R/26I distribution from the 2000 exit poll) and you get a
nearly even race, 47 Bush/46 Kerry.
Nearly even. That
goes along with the the 46-46 tie in the Pew Research Center poll
(which gave the Democrats a 4 point edge on party ID without
weighting) and the 48-48 tie in the Gallup poll (once weighted to
reflect an underlying Democratic 4 point edge). Not to mention the
two other recent national polls (Harris, Democracy Corps) that show
the race within one point.
Perhaps all this is
just a coincidence, but the pattern seems striking. Once you adjust
for the apparent overrepresentation of Republican identifiers in
some samples, the polls all seem to be saying the same thing: the
race is a tie or very close to it.
Jerome Armstrong has
more at MyDD:
On
the heels of Gallup.
Another poll that says there's been a 5-10% partisan swing toward
Republican turnout for 2004, with what proof? Regarding the NYT/CBS
poll, it's composed of 36% Republicans and 31% Demoratic voters.
However, that is not representative of the actual
turnout:
Democrats Republicans Independents
1992 34% 34% 33%
1996 39% 34% 27%
2000 39% 35% 26%
And from Atrios,
on the NYT/CBS Poll results. It includes a question asking who the
respondent voted for in 2000.
Answers:
Gore - 28
Bush - 36
Buchanan - 1
Nader - 1
Voted, won't say - 1
Didn't vote - 32
There's something
happening here, and what it is aint exactly clear. One thing is
clear, if you look at the poll numbers prior to the
Republican-weighted sampling done by Gallup and CBS/NYT's the race
is even.
Bush 36 Gore 28 Didn't Vote 32! What a
sample! Good grief.
UPDATE 9/20/04:
Via Ruy
Teixeira, I see that John
Harwood has an article out in the Wall Street Journal where he
points out that the large Bush lead in the latest CBS/NYT poll is
because of the high percentage of Republicans in this poll:
Last
week's CBS sample, in a mirror image of Pew's, contained four
percentage points more Republicans than Democrats. Because this
polarized contest has left roughly nine in 10 adherents of each
party supporting its nominee, such variation in the number of
Republicans and Democrats surveyed has an unusually large impact on
polling outcomes.
In a
close race, in fact, that can make the difference between an
apparent dead heat and a solid lead for one candidate. If the CBS
and Pew surveys are adjusted to reflect comparable numbers of
Republicans and Democrats, their results would have been virtually
identical.
Indeed
that's precisely what liberal polling analyst Ruy Teixeira did on
his Web log, called Emerging Democratic Majority. As the New York
Times report of the poll carried the headline "Bush Opens
Lead," Mr. Teixeira's blog declared, "CBS News/New York
Times poll has it close to even."
3.3
Other Polls: ABC, Fox, IDB, Newsweek Chris
Bowers of MyDD has, as usual, done some astonishing work examining
the history of Party ID in polling and applied recent Party ID stats
to polls that do not currently weight by Party ID - such as ABC, Fox,
IBD and Newsweek. No prizes for guessing the result! Here's Chris (and
he starts here with the CBS/NYT poll to show how Kerry's position has
actually improved if you fix the Party ID ratios), and bold text is
eRiposte emphasis:
On
September 9th, CBS News released results from its post-RNC poll, conducted
September 6th-8th, that was very good news for Bush, and very
bad news for Kerry. In fact, with the possible exception of the June
Harris poll, this was the worst poll for Kerry this entire
election cycle. In a poll that had a Party ID weighting of 33.5%
(354) Democrats, 32.1% (340) Republicans, and 34.5% (365)
Independents, John Kerry was quite simply not where he needed to be
in order to win the election. Kerry polled at only 80% among
Democrats, while Bush polled at 90% among Republicans. Even more
worrying for Kerry, Bush led 48-39 among independents. While not
overwhelming, this poll clearly showed a Bush lead. Even if the
Party ID figures were re-weighted in this poll to reflect either
1996 or 2000 turnout, Bush still led by five points.
This week, the
CBS poll, conducted from September 12th until September 16th,
showed the exact same overall result: Bush leading Kerry 50-42 among
registered voters in a two-way trial heat. On the surface, this may
not seem like good news for Kerry. It may even appear as though
Bush's convention bounce is sticking. However, this poll was
actually very good for Kerry and, as we shall see, highly unusual
because it showed real movement. While Bush leads Kerry by
exactly the same 50-42 margin among registered voters, among all
three different Party ID groups, Kerry actually improved his
position. Among Republicans, Bush fell from 90 to 87. Among
Democrats, Kerry moved up to 83 from 80. Among independents, Kerry
went from nine points down to being down by only a single point,
43-42. While the overall results look exactly the same as those form
last week, CBS
admits to having a potentially spoiled sample...What
happened was that the 33.5% Democrat, 32.1% Republican, 34.5%
Independent Party ID sample changed to a 36% Republican, 32%
Democrat, 32% Independent Party ID sample. Kerry's gains within the
Party ID groups were wiped out by a shift of the relative importance
of the Party ID groups within the poll as a whole.
...
Two polling
firms, Harris and Pew, have conducted Party ID surveys over a span
of thirty-five and seventeen years respectively...
Pew's 2004 survey was
conducted in June and July, and contained over 19,000 people in
their sample (under 0.5 MoE). Although I do not know for certain, it
is reasonable to assume that their previous surveys were conducted
among similar sample sizes. Since 1990, Republican self-identifiers
have varied between 27 and 30 percent of the population, except once
in the immediate post-9/11 survey when they rose to 31%. Democratic
self-identifiers have varied between 30 and 34% of the population,
except once, immediately before 9/11, when it rose to 35%. In eight
of the sixteen surveys since 1990, self-identifying Democrats made
up 33% of the population. This chart does not just show slow
movement in Party ID--it shows glacial movement.
...
In 2003, [the
Harris] poll included more than 6,000 registered voters, and thus
has a slightly higher Margin of Error than Pew, but the MoE is still
under plus or minus one. According to Harris, since 1990
Republicans have varied from between 28 and 33% of the population,
while Democrats varied from between 36 and 38% of the population
before dropping to 34 and 33 the last two years respectively. In
those two years, Republicans also dropped while Independents / Not
Affiliated / Other saw an increase. Further, with the exception of
2002, the Democratic edge in Party ID ranged between 5 and 9
points--a mere four-point swing. Overall, this chart also shows
extremely gradual movement, never once in a year seeing the same
kind of movement CBS recorded in a single week. Considering this,
there is simply no circumstance under which the shift recorded by
CBS, a poll with a Margin of Error of plus or minus, 3.5 can be
understood as accurate.
Party ID is indeed
a variable rather than an absolute constant. However, even over
periods of years in surveys with miniscule margins of error, it is a
slow moving, incrementally shifting variable. Considering this, from
the perspective of week to week polling in the final two months of a
Presidential election, in order for such polling to truly gauge the
state of the campaign, it should be treated as a constant. Given
this, it is my position that polls that weight their results
according to a fixed Party ID model offer a more accurate picture of
the campaign than those that do not weight their results by Party ID
(or other demographics).
...
I have gathered
together Party ID data from twelve recent polls. In six of the
polls, the Economist,
Harris,
ICR,
Pew,
Rasmussen
and Zogby,
the samples were weighted to fit demographic and / or previous
turnout models.
...
The similarity
between the results in these six polls is remarkable. The race
varies from Bush up four to Kerry up one, with no two polls
disagreeing about Bush's raw score by more than three points or
Kerry's raw score by more than four points. On average, Bush leads
by less than two points (47.3-45.5).
For the sake of
comparison, I have also been able to track down the internals of six
recent polls that do not weight their results according to Party ID
or other demographics: ABC,
CBS, Fox,
Gallup,
IBD
/ CSM and Newsweek.
...
These polls
show significantly more variance than the other six. But what is
perhaps even more remarkable is the variance they show among
themselves.
...
Interestingly,
had Party been weighted in the six most recent unweighted polls,
they would look almost exactly like the six recent weighted polls:
Table Five
* = likely voters
Bush Kerry Date
ABC 48 47 9/8
CBS 47 46 9/16
Fox* 46 47 9/8
Gallup 48 48 9/14
IDB 47 47 9/12
News 46 47 9/10
Yikes!
Bottomline?
Unweighted polls over short time periods are largely useless, if there
are significant shifts in the Party affiliation ratios of those
polled.
4. CONCLUSIONS AND LESSONS FOR
KERRY SUPPORTERS As you can
tell from the data presented here, it is very clear that polls
currently showing huge Bush leads have deep methodological flaws
(either from an LV standpoint, RV standpoint, or both). The fact
of the matter is that nationally, properly weighted polls show the
race to be essentially close and dead even*. This is evident in Chris
Bowers' always useful General
Election Cattle Call as of 9/17/04. Lessons
for Kerry supporters? 1.
Stop the suicide watch and the hand-wringing. 2.
Stop panicking over every other poll. Do not trust the
mainstream/conventional media or their nonsensical reproduction of
bogus polling data. Remember this?
October 27, 2000
...
Today's CNN/USA
Today/Gallup tracking poll continues to give George W. Bush an
advantage over Vice President Al Gore...
While not a
prediction of the voters' choice in November, Friday's results
show Bush garnering 52 percent of the vote and Gore drawing 39
percent. The survey of 851 likely voters was conducted October
24-26 and has a 3.5 percentage point margin of error.
A CNN/Time poll also
released today gives Bush a 49 percent to 43 percent edge over Gore,
which is statistically in agreement with today's CNN/USA
Today/Gallup tracking poll, given the polls' margin of sampling
error.
However, do contact the media and keep correcting
them as politely as possible, for the nonsense they publish or show
on TV - whether it is parading the Swift
Boat Liars for Bush or ignoring Bush's
countless flip-flops (both staples at Faux News and CNN) or
building nonsensical stories using flawed polls, etc. 3.
Keep your
eye on the ball: Participate in Get-Out-The-Vote (GOTV) efforts
and donate to the DNC. 4.
Stop complaining incessantly about Kerry's every move (or lack
thereof). Kerry knows
what he is doing. He is just up against the most powerful and corrupt
political machinery in the history of the United States and a
mainstream media that is largely compliant with the corruption or
willing to play he-said-she-said to propagate GOP fakery unchallenged
to the electorate (not to mention
talking heads and columnists who are largely inept or corrupt - read
the Daily Howler
if you want proof or visit this
page). 5. Liberal Oasis has some
additional advice:
Unfortunately,
Kerry's own messages were hemmed in by the mind-numbing
stupidity of the Beltway Dems, whining
to the press about strategy throughout the long weekend.
Instead helping with
strategy by amplifying Kerry's messages.
(See LO
on 4/28 for the most recent time this page called on whiny
Beltway Dems to shut up.)
For the last time
(this year) Establishment people: this is all-out combat, and we
have one leader.
You have a problem
with the campaign, then tell it to the campaign.
Telling the media is for
your ego. Not for the cause. Not for the country.
There are 56 days
left. Get your damn game faces on, and get behind the big guy.
4.1 Some
relevant Election 2000 History
a. Remember this?
October 27, 2000
...
Today's CNN/USA
Today/Gallup tracking poll continues to give George W. Bush an
advantage over Vice President Al Gore...
While not a
prediction of the voters' choice in November, Friday's results
show Bush garnering 52 percent of the vote and Gore drawing 39
percent. The survey of 851 likely voters was conducted October
24-26 and has a 3.5 percentage point margin of error.
A CNN/Time poll also
released today gives Bush a 49 percent to 43 percent edge over Gore,
which is statistically in agreement with today's CNN/USA
Today/Gallup tracking poll, given the polls' margin of sampling
error.
b. Don't forget that Gore was
"trailing" Bush in the polls in October 2000, literally
until Election Day:
The following charts show the difference between the
votes in favor of Gore vs. those in favor of Bush in the polls conducted
before the 2000 election. Note that the votes corresponding to
"Other/Unsure" are excluded - so keep those in mind, as well
as the error bars, when you interpret the data. My comments on the data
appear below the charts.
5. A DEAD-EVEN RACE:
FEEDBACK TO SENATOR KERRY AND THE DEMOCRATIC PARTY
Kerry and the DNC of course need to
translate the dead-even race to a win for Kerry - this is going to be a
challenge especially if Bush springs an "October Surprise" or
two. But, I will not try to
second-guess the Kerry camp's strategy at this point (as Liberal
Oasis has wisely advised) since they know the internals of all these
polls and the pulse of the electorate better than anyone else.
What is clear is that although Kerry seems to be drawing ahead of
Bush when it comes to Independents, the percentage of Democrats who
plan to vote for him seem smaller than the percentage of Republicans
who plan to vote for Bush. In other words, it appears Kerry
needs to shore up his base substantially and motivate them to come out
to the polls on Election Day, which makes GOTV efforts all the more
critical. (There is at least some
evidence that Kerry's lead among women voters, traditionally the
Democratic base, has slipped - the latest Pew poll shows his lead down
to 5% amongst women, down from 10% in August).
As I wrap up this piece, there is
one piece of advice for the Kerry campaign though. They HAVE to replace their absolutely
inept media representatives (who only add to the damage
caused by the inept and lazy talking heads themselves) with
competent ones. The nonsense from some of their representatives
appearing on TV has gone on far too long and the Kerry campaign is making
Kerry's life more difficult by not choosing the right people to appear
in these shows - people who can
clearly rebut/refute GOP talking points/spin/lies and articulate
Kerry's position/platform rather than question it.
6.
APPENDIX: Why do "leading" media outlets continue to
publish stories using highly questionable results?
Well, for the same reasons that they
continue to "stand by" highly discredited or fake stories on
other topics. For self-promotion and to protect their
"reputation". No surprise here. I'll
let Steve
Lovelady at the Columbia Journalism Review's Campaign Desk handle
this through the example of the CBS/NYT poll:
Campaign Desk took
a look at this subject -- when polls go bad -- yesterday, but
today's brazen display by The New York Times compels us to
revisit it.
The Times
brings us yet another poll measuring public preferences in the
campaign for the presidency (following four other wildly-varying
polls this week), and decides that this one deserves play as its lead
story on page one.
Why?
As nearly as we can
tell, the only "why" is this: The Times itself
(along with CBS) commissioned this particular poll.
A little history:
Earlier this week, a Pew Research Center poll taken between
September 8 and September 10 gave President Bush a whopping 15-point
lead among likely voters. Then on Thursday, Pew reversed itself by
announcing that a poll taken from September 11 to September 14 found
Bush leading among likely voters by all of one point. And a Harris
Interactive poll released the same day found the presidential race
essentially tied. Further compounding matters, a Gallup poll
released the same day gave Bush an eight-point lead among registered
voters and a 13-point lead among likely voters.
Finally comes the new
Times-CBS poll, indicating Bush has a nine-point lead among
likely voters.
Got it? It's Bush by
15 points; no, make that one point; oops, call it a dead heat; no,
it's actually a 13-point lead; or is it nine points?
To its credit, The
Times does run a sidebar
by Carl Hulse inside the paper, offering up pollsters' explanations
as to why nothing they say makes any sense. Nonetheless, there The
Times is, offering up column one of page one to tout its own
poll.
Running a sidebar on
page A10 that says that all of these contradictory polls should be
taken with a grain of salt the size of a bowling ball, but still
using your lead story to shill for the particular poll that you
happened to pay for, just doesn't cut it.
This is not
journalism; it's self-promotion, and we say to hell with it.
[eRiposte emphasis]
Amen.
FOOTNOTE
added on 9/23/04:
Sometimes I use the words
dead even or "tie" or "tied" in this article to
represent a "tossup". In hindsight I realize I should have
used the word tossup, but I don't think "tie" or "dead
even" is too far off the mark.
|