Why do pollsters keep getting conservative or even liberal victories wrong recently?

1,351 Views | 14 Replies | Last: 4 yr ago by calbearinamaze
wifeisafurd
How long do you want to ignore this user?
Morrison Wins in Australian Election, Confounding Pollsters https://nyti.ms/2JoTemB

Forget the policy issues, Morrison, Trump, Netanyahu (actually the polls were all over the place), Brexit, Orban , Italy (though again polling was all over the place) etc. The other thing is that the polls don''t seem to work to well with populists like Trump or even liberal candidates e.g., Ocasio-Corttez over Crowley.

I admit wrong oills are not new (Dewey beats Truman), but you would think that polling has become much more scientific to keep getting it wrong so much lately.

An English academic perspective on polling issues:

The Twilight of the Polls? A Review of Trends in Polling Accuracy and the Causes of Polling Misses http://bit.ly/2LT6NNn

There are people here who know this stuff (I don't). Thoughts?

Another Bear
How long do you want to ignore this user?
I don't trust polls any longer. If I wanted to go full tin foil hat, I'd say they've been hacked as well...but no way I can prove it, let alone how. Perhaps it's just the general chaos currently going on..but distrust in polls would be a typical authoritarian move, makes it easier to gaslight people.

Also the Aussies had their elections interfered with by the Russians in 2018 and there's evidence Russia was involved in Brexit. A search will give plenty of results. Seems the Brits are in bigger denial than the U.S. about the Russians.

I don't think it's a coincidence all those authoritarian pols got elected. Trump, Morrison, Brexit, Orban, Bebe...and the first three also had Russian involvement.

So here's the question, how does Russia benefit from authoritarian or extreme conservative types in office around the world? The quick and easy answer is dictators like each other and support each other, and it's easier for the Russians go get what they want. Look at Trump and the kowtowing to Putin, the Helsinki debacle, groveling. That is NOT how an American president acts.

Here's the thing, while the western industrial nations have done very well with economy and technology, moving forward, the Russians have not. On that note, I think the Russians are still fighting the Cold War like earth might be destroyed while the U.S., UK and Australia have moved on.

That's all I got...but I'd guess Russian involvement. They benefit from the disruption of democracy.
sycasey
How long do you want to ignore this user?
I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.
wifeisafurd
How long do you want to ignore this user?
sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.


sycasey
How long do you want to ignore this user?
wifeisafurd said:

sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.
I mean, all of these articles basically lay out reasons for why polls have margins of error, they just don't call it that. It's always been an inexact science.
B.A. Bearacus
How long do you want to ignore this user?
I have not researched the topic much, but the move to people having cell phones as their primary or only number (and regulations specific to cell phones) and the scourge of robocalls (I don't pick up at all unless you're in my contacts) has to have different effects across demographics.

https://www.politico.com/story/2019/02/27/phone-polling-crisis-1191637
wifeisafurd
How long do you want to ignore this user?
sycasey said:

wifeisafurd said:

sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.
I mean, all of these articles basically lay out reasons for why polls have margins of error, they just don't call it that. It's always been an inexact science.
so what you are saying then is with all the problems with polls these days, the margin of error is growing? Yes? Not challenging, just trying to understand...
sycasey
How long do you want to ignore this user?
wifeisafurd said:

sycasey said:

wifeisafurd said:

sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.
I mean, all of these articles basically lay out reasons for why polls have margins of error, they just don't call it that. It's always been an inexact science.
so what you are saying then is with all the problems with polls these days, the margin of error is growing? Yes? Not challenging, just trying to understand...
I don't know that it's actually growing. Again, these polls were already close in the first place, so a result that goes "against" polls that only predict a 51-49 margin isn't that big an error.

I tend to follow Nate Silver on stuff like this. This is why his model tends to just give percentage chances of likely results, not definitive statements.





And also his protege Harry Enten (read the whole thread):


calbearinamaze
How long do you want to ignore this user?
Finally, OMG polls are not perfect. Stop treating them like something carried by Moses. They are tools. Seems to me the Australian polls pointed to a tight finish overall... and that's what occurred.

Etten sums it up well about polls and, by extension, surveys in general. Thanks for the post.




If you believe in forever
Then life is just a one-night stand
If there's a rock and roll heaven
Well you know they've got a hell of a band
wifeisafurd
How long do you want to ignore this user?
sycasey said:

wifeisafurd said:

sycasey said:

wifeisafurd said:

sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.
I mean, all of these articles basically lay out reasons for why polls have margins of error, they just don't call it that. It's always been an inexact science.
so what you are saying then is with all the problems with polls these days, the margin of error is growing? Yes? Not challenging, just trying to understand...
I don't know that it's actually growing. Again, these polls were already close in the first place, so a result that goes "against" polls that only predict a 51-49 margin isn't that big an error.

I tend to follow Nate Silver on stuff like this. This is why his model tends to just give percentage chances of likely results, not definitive statements.





And also his protege Harry Enten (read the whole thread):



Even Esten mentions the problem of non-responsive voters, but if the polls showed a 2% difference that isn't much.

As for the few points comment, there are the last polls on the eve of the 2016 election:

Economist: Clinton plus 4
NY Times/CBS. Clinton plus 4
ABC/Wash. Post Clinton plus 4
Fox. Clinton plus 4
Monmouth U. Clinton plus 6
Lucid: Clinton plus 5
Insights West Clinton plus 4

And the two outliers:

Rasmussen: Clinton plus 2
IBD: Trump plus 1

The average of major polling on election eve was Clinton over 4%. That is not a few points. In fact, Mr. Silver went on TV and said the polls before the election "didn't pass the common sense test." (He was around Rasmussen at 2%). So I think he is inferring that the 7 major polls above were modeled incorrectly.

In fairness, he also noted that on a state by state basis, his polls showed Clinton's leads had eroded in key battle ground states and that the election was a toss-up, but no one was listening. Particularly other pollsters and pundits. I had not heard Silver speak before, and he is an interesting guy.
sycasey
How long do you want to ignore this user?
wifeisafurd said:

sycasey said:

wifeisafurd said:

sycasey said:

wifeisafurd said:

sycasey said:

I haven't followed it closely, but I believe the polls had the Australian election close, meaning it could go either way. Same with the US election in 2016 and with Brexit. The issue is that people in the media need to understand why "margin of error" exists.

Trump: Pew says non-response bias.
Why 2016 election polls missed their mark http://pewrsr.ch/2fmNGH6

Aussies: herding bias apparently

How did pollsters get the Australian election result so wrong? https://www.newscientist.com/article/2203837-how-did-pollsters-get-the-australian-election-result-so-wrong/#.XOOIu6mFpE8.twitter

Brexit and Scottish National Referendums: various modeling problems

Pollsters now know why they were wrong about Brexit https://www.businessinsider.com/pollsters-know-why-they-were-wrong-about-brexit-2016-7?utm_source=twitter&utm_medium=referral&utm_content=topbar&utm_term=desktop&referrer=twitter via @businessinsider

None of the experts have it down to margin of error. One trend that keeps getting mentioned is that voters now screen calls and don't respond to pollsters and certain groups (e.g. less educated) are underrepresented.
I mean, all of these articles basically lay out reasons for why polls have margins of error, they just don't call it that. It's always been an inexact science.
so what you are saying then is with all the problems with polls these days, the margin of error is growing? Yes? Not challenging, just trying to understand...
I don't know that it's actually growing. Again, these polls were already close in the first place, so a result that goes "against" polls that only predict a 51-49 margin isn't that big an error.

I tend to follow Nate Silver on stuff like this. This is why his model tends to just give percentage chances of likely results, not definitive statements.





And also his protege Harry Enten (read the whole thread):



Even Esten mentions the problem of non-responsive voters, but if the polls showed a 2% difference that isn't much.

As for the few points comment, there are the last polls on the eve of the 2016 election:

Economist: Clinton plus 4
NY Times/CBS. Clinton plus 4
ABC/Wash. Post Clinton plus 4
Fox. Clinton plus 4
Monmouth U. Clinton plus 6
Lucid: Clinton plus 5
Insights West Clinton plus 4

And the two outliers:

Rasmussen: Clinton plus 2
IBD: Trump plus 1

The average of major polling on election eve was Clinton over 4%. That is not a few points. In fact, Mr. Silver went on TV and said the polls before the election "didn't pass the common sense test." (He was around Rasmussen at 2%). So I think he is inferring that the 7 major polls above were modeled incorrectly.

In fairness, he also noted that on a state by state basis, his polls showed Clinton's leads had eroded in key battle ground states and that the election was a toss-up, but no one was listening. Particularly other pollsters and pundits. I had not heard Silver speak before, and he is an interesting guy.



Remember that those national polls are basically measuring the national popular vote, which Clinton DID win by about 2 points. So the error was about 2, which is pretty normal. The problem for her was how that affected the Electoral College, it left her open to a Trump squeaker in a few swing states, and that is what happened. Silver definitely saw this as a possibility, which is why I've kept following his commentary and left behind some of the other polling analysts who were way too confident (Sam Wang, for example).
OneKeg
How long do you want to ignore this user?
Were the 2016 polls you listed for the popular vote? If so, Clinton won that by 2+% so many of them were within a couple points, no?

To be clear, I'm not trying to re-litigate the 2016 election. I do think polls should try to do a better job or accounting for local complexifying rules like the US electoral college. Which I think Silver did try to do when 538 gave Trump a 1 in 4 chance, and Silver kept saying over and over around election-time that while he didn't think it was the likeliest outcome, it could absolutely happen.
calbearinamaze
How long do you want to ignore this user?
Economist: Clinton plus 4

NY Times/CBS. Clinton plus 4
ABC/Wash. Post Clinton plus 4
Fox. Clinton plus 4
Monmouth U. Clinton plus 6
Lucid: Clinton plus 5
Insights West Clinton plus 4

And the two outliers:

Rasmussen: Clinton plus 2
IBD: Trump plus 1


I wasn't going to come back in, but sycasey on onekeg made very good and, quite frankly, basic points
about the polls being about the national election.

Also, be very careful about the term "outlier". There many quite legitimate ways to test for outliers.
Please believe me, I don't have the energy right now to check the margin of error (MOE) for all of the
polls. It turns out...all I had to check on was the Rasmussen. The MOE was+/-2.5% The bumps
the upside statistically to 4.5%. As far as Ramsussen being an outlier.....basically game over.


If you believe in forever
Then life is just a one-night stand
If there's a rock and roll heaven
Well you know they've got a hell of a band
wifeisafurd
How long do you want to ignore this user?
BEARUPINDC said:

Economist: Clinton plus 4

NY Times/CBS. Clinton plus 4
ABC/Wash. Post Clinton plus 4
Fox. Clinton plus 4
Monmouth U. Clinton plus 6
Lucid: Clinton plus 5
Insights West Clinton plus 4

And the two outliers:

Rasmussen: Clinton plus 2
IBD: Trump plus 1


I wasn't going to come back in, but sycasey on onekeg made very good and, quite frankly, basic points
about the polls being about the national election.

Also, be very careful about the term "outlier". There many quite legitimate ways to test for outliers.
Please believe me, I don't have the energy right now to check the margin of error (MOE) for all of the
polls. It turns out...all I had to check on was the Rasmussen. The MOE was+/-2.5% The bumps
the upside statistically to 4.5%. As far as Ramsussen being an outlier.....basically game over.



I meant outlier from the other polls which had Clinton wining with 4% or more. Not a statistical outlier. Considering the way the election turned, I can see SYC's point about only being 2% off on the popular vote.
calbearinamaze
How long do you want to ignore this user?
I meant outlier from the other polls which had Clinton wining with 4% or more. Not a statistical outlier. Considering the way the election turned, I can see SYC's point about only being 2% off on the popular vote.

Of course. I knew what you meant.

The Rasmussen poll is not a statistical outlier. I mentioned there are many ways of
testing for outliers. I left out "depending on the situation at hand" I'm using a relatively simple way.

In this case, with an MOE of +/-2.5%, the upside boundary of 3.5% I MADE A MISTAKE, So, I will
assume...that the 4% for Clinton polls also had MsOE 2.5% or so...putting their lower boundaries
at 2.5%...covering the the Rasmussen easily.

MsOE are extremely important and not well understood by the general public. That is one reason why articles
... except in rare cases... never mention them. "A leads B, but that's within the margin of error so really
...statistically A doesn't lead B" is not real sexy headline.

How can I lack energy and be writing this in the middle of the night? At some point, I'll send you
a PM. Hint: it's because something extremely sad has occurred in my life.
.


If you believe in forever
Then life is just a one-night stand
If there's a rock and roll heaven
Well you know they've got a hell of a band
calbearinamaze
How long do you want to ignore this user?
I made another mistake.

In putting their lower boundaries at 2.5%...covering the the Rasmussen easily. the 2.5 should be 1.5.

If you believe in forever
Then life is just a one-night stand
If there's a rock and roll heaven
Well you know they've got a hell of a band
Refresh
Page 1 of 1
 
×
subscribe Verify your student status
See Subscription Benefits
Trial only available to users who have never subscribed or participated in a previous trial.