The political pundit class was jolted into a bit of a frenzy this morning after the release of the New York Times/Sienna’s latest presidential poll, which showed Donald Trump leading Joe Biden nationally by 5 percentage points among registered voters and 4 percentage points among likely voters.
Sadly, that is not the only issue with this poll and maybe not the worst one. The NYT also violated standard pollling procedures by not asking the horse race question first which likely biased the respondents against Biden. The reason I say it is worse is a bad polling sample will happen 5 times out of 100 even if a pollster is doing everything correctly. Messaging up the question is deliberate and, given that they changed the question ordering for this poll compared to their previous poll in December, really looks like the NYT was trying to produce the result, not find the result
If it were "weighted," then the results of the urbanicity crosstabs should not match the results of the poll. Under no circumstances should the crosstabs (which match the topline when unspooled) have 35% rural voters. That's insanity. It also does not make sense for the results of a poll showing Biden dropping by a relative 8 points, in relation to 2020, to be as follows:
Urban: No Change (Biden +22 in 2020)
Suburban: No change (Biden +2 in 2020)
Rural: Trump improves by 18 (Trump +15 in 2020)
I have no problem with crosstab RESULTS looking funky, because they are small sample sizes. However, a poll that has 35% of rural voters in the SAMPLE should be discarded, before you even look at what the results show. If rural voters make up 35% of the poll respondents, yet the crosstabs MATCH the results, that would either imply that the weighting was done VERY poorly to allow for an 84% increase in rural voters, or that they forgot to weigh the poll, which is also bad.
Sadly, that is not the only issue with this poll and maybe not the worst one. The NYT also violated standard pollling procedures by not asking the horse race question first which likely biased the respondents against Biden. The reason I say it is worse is a bad polling sample will happen 5 times out of 100 even if a pollster is doing everything correctly. Messaging up the question is deliberate and, given that they changed the question ordering for this poll compared to their previous poll in December, really looks like the NYT was trying to produce the result, not find the result
You share the specific data points but can you show a link to the actual poll as well? Others may want to do some investigation themselves.
The poll does say that it is weighted for Metropolitan areas. You just have to scroll a wee bit further down the page.
There's a problem with that -
If it were "weighted," then the results of the urbanicity crosstabs should not match the results of the poll. Under no circumstances should the crosstabs (which match the topline when unspooled) have 35% rural voters. That's insanity. It also does not make sense for the results of a poll showing Biden dropping by a relative 8 points, in relation to 2020, to be as follows:
Urban: No Change (Biden +22 in 2020)
Suburban: No change (Biden +2 in 2020)
Rural: Trump improves by 18 (Trump +15 in 2020)
I have no problem with crosstab RESULTS looking funky, because they are small sample sizes. However, a poll that has 35% of rural voters in the SAMPLE should be discarded, before you even look at what the results show. If rural voters make up 35% of the poll respondents, yet the crosstabs MATCH the results, that would either imply that the weighting was done VERY poorly to allow for an 84% increase in rural voters, or that they forgot to weigh the poll, which is also bad.