Methodology
How These Polls Were Conducted
Here are the key things to know about this set of polls from The New York Times, The Philadelphia Inquirer and Siena College:
• We spoke with 4,097 registered voters from April 28 to May 9, 2024.
• Our polls are conducted by telephone, using live interviewers, in both English and Spanish. Nearly 95 percent of respondents were contacted on a cellphone for this poll.
• Voters are selected for the survey from a list of registered voters. The list contains information on the demographic characteristics of every registered voter, allowing us to make sure we reach the right number of voters of each party, race and region. For this set of polls, we placed nearly 500,000 calls to about 410,000 voters.
• To further ensure that the results reflect the entire voting population, not just those willing to take a poll, we give more weight to respondents from demographic groups underrepresented among survey respondents, like people without a college degree. You can see more information about the characteristics of our respondents and the weighted sample at the bottom of the page, under “Composition of the Sample.”
• When the states are joined together, the margin of sampling error among registered voters is plus or minus 1.8 percentage points. Each state poll has a margin of error ranging from plus or minus 3.6 points in Pennsylvania to plus or minus 4.6 points in Georgia. In theory, this means that the results should reflect the views of the overall population most of the time, though many other challenges create additional sources of error. When computing the difference between two values — such as a candidate’s lead in a race — the margin of error is twice as large.
If you want to read more about how and why we conduct our polls, you can see answers to frequently asked questions and submit your own questions here.
Methodology
The New York Times/Philadelphia Inquirer/Siena College poll in Pennsylvania and the Times/Siena polls in Arizona, Georgia, Michigan and Nevada were conducted in English and Spanish on cellular and landline telephones from April 28 to May 9, 2024. In all, 4,097 registered voters were interviewed. When all states are joined together, the margin of sampling error is plus or minus 1.8 percentage points for all registered voters and plus or minus 1.9 percentage points for the likely electorate.
The margin of sampling error for each state poll is plus or minus 3.6 percentage points in Pennsylvania, plus or minus 4.2 points in Arizona, plus or minus 4.5 points in Michigan, Nevada and Wisconsin, and plus or minus 4.6 percentage points in Georgia.
The Pennsylvania poll was funded by a grant from the Lenfest Institute for Journalism. The poll, which was designed and conducted independently from the institute, includes a deep look at voters in the Philadelphia suburbs using a statistical technique called an oversample. The results are weighted so that in the end, the poll properly reflects the attributes of the entire state and is not biased toward those voters.
Sample
The survey is a response-rate-adjusted stratified sample of registered voters on the L2 voter file. The sample was selected by The New York Times in multiple steps to account for the oversample of the Philadelphia suburbs, differential telephone coverage, nonresponse and significant variation in the productivity of telephone numbers by state.
The L2 voter file for each state was stratified by statehouse district, party, race, gender, marital status, household size, turnout history, age and homeownership. The proportion of registrants with a telephone number and the mean expected response rate, based on prior Times/Siena polls, were calculated for each stratum. The initial selection weight was equal to the reciprocal of a stratum’s mean telephone coverage and modeled response rate. For respondents with multiple telephone numbers on the L2 file, the number with the highest modeled response rate was selected.
Fielding
The samples for each state were stratified by party, race and region and fielded by the Siena College Research Institute, with additional field work by ReconMR, the Public Opinion Research Lab at the University of North Florida, the Institute for Policy and Opinion Research at Roanoke College, and the Center for Public Opinion and Policy Research at Winthrop University in South Carolina. Interviewers asked for the person named on the voter file and ended the interview if the intended respondent was not available. Overall, 94 percent of respondents were reached on a cellular telephone.
The instrument was translated into Spanish by ReconMR, and Spanish-speaking interviewers were assigned to the modeled Hispanic sample. Bilingual interviewers began the interview in English and were instructed to follow the lead of the respondent in determining whether to conduct the survey in English or Spanish. Monolingual Spanish-speaking respondents who were initially contacted by English-speaking interviewers were recontacted by Spanish-speaking interviewers. Overall, 19 percent of interviews among self-reported Hispanics were conducted in Spanish, including 21 percent in Nevada and 17 percent in Arizona.
An interview was determined to be complete for the purposes of inclusion in the ballot test questions if the respondent did not drop out of the survey by the end of the two self-reported variables used in weighting — age and education — and answered at least one of the age, education, race or presidential election ballot test questions.
Weighting — registered voters
The survey was weighted by The Times using the R survey package in multiple steps.
First, the samples were adjusted for unequal probability of selection by stratum.
Second, the five state samples and the samples of the the Philadelphia suburbs and the rest of Pennsylvania were weighted separately to match voter-file-based parameters for the characteristics of registered voters by state.
The following targets were used:
• Party (party registration if available, else classification based on a model of vote choice in prior Times/Siena polls) by modeled L2 race (except Wisconsin)
• Age by gender (self-reported age, or voter file age if the respondent refuses)
• Race or ethnicity (L2 model)
• Education (four categories of self-reported education, weighted to match NYT-based targets derived from Times/Siena polls, census data and the L2 voter file)
• White/nonwhite by education (L2 model of race by self-reported education, weighted to match NYT-based targets derived from Times/Siena polls, census data and the L2 voter file)
• Marital status (L2 model)
• Homeownership (L2 model)
• State regions (NYT classifications by county or city)
• Turnout history (NYT classifications based on L2 data)
• Vote method in the 2020 elections (NYT classifications based on L2 data)
• History of voting in the 2020 presidential primary (Pennsylvania and Wisconsin only, NYT classifications based on L2 data)
• Census block group density (Arizona only, based on American Community Survey five-year census block-group data)
• Census tract educational attainment
In Pennsylvania, the samples from the Philadelphia suburbs and the rest of the state were combined and adjusted to account for the oversample of the Philadelphia suburbs.
Finally, the six state samples were balanced to each represent one-sixth of the sum of the weights.
The sample of respondents who completed all questions in the survey was weighted identically, as well as to the result for the general election horse race question (including leaners) on the full sample.
Weighting — likely electorate
The survey was weighted by The Times using the R survey package in multiple steps.
First, the samples were adjusted for unequal probability of selection by stratum.
Second, the first-stage weight was adjusted to account for the probability that a registrant would vote in the 2024 election, based on a model of turnout in the 2020 election.
Third, the five state samples and the samples of the Philadelphia suburbs and the rest of Pennsylvania were weighted separately to match targets for the composition of the likely electorate. The targets for the composition of the likely electorate were derived by aggregating the individual-level turnout estimates described in the previous step for registrants on the L2 voter file. The categories used in weighting were the same as those previously mentioned for registered voters.
In Pennsylvania, the samples from the Philadelphia suburbs and the rest of the state were combined and adjusted to account for the oversample of the Philadelphia suburbs.
Fourth, the initial likely electorate weight was adjusted to incorporate self-reported vote intention. The final probability that a registrant would vote in the 2024 election was four-fifths based on their ex ante modeled turnout score and one-fifth based on their self-reported intention, based on prior Times/Siena polls, including a penalty to account for the tendency of survey respondents to turn out at higher rates than nonrespondents. The final likely electorate weight was equal to the modeled electorate rake weight, multiplied by the final turnout probability and divided by the ex ante modeled turnout probability.
Finally, the six state samples were balanced to each represent one-sixth of the sum of the weights.
The sample of respondents who completed all questions in the survey was weighted identically, as well as to the result for the general election horse race question (including leaners) on the full sample.
The margin of error accounts for the survey’s design effect, a measure of the loss of statistical power due to survey design and weighting. The design effect for the full sample is 1.33 for registered voters and 1.5 for the likely electorate. The design effect for the sample of completed interviews is 1.38 for registered voters and 1.53 for the likely electorate.
Historically, The Times/Siena Poll’s error at the 95th percentile has been plus or minus 5.1 percentage points in surveys taken over the final three weeks before an election. Real-world error includes sources of error beyond sampling error, such as nonresponse bias, coverage error, late shifts among undecided voters and error in estimating the composition of the electorate.
