Where Did You Get This Number?

A Pollster's Guide to Making Sense of the World

Price may vary by retailer
BUY FROM:
BUY FROM SIMON & SCHUSTER:

About The Book

CBS News’ Elections and Surveys Director Anthony Salvanto takes you behind the scenes of polling to show you how to think about who we are and where we’re headed as a nation.

As Elections and Surveys Director for CBS News, it’s Anthony Salvanto’s job to understand you—what you think and how you vote. He’s the person behind so many of the poll numbers you see today, making the winner calls on election nights and surveying thousands of Americans. In Where Did You Get This Number? A Pollster’s Guide to Making Sense of the World, Salvanto takes readers on a fast-paced, eye-opening tour through the world of polling and elections and what they really show about America today, beyond the who's-up-who’s-down headlines and horse races. Salvanto is just the person to bring much-needed clarity in a time when divisions seem to run so deep.

The language of polling may be numbers, but the stories it tells are about people. In this engaging insider’s account, Salvanto demystifies jargon with plain language and answers readers’ biggest questions about polling and pollsters. How can they talk to 1,000 people and know the country? How do they know the winner so fast? How do they decide what questions to ask? Why didn't they call you? Salvanto offers data-driven perspective on how Americans see the biggest issues of our time, from the surprising 2016 election, to the shocks of the financial crisis, the response to terrorism and the backlash against big money. He doesn’t shy away from pointing out what’s worked and what hasn’t. Salvanto takes readers inside the CBS newsroom on Election Night 2016 and makes readers rethink conventional wisdom and punditry just in time for the 2018 midterms. He shows who really decides elections and why you should think about a poll differently from the forecasts popularized by Nate Silver and others.

Where Did You Get This Number? is an essential resource for anyone interested in politics—and how to better measure and understand patterns of human behavior. For any American who wants to get a better read on what America is thinking, this book shows you how to make sense of it all.

Excerpt

Where Did You Get This Number?
Chapter One

The Seen and the Unseen

At 2:52 a.m. on Election Night at the CBS News Decision Desk, I reached for the intercom button with my left hand and told the broadcast’s control room we were ready.

What had once seemed unlikely was now certain.

I was staring at big monitors filled with numbers, at vote reports coming in from Pennsylvania: the polls had closed seven hours ago and Donald Trump and Hillary Clinton were still less than a point apart. We weren’t really sure those reports were finished, but now we knew we’d seen enough of them; enough to know for sure what was happening.

Figuring out just what enough is, that’s always the hard part on nights like this.

With my right hand I pulled up a small menu on the screen and then a little gray box that’s been deliberately tucked away, in a spot where it can only be pushed on purpose. It has the “W” in it—for “Win.”

“We’re calling Pennsylvania for Trump,” I said into the headset. My click signaled graphics to light up the state in red on the big U.S. map, it sent the state’s electoral votes over into the Republican column, and pushed Trump’s count over the 270 he needed—which is why we’d been so focused on Pennsylvania for the last hour. I need to be right on every call, but I really need to be right on the one that decides everything. “. . . and projecting the presidency with it,” I followed. “Donald Trump, elected.”

The name “Decision Desk” is newsroom slang for what’s really a long U-shaped set of tables behind the anchors here in CBS’s Studio 57, the hub of our Election Night coverage. If you watch the show you can see those tables stacked with computers and the monitors that we’re squinting at all night, as data feeds stream in with vote results from every corner of the country. There’s a big lighted sign hanging over it that reads “Decision” and from here my small cadre of pollsters and professors and I call the races for the network: who’s leading, who’s trailing, who wins, and who loses across fifty states, hundreds of contests, 130 million votes. There were times in past years when the Desk was hidden out of sight in a different room with no cameras—let alone any big lighted signs—and known only by its proclamations of who won. But I love that we’re out here in the main studio where the viewers can see and hear from us as the night rolls along. These are, after all, their votes we’re looking at.

In Studio 57 there is no offstage: the anchors sit at a round glass table in the center; there are producers around the perimeter typing and talking softly but urgently; robotic cameras whirling remotely and black-clad cameramen with Steadicams pointing from place to place; giant screens, floor to ceiling, are alight with the red and blue vote map; live streams with remote views of cheering crowds at some campaign headquarters and sunken faces at others; displays flashing Twitter feeds scrolling past too fast to read.

For eight straight hours that had all been a swirl of kinetic energy but now, at this moment, nearly 3 a.m. and the presidency decided, it slows a bit and the focus shifts elsewhere: Donald Trump is at a podium in New York addressing his supporters.

I took off the headset and stepped back from the desk. After we call a presidential race, I always try to take a moment and recognize the history we’ve just seen, whoever wins. On the exposed-brick wall next to us, the set designers had laid out old tchotchkes, campaign memorabilia, photos, all juxtaposed against our technology and those screens, reminders that the Decision Desk has a great history of its own. I have one shot of Walter Cronkite in 1968 in front of a board showing Richard Nixon’s and Hubert Humphrey’s vote counts; right next to my seat is another black-and-white photo of pinstripe-suited CBS staffers in the 1950s frantically tallying votes in chalk on a big blackboard. These are the traditions we follow even as the politics and the technology change through the years. The dress codes change, too, it occurs to me: those guys in the photo are still dutifully wearing their suits even amid the chalk smears, and at this point I’m in rolled-up shirtsleeves, jacket off, tie down. I just pull the tie back up when I go on camera beside that big touchscreen to explain what’s going on—which I’d done more of on this night than any before.

Tomorrow would be busy for me, too. Everyone would be trying to understand how Donald Trump had defied the expectations: the ones set by the pundits, the forecasters, and some—though hardly all—of the polls. We’d just seen some of the answers as all those votes had poured in. At the Decision Desk, our job is to show you what’s happening when you can’t see everything; when you don’t know every vote, or every county, or every person. But the numbers and the winners we light up are just the attention-grabbing parts at the end. The real discoveries come in trying to figure those numbers out in the first place.

This story begins with how we do that on Election Nights, and what this very late, very close one showed us.
Sunday, November 6, two days before Election Day
Two days earlier, on the weekend before Election Day, we’d gathered in the studio to rehearse scenarios that might unfold that Tuesday night, kicking around ideas of what we’d need to see to make a call in every state, and what we’d say on air in each instance. We’d run all the anchors and producers through scenarios for both a Trump win and a Clinton win; what states might flip, what the timing might look like in either case.

In the newsroom someone asked me as we worked through the Trump scenario: “Why does The New York Times say Clinton is going to win?” The Times wasn’t actually declaring her a winner at that point, of course. But the forecasts running on the paper’s website, which were trying to predict the contest in advance, did seem to us to be overconfident in Clinton, offering up assessments of her chances that most people would mistakenly interpret as certainties. On our Face the Nation program that Sunday, we’d released our own final round of pre-election polls from the states of Ohio and Florida showing they were moving in Trump’s direction. The national polls also had him closing the gap. And if all that was moving, other states we hadn’t polled might be changing, as well. It couldn’t be ruled out.

I’d run into our chief White House correspondent, Major Garrett, a few days before that in the Green Room, the waiting area before we went on CBS This Morning. Major had spent the year covering the Trump campaign. I was planning to go on and describe a tough-but-doable Electoral College path for Trump that ran through the states of the Upper Midwest (which later turned out to be the one he took) or maybe Colorado (which he didn’t). I’d been describing Trump as “down, but not out.” I started bouncing that idea off Major, but he beat me to it. “He could win,” he said emphatically, noting the size and enthusiasm of Trump crowds he’d seen in every venue at every hour of the day. I nodded. Neither of us thought that meant Trump would win, but we agreed the possibility was there.
12:00 Noon, Election Day, Midtown Manhattan
My Election Day began well away from the Broadcast Center and the Decision Desk. Fifteen hours before I made that call for Trump, long before the studio heated up and we started the broadcast, I’d gotten to see an early indicator of what might happen. Just after noon I’d gone into the TV networks’ Quarantine room, at an undisclosed location in Manhattan—essentially a borrowed, nondescript office space with no windows—where a handful of representatives from each of the participating networks can go to privately see early exit poll data while voting is still going on, hours before it will be made public.

The exit polls are the first way we get a look at what’s unfolding on Election Day. Thousands of interviewers had fanned out across the country from the opening of voting places that morning, some in place by 6 a.m., heading out on behalf of the TV networks to thousands of precincts. They were handing out questionnaires to voters leaving those polling places: single sheets of paper with large-type font, our TV logos printed across the top along with “CONFIDENTIAL” in capital letters, and a handful of questions arrayed below like “In today’s election for President did you just vote for:” then checkboxes along with the candidates’ names, and “When did you finally decide for whom to vote?” “What was most important in your decision?” and all ending with boldfaced instructions to fold the paper and drop it in a box. When all was said and done they’d collect more than 100,000 of these questionnaires.

At this point in the afternoon, those interviews with voters had just begun. The interviewers transmitted the first set of the morning’s initial data from the interviews, then went back and kept interviewing. My counterparts at the other networks and I all want to see those first reports to get a jump on preparing stories, but because they’re so preliminary, and because people are still voting across the country, we need to make it as private as possible. So it’s only piped into the Quarantine room (dubbed the “Q-room” for short) where there are no Internet ports, no Wi-Fi, and as you enter you turn in all your electronics to a guard at the door, and agree that you can’t leave until 5 p.m. You even get escorted to the bathroom down the hall, complete with our it-doesn’t-get-old joke that someone who wanted to leak the numbers might hide a cell phone behind the toilet, like Michael Corleone with the gun in The Godfather.

Sitting in the Quarantine room, in my read of the morning’s data, the presidential contest was effectively even. That first round of data suggested Clinton was up, but only narrowly at best, and there was plenty of reason to think the trend was moving toward Trump as the voting went on that afternoon. The question was whether it would keep moving, in enough places, and I suspected it could. Through the afternoon those exit polls are incomplete. Millions of people still had yet to cast ballots and the voting was still open everywhere. The trick for us is to figure out who’s still left out there to vote, who it is we’re not seeing yet among our interviews; which kinds of voters might show up later that night, or who might have even just skipped taking the poll altogether.

I ran some comparisons against the precincts, and noticed people who might still vote were older, whiter, and working-class: just the kind of voters Trump was going after.

The first task in calling races is knowing the difference between the seen and the unseen. If what we’re looking at any moment—in this case, those first exit polls—could be different from the rest of them, we wait. We don’t know the whole story yet.

I left the Quarantine room after 5 p.m. and made my way across town to the Broadcast Center. When I got there at 5:30 all the producers and anchors had gathered in the conference room for a quick briefing as we prepared to go on the air live. “This,” I told everyone, “is a contested race.” And I told them to get set for a very late night.

That much, at least, I was sure about.
7:30 p.m., CBS Decision Desk, Studio 57
The show is on the air now. The big U.S. map waits to be filled in, state by state, with red and blue for the winners as we call them.

When I was a kid I remember staying up late in front of the TV on Election Nights like this, but I didn’t watch for the politics, at least not at first. I was initially drawn in because I liked sports, and to me all this was being presented in the same sort of way as a big game. They showed stats and numbers on the screen, that U.S. map like a scoreboard; there were winners and losers, and it all looked like the Super Bowl of Serious Things. I remember thinking it was cool that it all happened despite its enormous, national scope—in fact, it was amazing because of its scope.

As it turned out, I was wrong about Election Night being like sports.

A game unfolds play by play, and events and scores come sequentially until time’s up. But tune in on Election Night and you’re watching the results from events that are over. When we talk about a state, the polls have closed there, all the ballots are sitting in bins or a counting machine and so the result—the voters’ choice—is final. It’s just that nobody knows what it is yet.

It’ll emerge as a mosaic of reports that fill in across the country slowly and randomly, never the same way every year. Voting in the United States is run at the local and state level, and everyone’s little portion of those 130 million votes is, by and large, counted somewhere close to you, with each town or county or state going at its own pace. Knowing that’s going to happen means we try to get a sense of the whole picture from those tiny pieces as they pop up one by one. We look for patterns, commonalities in what’s emerging. We’re not predicting what will be, we’re revealing a story about what is. And my job now, on these nights, is to assemble that picture as fast as I can and relay it to you. That’s part of the tradition we carry, too.

And because there’s a kid out there somewhere whose parents are telling him to go to bed.

•  •  •

The voting closes in North Carolina, a battleground both campaigns are contesting hard, but exactly what’s happening there is uncertain. I look at the statistical models on our screens and they show Clinton is up as the first reports arrive, with a sizable edge, but one we don’t suspect will hold up. These first tallies we’re seeing are from absentee ballots, which were mailed in and dropped off days or weeks ago. They’ve probably been sitting around, queued up first for the tabulating machines at county offices. If they’re first to get counted, they’ll be first to get reported now. From the television screens around the studio, we overhear our reporters at campaign headquarters say the Clinton campaign is confident about them because absentee voters tend to vote more Democratic, are a little younger, but working jobs with shifts and less flexible hours. I’d heard people describing Clinton as having “votes already in the bank” heading into tonight because of that. Meanwhile, though, the exit poll interviews from this afternoon tell us Trump did better with the voters who’d shown up to cast ballots today, and when their votes get counted, we figure Trump could make that Clinton edge shrink, or even disappear. It’s that race caller’s—and pollster’s—rule again: if what you’re looking at right now could be different from everything else, be careful.

Sure enough, within the hour, our models show Clinton’s lead down to two points and shrinking, her “votes in the bank” being quickly devalued as Trump starts to overtake her. Clinton would ultimately lose the state, her edge in advance votes not enough to offset Trump’s turnout surge on Election Day. In the counties where Trump was doing best, turnout was running almost even with what we’d expected, and sometimes higher. In the ones Clinton was carrying, turnout was just a shade under expectations. That, it turned out, was another sign, the start of a pattern we’d need to remember later.
7:58 p.m.
The broadcast gets set to return from a commercial break and the control room asks me for the “top of the hour” rundown: how we’ll characterize the races in states where voting closes in two minutes, at eight o’clock.

Nancy Cordes, our correspondent who’s been traveling with the Clinton campaign, is at the Javits Center in New York City amid a huge crowd clearly expecting to celebrate a presidential win in a few hours.

Nancy reports that the Clinton campaign says they feel very good now about Florida, one of those eight o’clock states, in particular because of Miami and the Hispanic vote there. Another of the assumptions heading into the night was that Hispanic voters would propel Clinton across the fast-growing states of the Sun Belt.

At the Desk, what I see is a different—and bigger—picture.

I tell the control room we’ll describe Florida as a pure toss-up at 8:00, not a Clinton lead. While she is indeed doing well in Miami (it’ll turn out later that she does better than Obama did there). Miami is not a barometer. In fact, no single place is. All our state estimates on Election Night have to take into account everything we can see from all over the state, not just a key county or two—and right now it looks like Clinton is getting beaten most everywhere else in Florida.
8:45 p.m.
We’re starting to light up that U.S. map in red and blue now. We’ve made a string of easier calls in lopsided, partisan states: Trump picks up Alabama and Tennessee. Clinton strings some together in the Mid-Atlantic: Delaware, Maryland, and New Jersey.

At the Desk we blend a range of expertise and we divide up the roles. Statisticians Doug Rivers and Delia Bailey spend much of this night looking for patterns across precincts; Mark Gersh, who does congressional campaign strategy for a living, sits with a red Alabama cap perpetually pulled down to his eyebrows, staring at a long sheet of graph paper with vote numbers printed on it—which I think is just a refresher course for him, because he’s somehow managed to memorize the vote history of every county in America. Steve Ansolabehere knows all the voting systems and David Winston knows key states like Ohio inside out. There are Democrats and Republicans here, but one wouldn’t know it from the discussions. To all of us this night is the same puzzle to solve. The campaigning is over, anyway. My rule, handed down to me from my predecessors, is that everyone needs to agree before we call a race. Our vice president of news standards, Al Ortiz, keeps tabs on what we report, and from time to time the president of CBS News, David Rhodes, swings by, and they ask when we can make a call, but without pressure, and it’s just too early to say.

Seated next to me is Murray Edelman, a consultant for us now, but who more than forty years ago helped build the foundation of how we do things today.

Back when networks began competing to call races in earnest in the 1950s and 1960s, computers—which, Murray reminds me, were so big at the time they could take up an entire room and were fed by punch cards—had come along with advances in statistics that made it both alluring and possible to quickly figure out what was happening on Election Night. The networks were taking up the challenge. They’d even take out full-page ads in the newspapers the day after an election for bragging rights. I have one that lists a long string of vote estimates from every state—in tiny, eight-point type, each down to the decimal point—that their big computers had spit out the night before.

Edelman was a young graduate student in 1967, recruited along with CBS’s new elections head, Warren Mitofsky, beginning an era of innovations that would become legendary in the polling world. Even if I hadn’t seen Murray’s yellowed notebooks from the era, with his handwritten formulas on them, one could tell that so many of our techniques began as his creations, just from the way he describes computations as we test them out, how they’ll get better or worse with every new update, like a parent who knows what will set off or soothe their kid’s moods.

They made sure every precinct in the state had the chance to be included in the models that they watched on Election Night, the ones that show the story—in number form—of what’s emerging. They divided a state into four or five regions and waited for votes from every one of them, like buckets that had to fill up, so that a Decision Desk could know it was accounting for every part and every place. It became a cornerstone of the modern approach. Nowadays, the precincts for exit polls are still picked at random every year; our interviewers go to big places and small ones, to country and city precincts, Democratic-leaning places, Republican-leaning places, and everything in between. No place is picked for an exit poll, or discarded for that matter, just because of how we assume it’ll vote. Because we never know in advance where the big vote shifts might occur, and surprises are the last thing you want.

This is also why I cringe a little when I still hear people point to spots and say things like “watch this little town that always votes for the winner.” The key to any good poll is quite the opposite of that: it’s making sure that everyone gets represented.
9:00 p.m.
We call Kansas for Trump the moment its polls close. Both Michigan and Wisconsin close and, ominously for Clinton, we see tight races, nothing even statistically close to being a “call.” These were both states she’d been expected to win.

The exit polls start telling more of their story, too. The interviewers talk to voters until the voting ends, then they go in and talk to election officials, as well, and get the official precinct results straight from them. We update our models with the counts from the officials, and the small leads Clinton had in some areas of Michigan and Wisconsin shrink even further.
10:00 p.m.
It was sometime around the 10:00 mark that I think the idea of this whole race as a complete toss-up, and perhaps a Trump win, really emerged for the campaigns the same way it had appeared to us at the Desk early on. Major Garrett, reporting from Trump headquarters, says that while earlier this evening there was despondency, now there’s a cautious optimism. From the anchor table, Norah O’Donnell says to viewers, “If you’re a Clinton supporter you’re popping an anti-anxiety pill right now.”

The votes have been coming in from Michigan for an hour now, and our estimates put Trump up there by two. By this time of night, in 2012 or 2008, Michigan had shown a wide lead for Democrats. When the Trump campaign said during the year they’d contest Michigan, they’d drawn some derision. But like a lot of the Upper Midwest, it’s got a higher share of working-class voters now propelling him. John Dickerson, one of our anchors, notes that four years earlier, I’d called not only Michigan much earlier, but Pennsylvania, too, for Democrats. Obama had been well on his way, cruising through states that always voted Democratic. Not so for Clinton tonight.

There’s an even more recent comparison that catches my eye. I look at Michigan’s and Wisconsin’s county maps, where each county was colored in for the candidate ahead in it, now that we’ve got more votes to look at. The pattern looked familiar, like the county maps we had seen that spring, on primary nights, in the Democratic contest between Clinton and her rival for the nomination, Bernie Sanders. Back in the springtime, in those primary races in Michigan and Wisconsin, Clinton had done relatively well in the cities like Milwaukee and Detroit, and Sanders had done well everywhere else—in the outer suburbs, the rural counties. Now in November, on this Election Night, the urban counties were lit again for Clinton, but once again the vast landscape beyond them was lit for her opponent—this time, for Trump. I wondered if that was a holdover, a story unfolding in county map form: out in those rural areas, maybe there were Democrats who never did get past a primary loss for Sanders, deciding to sit this one out.

No one yells at anyone at the Decision Desk; at least not at ours. It’s not like on dramatic shows and in movies about newsrooms where these kinds of nights are portrayed with shouting and people running around, demanding some race be called or not. For one thing, there’s a live broadcast happening ten feet away and sometimes we’ve been reminded to hush, but it’s mostly because the language we use has to be in methodical, operating-room style. There’s no other way to get through it all.

We start looking at Ohio, and whether we can make a call for Trump.

David Winston says: “Franklin County is reporting now. It’s sixty-four percent Clinton.”

Murray Edelman asks: “Any change to the models?” He wants to know if that report changes the overall leader in our estimates of the race, or is it what we’d expected to see in Franklin County, anyhow.

Gersh is already looking and chimes in: “No.”

I ask: “How does the size look?” “Size” is shorthand for turnout, the size of a county expressed in vote terms. This also becomes a check on the quality of our data. If there’s anything out of the ordinary from a county, such as a far bigger or smaller turnout than ever before, like twenty thousand votes reported in a place that holds two hundred people, the system will put up a warning flag, and we’ll want to double-check it.

Doug Rivers: “It’s the same as expected.”

We always use the past vote in a county for Democrats and Republicans as a benchmark for comparison to the moment. The fact that we can do this tells a political story in itself: so many voters are so consistently partisan, that a comparison allows us to pick up a shift in votes that can be telling. In most every state, red or blue, we find cities tend to be more Democratic and Republicans do better in suburban and more rural areas farther out, but it’s how much better or worse they’re doing tonight that matters. A Republican getting 55 percent in counties where Republicans usually get 50 percent is having a good night. A Democrat getting 60 percent in places Democrats usually get 70 is having a bad one.

As we head toward 10:30, Ohio becomes an example of calling a race off a pattern that’s emerging, long before we see all of its votes. In county after county, mostly in smaller rural ones, we see Trump doing better than other recent Republican candidates have done, beating the Republican benchmarks. Winston is keeping an eye on it now and it’s getting close to a “computational call” status, he says, which means we think vote trends we’re seeing are so widespread, not relegated to just a few places, that they can’t be idiosyncratic. Something real is happening.

With about three-quarters of the vote in from Ohio, Clinton is running ahead of a typical Democrat in only six of eighty-eight counties. Trump is ahead of usual Republican benchmarks in all the others, and up in them by ten points on average. We apply that key test: is what we’re seeing so far telling us enough about the whole of the state? Watching all these counties moving toward Trump, one after another, gives us confidence that it is.

I tell the control room we’re at “likely” status to call the race, but stand by, there’s still one last thing to check. Trump is winning in more sparsely populated places; all these counties he’s doing well in have only a relative few votes in each. One by one they’re adding up their small tallies and for Clinton’s chances it is, as I describe it at one point in the night, looking like death by a thousand paper cuts.

But we need to be sure that big cities like Columbus or Cleveland won’t deliver enough votes to push Clinton ahead. These kinds of urban counties have so many voters they can sometimes change a state all by themselves. In fact, Democrats often rely on them to do just that. Tonight Cleveland is counting more slowly than other places. We assume there are about 650,000 votes, and half of them are still uncounted.

We plug in a most-outrageous-case scenario. Clinton is tracking in Cleveland just under what a typical Democrat, or Obama, got in past years, at 67 percent of what’s in so far. So what would happen if she got an astounding 80 percent or even 90 percent? (Which she won’t: I test the extremes to be extra sure before making a call like this.) We figure she’d still lose the state. And besides, the turnout there—as in some other Democratic areas—looks laggard compared to other counties.

I give the control room the call, and Ohio lights up Republican red. We move on from Ohio, but we don’t forget that pattern we saw, the way it favored Trump. We’ll use that later.

•  •  •

People always ask me if I get nervous on these nights, but I’ve learned that anxiety—in this or in anything, maybe—for me comes from not knowing what’s coming next. Our whole job is to know what’s going on, though, and we don’t make a call unless we really believe we do. So I try to use nervousness as a test, asking myself if I think I’ll be nervous after I call this race. What I don’t want is for more votes to come in and make me wonder if it had been a smart call or not, looking back once it’s done and declared. So far on my watch we’ve never had to reverse anything. When you take this job you do your best to protect all that network history and reputation with every call, but it’s important to have a sense of perspective, too. I have colleagues across this news organization covering war and facing real life-threatening danger around the world. My friends and colleagues stand in combat zones and riots and hurricanes. This is just a storm of statistics.

The first time I ever sat at one of these Decision Desks, I had just been hired at CBS, not too long out of graduate school; it was 2002 and my approach was a little more cautious. My then-boss and mentor, Kathy Frankovic, had plunked me down and assigned me the easy races, expected blowouts, and my suggestions for calls—which is all the weight they carried, just suggestions—would then ladder up to her. There was a governor’s race where the incumbent was heavily favored to win, and though he ultimately did, right after poll closing his lead was small. It was my first time trying to find the mix of gumption and calculation that the job demands; even with all the numbers flashing on all those screens, there is no formula for that. I didn’t want to look hesitant. But I wanted everyone to know that as the new guy, I understood the metrics, and that I wouldn’t make calls just based on what I “thought” would happen. Kathy asked if I was ready to call and I recommended no. I got a quick “okay then,” and that was that. I called it a bit later when it made more sense. It turned out no one really cared what time that race got called. She just wanted to see that even under pressure I was thinking it through. That’s still very often just the best, simplest test.
Midnight
Hours pass and neither Trump nor Clinton has won much more; things are still too tight. The Upper Midwest is a large bloc of uncalled states, grayed out on the screen, neither lit up red nor blue. Wisconsin is close, as are Michigan, Pennsylvania.

We come back from a commercial break and I do a live segment at the touchscreen, where I game out possible Trump paths to victory from here, state by state—what might happen if we do call each of them for Clinton or Trump. “He has to win Wisconsin?” one of our anchors, Scott Pelley, asks me about Trump’s prospects.

“He’s got options,” I respond, “but it’s most likely his path goes through Wisconsin,” I say, pointing it out on the state map from among its other unlit neighbors across the Rust Belt.

Minnesota is grayed out, too. Before the night began we had defined the “Battleground” list, the states to watch most closely because they could swing either way. Minnesota wasn’t included. It had voted Democratic in every presidential election since 1972 and we’d assumed it would keep on doing so. Over at a producer’s desk, they pitch a story to the broadcast control room to do a deep-dive look at its exit polls.

“Minnesota isn’t on the Battleground list,” the control room says, surprised that it’s being flagged for attention.

The producer responds, “That’s the point!”
1:00 a.m.
Sometime past 1 a.m. the commentators on set start musing on what a Trump presidency could look like. We haven’t called the presidency, but people were reading their own patterns now just by staring at the national map. We haven’t called either Maine or New Hampshire, two usually Democratic states we would have put in Clinton’s column hours ago had this night been going her way.

I eat another energy bar. Our producer has been kind enough to stash a plastic bag full of them under the desk for me. There was catered food somewhere, but I never left the desk long enough to find it. Murray Edelman’s trying to eat salad—I’m not sure where he got it—though he doesn’t have a fork, and no one is about to cross the studio or leave the screens to get one.

We divvy up the states among the team. We’ve called Nevada for Clinton, and Colorado, too. Our decision models on those never really wavered from pointing to wins for her. They were not like the midwestern states—but then, these western states apparently didn’t have as many of the kinds of voters who were in flux, late deciders. Nor did they show the same kind of patterns that we were watching in the big eastern cities, where turnout in Democratic strongholds was still running well below 2012’s levels. We finally call Maine, too, but these Clinton wins feel like afterthoughts. The presidency still hinges on the Midwest. Bob Schieffer notes that Trump is within range. Trump was 26 electoral votes away. The cameras pan across the map and I hear Scott say “. . . after hours of counting, the votes are so close CBS News has been unable to make a projection.” Our cameras cut to the live shot inside the Javits Center, Clinton’s gathering, an aerial view of the stage, an empty wood podium. At our anchor desk, Norah O’Donnell reports that Clinton’s campaign chair, John Podesta, was headed to that stage to tell the crowd things were too close to call. “Is it too close to call,” anchor Gayle King says, “or not wanting to accept reality?”

•  •  •

We all use that phrase “too close to call” a lot on these kinds of nights. At the Desk the language we’re thinking in is more nerd-speak: “What we want to know is so small and precise that our measurements can’t show it yet.” It’s something every pollster faces whenever you set out to describe what’s going on, though not usually under this kind of time crunch: what kind of error can you live with?

The answer depends on what you need to explain. If we were discussing “favorable” ratings in a regular public opinion poll, no one would parse the difference between a couple of points here and there. But tonight we want to know the winner in what will be a one-point election, so now we need an estimate that’s got even less error than that.
2:00 a.m.
The broadcast went live to Trump campaign headquarters, now full of people in red “Make America Great Again” hats as Major Garrett did his stand-up report. Trump was up a point in Pennsylvania, just over seventy thousand votes out of millions cast. Major says that Trump’s people felt confident they could win it but were waiting for confirmation from the news organizations. That word, confirmation, caught my ear. Certainty is the bar everyone sets for us.

Murray and Mark Gersh parse Pennsylvania and are working through scenarios, trying to get to that sense of certainty. Mark is hunched with that Alabama cap pulled down and no space between its brim and his spreadsheet. In Wisconsin, I see Milwaukee has only three-quarters of the total votes we expected to be cast there, compared to other counties that are showing turnout closer to the way they typically do. That’s a big difference in a tight race. If we’d seen this three hours ago we’d have dismissed it as too unusual, too hard to believe, but tonight we’ve already seen lower turnout from Democrats in other key places in other key states, the just-not-quite-good-enough support that Clinton’s getting.

Seeing consistent patterns elsewhere makes you confident you can fill in the gaps in places that are still unknown. We call Wisconsin for Trump. In the end, when the count was finished, heavily Democratic Milwaukee would cast a lot fewer votes than it did in 2012.

In Pennsylvania, our statistical models still have Trump ahead in the Lehigh Valley, the working-class areas in the eastern part of the state, where coal and manufacturing jobs declined but the new economy hadn’t taken root to replace them. It’s unusual for a Republican to do so well there, but it, too, is consistent with Trump’s campaign and with what we’re seeing tonight. These are the sorts of voters he’s been winning, and only Philadelphia now stands between Trump and the presidency. We start subtracting, trying to figure if votes in the city will change the direction of the state. But while turnout is up statewide in Pennsylvania, turnout in Philadelphia looks down. We’re trying to find out if they’ve actually counted everything there, and we might not otherwise believe that turnout pattern, either, except that it’s another case of what we’ve been seeing all night, in an array of places—especially heavily Democratic places—in state after state. The patterns tell a story. We believe what we consistently see, so consistently now that we didn’t need to see any more. It was enough to know.

I reach for the intercom one more time and make that call for Trump.

•  •  •

Somewhere in the hours following, everyone left, the lights dimmed. Just after 5 a.m., as dawn approached, I was still hanging around at the Desk. I’d been up twenty-four straight hours now, having done a CBS This Morning segment on what was now the previous day. I checked one more time in case any returns had still trickled in from the West Coast, three time zones back, and got ready to leave.

In the coming days I listened to people spurred by the surprises of this night ask about how polling worked, and how we measure things—and whether, in a larger sense, there was something more happening in the country, something they couldn’t have made out from a map or a single number. I thought they were right to be asking these questions. And it made the events of this night feel more like the beginning of a story than the end of one.

I crossed the newsroom on my way out and I wandered past the CBS This Morning producers as they prepped their morning-after show, just to check if they needed a recap from me, maybe to explain what I’d just seen. At first, with their heads down at their keyboards, they half nodded and said maybe, the show lineup was still coming together. Then they looked up at me, and I could tell they were a little taken aback by what must have been my haggard look and the bags under my eyes. They smiled and very kindly said no, thanks. Go get some sleep.

About The Author

Clarke L. Smith/CBS News

Anthony Salvanto, PhD, is CBS News Director of Elections and Surveys. He currently conducts all polling across the nation, states, and congressional races, and heads the Decision Desk that projects outcomes on Election Nights. He appears regularly on Face the Nation, the CBS Evening News, CBS This Morning, and more. Where Did You Get This Number? is his first book.

Product Details

  • Publisher: Simon & Schuster (August 2018)
  • Length: 256 pages
  • ISBN13: 9781501174834

Raves and Reviews

“At CBS News we all knew Anthony Salvanto was the best of the best in polling, but in this book we discover he’s also a great storyteller. Salvanto mixes election night dramas with candid insight into the strengths and weakness of polling and the growing influence that public opinion sampling plays on everything from the products we buy to the candidates we choose for public office. An excellent read for politicians, journalists and voters." —Bob Schieffer, CBS News                                    

"A revealing look at the numbers, how they're derived and interpreted, and how they sometimes fail us. Timely reading for the coming midterm elections." —Kirkus Reviews

“Anthony Salvanto's Where Did You Get This Number? is a witty, approachable, relevant and fun read…Whether you are a political junkie, or someone who uses market research to better understand your customer, or simply want to better understand how Trump beat Clinton, it is must reading. After doing research and surveys for hundreds of clients around the world, I wish all of them had read this book to understand how to turn data into insight and actionable intelligence. My favorite parts include his anecdotes about tasting his grandmother's spaghetti sauce to explain to non-statisticians how you can understand the entire public from just talking to some of them, and how picking the right movie to see with your diverse family is why all averages and percentages are not created equal. The book is a great resource to not only understand surveys and polls, but more importantly, to better understand other people…a real page-turner that's hard to put down.”  —David B. Rockland, Ph.D., retired Chairman and CEO, Ketchum Global Research & Analytics

“My admired friend Anthony Salvanto has written a good book about polling and you had better read it if you want to understand how professionals go about getting the right readings as they take the temperature of our country.”  —Peggy Noonan, Pulitzer Prize winning columnist, The Wall Street Journal

“In his new book Where Did You Get This Number?, Salvanto begins on election night, the events of which surprised him less than they did most Americans. He proceeds, in digestible and timely fashion, to demystify the world of polling and pollsters.” —The Guardian

“Where Did You Get This Number?” is a useful and easy-to-read primer on the basics of modern polling. —The Washington Post

Resources and Downloads

High Resolution Images