Preface

Like many people, I could not get to sleep on November 3, 2020. It was election night in America, and the presidential race between Donald Trump and Joe Biden remained unsettled when I finally turned in for the night.

I am sure many others also had trouble sleeping, not knowing who would be president. Maybe they were excited that their candidate seemed likely to win. Or perhaps they were fearful that the “other guy” might prevail.

On this night, a divided country united on one thing—the frustration of having no certainty about who our next president would be.

I was wide awake for a different reason. As the night wore on, it became apparent that the 2020 polls were performing poorly. Pollsters, some of whom are friends of mine, were taking a beating on cable channels and Twitter. Reporters were misinterpreting the polls. Pollsters were getting defensive. Clouds were forming that seemed likely to threaten the credibility of polling going forward.

It was as if the election had become a referendum on pollsters’ performance rather than the engine of democracy.

I am part of the market research industry. This field uses surveys and polls (and other methods) to guide decision makers in pretty much any organization you can imagine: Fortune 500 companies, nonprofits, even governments. When you see a new advertising campaign, a new product on store shelves, fresh features on products you already use, updated packaging, or an increase in the prices you pay, market research probably informed these changes.

Market research is big business. Estimates vary, but market research spending in the United States is likely in the $20 billion range.2 US market research spending is larger than the whole economies of North Korea or Jamaica.

Pre-election polling is a small but resonant part of the market research field. It is the segment of market research most people know. The US presidential race is polling’s Super Bowl.

It is a well-kept secret that although pre-election polling is the most public-facing aspect of what researchers do, it is hard to make money at polling. Most pollsters conduct polls because it helps build their brands and sell other, more profitable work. Others do so because they enjoy media exposure. Who does not want to be on TV or a podcast discussing their views about the next election?

Polling failures reflect poorly on the broader field of market research. Pre-election polls have failed many times throughout history. They were struggling again in 2020, which is one reason why I could not sleep.

I am the president of a market research firm and have worked in survey research for more than 30 years. I have seen firsthand how the quality of data collected from surveys and polls has deteriorated over the past decade.

Survey research and polling is the only field I can think of where advances in technology over the past 20 years have reduced quality. We are lucky if 5% of those we invite to take a survey take part. We routinely toss a third or more of surveys from our datasets because we catch the respondent cheating.

My insomnia emerged from a worry that the performance of the 2020 polls would be a watershed event, not only for political polling but for the whole field of survey research. My profession is in trouble.

Many of us who work in market research kindled our interest in this field by studying opinion polling. Market research was created as a by-product of polling. It did not take polling’s founders long to realize that polling for companies and brands was lucrative.

Trust in market research is dependent on the success of pollsters. Would recent polling failures herald the decline of a field that I had grown to love?

I had previous bouts with anxiety over pre-election polls. I fretted about the deteriorating quality of polling data after the 2016 election, when every credible pollster in the nation predicted Hillary Clinton would defeat Donald Trump.

Sometimes it is an everyday conversation that can enlighten a problem. My wife, Sue, and I stayed at an Airbnb in Florida in January 2017. The host, Dan, was an ardent Trump supporter. At one point, he asked me what I did for a living. When I replied that I was a survey researcher, our conversation quickly turned to why the polls failed to predict the 2016 election winner.

Listening to Dan, I realized the implications of the failure of 2016 pre-election polling for my company. Dan felt he could now ignore all polls—on issues, approval ratings, voter preferences, and more. Instead, he could trust the leaders he chose to believe in to tell him what America thought.

I found myself getting defensive. After all, the 2016 polls were not off by much. Polls missed by more in 2012 than in 2016, yet nobody seemed to care in 2012. The problem seemed that the 2016 polling errors resulted in an incorrect prediction, and our sensationalized media environment magnified these errors.

We can still trust polls to provide a sense of what the citizenry thinks about the issues, right?

Not according to Dan. He felt political leaders should ignore the polls because polls and pollsters are not trustworthy. Dan was dug into his positions. The damage was done. He would never trust a poll again.

A new term for this has been bandied about: poll denialism. Poll denialism is a refusal to believe any poll results because of polling’s past failures. Survey researchers like me should be scared that this has a name.

Poll denialism is alarming, not just to the market research field but also to our democracy. Poll results are a vital way political leaders keep in touch with the needs of the public. Polls shape public policy. Politicians who ignore or deny them are ignoring public opinion. When the public ignores the polls, it sets a stage for autocracy.

George Gallup once remarked, “Polling is merely an instrument for gauging public opinion. When a president or any other leader pays attention to poll results, he is, in effect, paying attention to the views of the people.”3

Gallup felt opinion polling was a high calling and noble field. Gallup named his master work on polling The Pulse of Democracy.

Market research remains tightly associated with political polling. My clients have not become as mistrustful of their market research as the public has become of polling, but they would be justified in having doubts. At least, they should be asking hard questions of research agencies such as my company.

Most of what market researchers study is more complex than pre-election polling. It is easy to ask someone if they will vote for candidate A or candidate B. Ever try to project the market size for a new groundbreaking product from a survey? Or to tell a marketer what the effect of a 10% price increase will be on sales? Or if it is worth the billion-dollar investment to buy a competitor?

These are complicated decisions that rest on the quality of market research findings. Researchers like me support these decisions every day.

Pre-election polling is the simplest type of survey research. If researchers cannot predict the next president, why should a client believe our market forecasts?

Market research is fantastic. It is a field I would recommend to any college student who has an analytical bent and likes to explore why people behave the way they do. Great researchers beautifully meld psychology, statistics, and business skill. The field abounds with intelligent, caring people.

For me, market research has been an excellent alternative to a career in academia. It is like being an academic researcher who works at 10 times the pace of a college professor on a much broader array of subjects. I have worked for hundreds of organizations in dozens of contexts and have had the sincere privilege of working for and with brilliant people for three decades.

But I am worried to the point of sleeplessness. I am afraid that election pollsters have mortally wounded the field I love. I am nervous that people’s perception of pollsters will harm our democracy. And that pollsters will continue to mismanage the response to the abuse dispensed their way.

I attended many polling events after the past six presidential elections. Each time, leading pollsters convened, shared data, and tried to figure out what happened to improve the polls for the next cycle.

These confabs quickly devolved into posturing, defensiveness, and salesmanship. Each time, they resulted in trade groups authoring white papers that defended pollsters’ performance and made a few recommendations for future methodological tweaks. The result was even worse polls four years later.

Polling is not a field poised to fix itself.

Academics will investigate what is going...