Polling a ‘Foreign Tribe’

As the US presidential elections wind their way towards a painful and tortuous conclusion, I’ve been thinking a lot about the difference between reporting on a survey that offers a snapshot in time, as opposed to using polls, and past history, to predict what the results might be in a few days, weeks, or months. My colleagues and I have done lots of surveys over the last eight years in Colombia, India, Mexico, Morocco, Nigeria, and the USA. In all this work, we offered estimates of how many people in a given population engaged in this or that behavior, supported this or that policy, or thought positively/negatively about a particular organization or institution. There is always a confidence interval around every one of those estimates influenced in large part by the number of valid responses. This allows you answer questions like, “what percentage of the population living in the Moroccan cities of Casablanca or Rabat supported this or that political party in May 2015?” Or, “what is the statistical association between attending mosque and believing that women’s rights are human rights?” If I wanted to ask how many people were likely to vote for a specific Moroccan political party in the future, however, I’d have to do so much more than extrapolate from the previous work I’d done with my colleagues. I’d want to have more polls over time, so that I could track trends, and I’d want to figure out a way of predicting who would be likely to actually participate in the election, assuming that participation was voluntary. In the US, pollsters did not do as good a...
How Telecom Mechanisms Have Impacted Polling, Part Two

How Telecom Mechanisms Have Impacted Polling, Part Two

Some populations are inherently difficult to survey. As a result, their voices rarely figure into studies of public attitudes, financial or medical needs, or consumer preferences. The most overlooked populations include people without access to the Internet; immigrants, refugees, and persons internally displaced within their own countries; homeless persons; and people living in low-income, repressive, or war-torn countries.  Some organizations have made significant steps towards addressing this problem, and one such group is RIWI, a publicly-traded company based in Toronto, Canada. In the intermediate future, my company, Azimuth Social Research, hopes to partner with RIWI on a cross-national survey project.  RIWI’s Random Domain Intercept Technology (RDIT) invites web users worldwide to participate in surveys in their own language. RIWI bypasses national or sub-national web restrictions by leveraging its ownership, or temporary control, of hundreds of thousands of unused, expired, or abandoned web domains. As RIWI explains, web users often stumble across one of these unused domains while surfing the Internet. Often, this stumble takes users to a “404” or similar error page, but when users run across one of RIWI’s sites, they are instead invited to take part in a quick, anonymous survey. RIWI’s system auto-detects the user’s geographic location, which allows them to offer the survey in a relevant language and to geolocate (whenever possible) the respondent’s position.  RIWI’s method is useful in part because it circumvents government restrictions, as no other agency has a list of RIWI-controlled domains. The RIWI domain inventory, moreover, is constantly changing. As a result, RIWI’s survey work is particularly useful in countries such as China, where the government keeps tight control over...

From Qual to Quant and Back

I started my research career as a research consultant for Human Rights Watch in Israel and the Palestinian territories. My “data” came from what qualitative research:  lengthy, face-to-face interviews with knowledgeable people such as politicians, lawyers, human rights activists and ordinary citizens who have either witnessed, or personally experienced, interesting events. (There are other kinds of qualitative research, of course – focus groups, ethnography, journal entries, and more). Interview-based research is physically and emotionally demanding. It requires building a network of contacts to gain access to the right people; persuading those people to trust and speak with you, often for hours; engaging in emotionally intense, face-to-face discussions; carefully building trust and rapport; and then analyzing piles of typed transcripts, scribbled notes, or taped conversations. You either furiously write during the interview while also trying to listen, engage, and come up with new questions, or desperately try to remember as many details as you can to record in writing later. Sometimes, you are lucky enough to tape the conversation and even hire someone to produce a written transcript. Then, however, you are faced with hundreds of pages to wade through. Throughout the 1990s and early 2000s I did interviews of this sort in multiple countries. Often, I used a translator, which adds complexity to the rapport-building exercise but also gives you more time to take notes during the interview, a research companion to talk about the work, and someone to help you navigate the process of finding the right people to speak with. My first translator was a Palestinian journalist, Walid Battrawi, now an important media figure in Ramallah, capital...
Surveys: Police Critics are not Anti-Capitalists

Surveys: Police Critics are not Anti-Capitalists

By James Ron and Doug Guthrie After protests swept through Kenosha, Wisconsin, we thought it was worth asking: Do supporters of Black Lives Matter (BLM) seek more than police restructuring? Do they actually want to replace the capitalist system with something else more akin to socialism? This is a meme out there, but the facts belie the meme of the moment. “Black Lives Matter” has become a civil justice Rorschach test today. For some, its hate-signaling leftist, antifa ideology. For others, it’s an aggressive effort to destroy the very principles the United States was founded upon: freedom, democracy and capitalism. Even Rudy Giuliani, the former mayor of New York City and President Trump’s personal attorney, has gotten into the name calling, describing the protests as a Marxist plot. There is some logic to the question of what’s behind BLM, as many of the fires set during demonstrations in Kenosha and elsewhere have targeted businesses and commercial buildings. In response, some corporations, including Apple, Target and Walmart, temporarily shut down their stores this spring. Is the call to “defund the police” really shorthand for “end capitalism?” To answer this question, we reviewed survey evidence from roughly 13,000 respondents in the United States and around the world. The data suggest that, on average, police critics in this country and elsewhere are in fact not opposed to business. Here and in other countries, the respondents who told us they mistrusted the police the most were also more likely to trust international and local businesses alike. Across the world, support for business goes hand in hand with skepticism towards police. This finding undermines...
How Telecom Mechanisms Have Impacted Polling, Part One

How Telecom Mechanisms Have Impacted Polling, Part One

Gallup, the Pew Research Center, and other US survey firms have historically employed all manner of strategies to gather vast amounts of opinion data. While mail-in and door-to-door polling were standard in the early- to mid-1900s, the 1980s saw an explosion in telephone surveys. According to Pew, telephone survey responses in the United States sat at a comfortable 36 percent in 1997.  Over the past few decades, however, pollsters in this and other wealthy countries have been forced to confront a drastic decline in the public’s willingness to pick up the phone and answer survey questions. From 1997 to 2012, average response rates in the United States dropped by 27 percent and now stand shakily at a minuscule 9 percent. Add to this the rising cost of using telephones to conduct surveys, and it makes sense that most polls in the U.S. and other wealthy countries are now conducted online. The advent of the Internet has dramatically shaped how pollsters do their work. People are still willing to answer questions online, particularly if they have already “opted in” to a pre-existing survey panel. Once substantial internet-based data is gathered, however, it becomes more difficult to compare with past data collected via telephone, as online and phone survey responses differ.  One of the key issues is how to identify respondents and non-respondents. If you can figure out who is systematically more likely to answer an Internet survey, you can get a better sense of how representative your online survey really is.  Response to internet questions is driven, first and foremost, by rates of web users worldwide. In the United States,...
Global Reputation of the Belt and Road Initiative, Part One

Global Reputation of the Belt and Road Initiative, Part One

In 2013, the Chinese government established an infrastructure and economic development plan for countries in the Asia Pacific region, Africa, and Central and Eastern Europe. In theory, the plan’s ultimate goal, dubbed the Belt and Road Initiative (BRI), is to empower poorer countries to enter the global market via the development of “belt” (overland roads and railways) and “road” (maritime and sea routes) infrastructure. Critics, of course, fear that BRI is a Chinese plan to gain more global power, supplant Western financial institutions, and exert direct and indirect control over these newly constructed and vital assets.    Foreign trade network development is not new for China; almost 2,000 years ago, the Han Dynasty established the famous Silk Road, which connected Central and Southern Asia’s populations and stretched into Europe. For years, China managed trade routes that stretched for thousands of miles. However, interference by the Crusaders and Mongolian tribes resulted in economic isolation for many countries positioned along the Silk Road, and that isolation has persisted across the centuries.  Reception of China’s Belt and Road Initiative has been mixed, with some countries praising the effort and others fretting over its potential long-term implications for the balance of power, global trade, and the debt burden of poorer countries.  How Do Participating Countries Feel About the Belt and Road Initiative? As of 2020, China has signed contracts with 137 countries and 30 international groups. While many of those countries are located in Asia, several African and Eastern European countries have also signed on. The Initiative has even stretched west, covering countries in the Caribbean and South and Central America. Altogether, two-thirds of...