Some populations are inherently difficult to survey. As a result, their voices rarely figure into studies of public attitudes, financial or medical needs, or consumer preferences. The most overlooked populations include people without access to the Internet; immigrants, refugees, and persons internally displaced within their own countries; homeless persons; and people living in low-income, repressive, or war-torn countries. 

Some organizations have made significant steps towards addressing this problem, and one such group is RIWI, a publicly-traded company based in Toronto, Canada. In the intermediate future, my company, Azimuth Social Research, hopes to partner with RIWI on a cross-national survey project. 

RIWI’s Random Domain Intercept Technology (RDIT) invites web users worldwide to participate in surveys in their own language. RIWI bypasses national or sub-national web restrictions by leveraging its ownership, or temporary control, of hundreds of thousands of unused, expired, or abandoned web domains.

As RIWI explains, web users often stumble across one of these unused domains while surfing the Internet. Often, this stumble takes users to a “404” or similar error page, but when users run across one of RIWI’s sites, they are instead invited to take part in a quick, anonymous survey. RIWI’s system auto-detects the user’s geographic location, which allows them to offer the survey in a relevant language and to geolocate (whenever possible) the respondent’s position. 

RIWI’s method is useful in part because it circumvents government restrictions, as no other agency has a list of RIWI-controlled domains. The RIWI domain inventory, moreover, is constantly changing. As a result, RIWI’s survey work is particularly useful in countries such as China, where the government keeps tight control over its population’s access to Internet platforms, domains, and services. 

RIWI’s methods also help researchers gain access to populations in regions of the world where law and order has broken down, or where the commercial or public landline infrastructure is spotty. Mobile phone penetration is sometimes high even in remote or conflicted area, and any user with Internet access has an equal chance of stumbling across one of RIWI’s domains. 

Gathering data safely and securely is key. RIWI’s surveys are anonymous, and it neither gathers nor stores information that might identify respondents. Its geolocation feature does not identify specific respondents, and in any case is indicative of a general area, rather than a specific address. 

There are drawbacks to RIWI’s methodology, of course, as there are for every survey method. The most important weakness, perhaps, is that RIWI can only reach people who already have Internet access. Although Internet penetration is high in many areas and growing in others, there are still populations with no or intermittent access. Web use, moreover, tends to be skewed towards younger males. RIWI compensates for the first by giving more statistical weight to the responses of older people and women, and address the second by further adjusting the weights to mimic what is already known about general population demographics. 

Thus, for example, if the proportion of highly educated RIWI respondents from a specific country is greater than the real proportion as per census data, RIWI will “down-weight” the responses of educated respondents and “up-weight” those of the less educated. This procedure can be replicated for gender, ethnic makeup, geographic location, socioeconomic status, and more. 

No survey weighting procedure is foolproof, and the gold standard remains careful, face-to-face surveys in which researchers knock on doors and religiously follow survey guidelines. In this method, researchers return again and again to houses where no one answers, and carefully select respondents by making exhaustive lists of each household’s residents and then using predetermined criteria to choose one to survey. The gold standard is expensive and sometimes impossible for reasons of safety or physical access, however. 

Another problem with RIWI’s method is the response rate, which ranges from 13-17%. That means that over 80% of people who are presented with a RIWI survey don’t click on it. Response rates, moreover, tend to taper off after the first few questions; more people will answer questions 1, 2, and 3 than questions 9, 10, or 11. Low response rates are not abnormal, however. In telephone surveys in the US, for example, the typical survey response rate is below 10%. 

Response rates are higher when you knock on doors or sample from a list of pre-booked respondents who have already agreed to answer your questions, typically on the Internet. The door-knock method is too expensive for most researchers in developed countries, so many researchers have turned to using the prearranged panels operated by survey companies such as YouGov, which fielded my own nationally representative poll of US residents in fall of 2018. (For articles based on this poll, see this in the New York Times, another in Foreign Policy, and a third with the Washington Post. More findings are forthcoming in a report co-authored with Human Rights Watch, the Center for Victims of Torture, and the University of Minnesota). 

Telephone, door knock or pre-arranged Internet panels are not always possible, and in these cases, RIWI offers a clear advantage. 

You can read a detailed summary of RIWI’s methodology, strengths and weaknesses on pp. 88-91 of this recent World Bank report. In future blog posts, I’ll write more about individual RIWI studies, as I find them intriguing.