If there’s one thing market researchers can all agree on it is that quality data is the primary goal. We must be able to trust the data in order to make decisions. But how can buyers ensure that their sample suppliers are on the same page?
Here are a few “buyer’s tips” to help guide the process and ensure that the latest technology is applied, toward the goal of uncovering quality data. These originally appeared in P2Sample’s eBook: “The Ultimate Buyer’s Guide: How to Select A Sample Supplier.”
Representation
In a world in which the industry has willingly departed from the principle of probability-based sampling, we must at the very least be sourcing respondents from a large and diverse number of sources. From online forums and blogs to search engines, shopping sites and social networks, and from gaming sites to app stores, automation makes it possible to recruit a wide variety of people from hundreds of sources while managing quality and costs.
Conversely, sample buyers should avoid companies that use a limited number of recruitment partners, or source from only one channel – such as only drawing respondents from mobile gaming networks.
BUYER TIP: Buyers should recognise that some traditional parameters that signalled quality in the past don’t necessarily still apply. For instance, the fact that a panel is double opt-in doesn’t mean it will yield high-quality data. Double opt-in has been used as a way of ensuring the person’s commitment, as well as an additional layer transparency surrounding exactly what’s on the table in terms of privacy. In this day and age, it is a stronger (but not mandated) sign of GDPR compliance. Nevertheless, if a panel supplier is doing little on the engagement and fraud detection fronts, double opt-in matters little. After all, a fraudster could automate his response. The bottom line is that each person needs to be real, profiled, engaged and treated respectfully, in accordance with the law. This can all be accomplished with single opt-in.
Profiling
Proper management of one’s panel makes it possible to develop deep and rich profiles with literally hundreds of demographics and behaviours. This provides immense benefits to both suppliers and respondents. For suppliers, it means effectively targeting participants, which boosts conversion rates and lowers waste. For the respondent, it yields better experiences and maximises the likelihood of completing a study and earning an incentive. But these benefits can only arise if the profile data points are put to use in field through APIs so that respondents aren’t asked questions which aren’t relevant for a study or for which suppliers already have data. This is commonly called “dynamic profiling” and it is less time-consuming and intrusive for the user.
BUYER TIP: Buyers should ask about profile points and how they are used in execution. A good supplier will have a strong commitment to automation and APIs, and will put these practices to use for the benefit of not only the bottom line, but also respondents.
Identity validation
Automation and artificial intelligence (AI), while poised to create overall efficiencies in our industry, are also making it easier for fraudsters. This reality means that suppliers must employ ever more vigorous practices to defend against fraud.
Traditionally, we’ve employed several techniques to mitigate fraud. These have included:
- Email and address verification;
- Captcha, which provides an artificial, puzzle-like block that requires human intervention;
- Honeypots, which employ computing codes to act as a trap that machines find irresistible but humans never notice; and
- Open-end questions, which make it difficult for most fraudsters to create realistic, genuine, open-ended answers at scale.
Employed on their own, these approaches are helpful, but are no longer sufficient. This new enemy is not a “bot”, and engages in fraud through a complex mix of machine and human power. We must take a similar approach.
The answer is to combine traditional methods, such as those mentioned above, with advanced technology like artificial intelligence (AI). AI goes beyond algorithmic decisions: It is self-learning and will continue to find new patterns. Algorithms are typically more static in nature and require constant human intervention. Using AI to fight fraud requires:
- A machine-learning method that compares pattern behaviour by analysing billions of data points for anomalies
- A large amount of historical data to provide accurate classification and help the machine learn
- A vendor with experience in identifying, understanding and managing fraud, providing a solid foundation
BUYER TIP: This is an issue that may require further probing. Often, suppliers will maintain that they have various methods of verification or digital fingerprinting solutions to fight fraud. Ask for concrete information and be wary of techniques that are not constantly evolving.
Engagement
Suppliers need to demonstrate a real commitment to respondent engagement by ensuring accuracy and attention to detail. With problems like router bouncing that create negative user experiences, suppliers must manage the process in field to avoid bad experiences. Of course, buyers still have a large role to play here, given that the most significant element within the experience is the survey itself. Yet, suppliers need to be vigilant in their own efforts to minimise disengagement, too.
Some suppliers make minimal effort on this front by kicking out mobile studies for non-mobile respondents, and similarly superficial attempts, but there is more to be done. To truly accomplish better engagement, we must start using technology to proactively manage experiences in field. This can include everything from employing automation to spot trouble in field — trouble which can be algorithmically identified by using things like low complete rates.
Engagement can also improve when suppliers take advantage of the millions of data points that are available to them. Using this data, backed by automated algorithms, surveys can be targeted appropriately so users spend less time trying to qualify for a study and find a faster path to a good experience. This kind of forward-thinking approach can help boost satisfaction, whilst also eliminating avoidable delays.
BUYER TIP: If a supplier isn’t actively managing engagement in field then buyers should be wary. Buyers should ask suppliers how their routers work; how they deal with qualification; how they measure bad experiences; and what they do to prevent them.
Transparency
This is an extremely simple concept: suppliers need to unveil the mystery surrounding recruitment, process and technology. Where are respondents coming from? How are they being obtained? What technologies are being employed to ensure quality? These are just a few of the questions that suppliers should answer as a matter of course.
BUYER TIP: Require transparency from start to finish. Ask the questions mentioned above and learn more about the sources.
