Exactly six years ago, JD Deitch – now our COO at Cint – wrote an article for Greenbook called The Future of Panels: Five Investments Needed to Improve Participation and Reduce Bias. He maintained that “among the many challenges market research firms face, none is more severe or important to solve than the mounting difficulty in securing a representative sample of survey research respondents.”
While this remains a challenge to this day, there is a light at the end of the tunnel when it comes to the “trouble with panels.” Back in 2015, JD wrote that there needed to be real change in “how sample is sourced, how much it costs to access real respondents, and on the future of panels themselves.” He recently noted that Cint is addressing each one of his five original points. We outline just how we’re doing it below:
Reach and diversity are essential:
In short, JD originally put forth that “firms will need to invest heavily in making programmatic acquisition work…The ability to recruit richly at scale and evolve as the market evolves will be an asset that provides competitive advantage. Implied in this greater attention to reach and diversity is the need for new measures of quality. Panel size and response rate are becoming meaningless. Both can be artificially manipulated in ways that do little to really help us understand bias. Response rate simply isn’t applicable in a non-panel environment. What matters now is feasibility across an ever-evolving panoply of sources. Reach and diversity are the only ways to increase feasibility and reduce coverage bias.”
Cint’s approach: We operate the world’s largest consumer network for digital survey-based research, with more than 144 million respondents from over 130 countries around the globe. The very number of people on which we can draw gives a higher probability of diversity in any given sample. In addition, our digital insights gathering platform has the ability to identify consumers within our supply network based on deep profiling. These profile points allow the further identification of people who fit the diverse, nuanced characteristics sought for a specific study.
Incentives and engagement must be meaningful:
JD wrote that “panels are in desperate need of a new value proposition.” From proprietary incentive systems to the impact of incentives on engagement, he covers the broken rewards ecosystem, pointing to the need for transferable currencies and other solutions to boost participation in the long run. He writes: “Automation and access to data on costs by source could easily lead to highly personalized rewards by profile that improve participation, with the only caveat that companies will need to pay greater attention to fraud…Panel companies need to acquire and reacquire/retain participants in ways that will force them to pay attention to—or to simply pay more for/to—the people they need.”
Cint’s approach: Fair consumer rewards are built into Cint’s model from the ground up. Cint’s Partner Success Teams work directly with our suppliers to educate them about acceptable incentives strategies. We are committed to ensuring that respondents are adequately rewarded, following metrics such as matching the value of the reward with the length of interview. Our suppliers know that rewarding respondents properly is good for engagement, data quality and the success of their businesses.
Suppliers need to stop putting up with abusive buyers: At the time the original article was written, calls for better device agnostic survey design were part of the mainstream conversation. “Panel suppliers are shouting the loudest, exhorting researchers to…overhaul their designs. Panel providers will, of necessity, start protecting their respondents.” He predicted that suppliers would begin to measure respondent satisfaction (or lack thereof) using automation to track a number of key metrics. In addition, JD wrote that routing experiences needed to be improved by using technology to track basic demographic information, better matching respondents with the right surveys.
Cint’s approach: Cint’s Customer Success Team works closely with buyers to ensure that survey designs will not only provide a good experience for respondents, but will also result in quality outcomes. We track key indicators that allow us to observe which studies are performing poorly without falling back on arbitrary standards. Our systems evaluate these indicators in field and will shut down studies that create a bad user experience.
From a routing standpoint, the Cint platform uses AI-based algorithms to analyse panellist profile data and past survey interaction behaviour against the survey attributes. This allows us to present the right survey to the right user at the right time. This means when panel suppliers send their panellists to a survey, which is optimized for successful data collection, they see positive performance metrics.
Respondents will be deeply and respectfully profiled. The above point leads right into the fourth concept surrounding better profiling, while keeping rising data privacy trends in mind. He was right – privacy regulations and consumer concerns surrounding their personal information has risen exponentially over the past six years – and promises to continue to do so. “Researchers will need to carefully watch the trends in personal data security. Mounting concerns over privacy suggest a future in which, at minimum, there’s little trust for unknown data collectors.” That said, better profiling is a key way to improving overall respondent experience, so solutions will need to be forward-thinking on this front.
Cint’s approach: Our platform gathers first-party profiling data from respondents in a fully GDPR compliant, consent-driven environment. We work with closely with our suppliers and panel owners to ensure panellists are deeply profiled within these legal rules and ethical principles. We offer 300+ profiling options, plus we have partnerships with multiple third-party solutions for further identity graphing and verification. This means that respondents have full control over what information they share, and there are enough deep profiling options available to appropriately target and tailor surveys for better respondent experiences and study qualification.
Quality practices will need to evolve:
While some quality issues can be minimized by addressing design and engagement (as discussed above), JD said that in 2015 “extant practices for ensuring representativity and response quality are insufficient.” He wrote that technology needed to be widely implemented that would automatically detect measurably questionable responses and behaviors, “at which point the respondent and his data would be eliminated or quarantined.”
Cint’s approach: Cint’s has implemented a formal Quality Program that encompasses the breadth of our operations and products coupled with formal performance measurements. The components of this program spans many of the points outlined above, as well as specifically addressing fraud through a multi-pronged approach. The Cint platform taps into the power of artificial intelligence and machine learning to recognize suspicious patterns and stay a step ahead of the organized, agile fraud of today. This dynamic approach is combined with more other techniques such as deduplication monitoring, IP checks, Captcha techniques and open-end validation for another layer of protection.
JD concluded his 2015 article with this statement: “It’s impossible to predict how the factors that are driving sample challenges will resolve themselves. What’s sure is that the forces are inexorable. The worst assumption the industry can make is assuming the future looks like the past. Research companies of all stripes, private and public, need to understand that they are not facing marginal cost decisions. Their actions require new thinking upon which sustainable futures can be built.”
At Cint, our commitment to quality and innovation helps us to address the “future of panel” – giving our clients representative, diverse sample; treating respondents with respect; and thoughtfully implementing technology to ensure efficiency, accuracy and quality at every step of the data collection process.
