Download the Esomar 28 Questions n PDF format (478kb)
INTRODUCTON TO THE ESOMAR 28 QUESTIONS
TO HELP BUYERS OF ONLINE SAMPLE
ESOMAR is the global organisation that strives to enable better quality research of markets, consumers and societies. As part of their guidelines for conducting online research on the Internet, ESOMAR has a series of questions designed to help market researchers when purchasing and conducting online research.
Cint has created a transparent online marketplace called OpinionHUB, which connects panel owners with sample buyers. Essential to the Cint platform are the quality standards and controls built into its processes for conducting research. With Cint’s marketplace platform, online sample buyers have full control over their fieldwork, whether they choose to do ‘self-service’ (DIY) or retain the use of Cint project managers who have been trained in research quality control. All systems and procedures adhere to the standards set forth in Cint’s accreditation to ISO 20252, which includes Cint’s focus on transparency to sample buyers.
This document demonstrates how Cint’s unique approach to undertaking online market research meets and often exceeds the requirements set by ESOMAR.
Cint’s business model offers a unique, transparent exchange model that brings together buyers and sellers of research samples. Within the Cint platform, panel owners can use our advanced panel management software, Cint Engage, to set-up, recruit, manage and maintain panels and subpanel groups, for their own insights needs, as well as using Cint’s platform to publish their panels for access by sample buyers. Panel owners in the Cint platform include market research agencies, media owners, publishers (digital and traditional), non-profits and companies with access to large web-traffic. All panels are double-opt-in (DOI), research-only, and ESOMAR and ISO compliant.
As a result of the unique way Cint builds supply within its insight marketplace, buyers (clients) can reach over 10 million panelists in more than 55 countries in 750 individual panels. Samples can be purchased via managed services where Cint responds to an RFQ and a PM runs the project or by using Cint’s self-service solution, Cint Access. Cint Access enables clients to check feasibility, calculate pricing, set-up target groups, assess quotas, order sample and manage their fieldwork, all from an easy-to-use dashboard-based interface. Cint also offers Cint Link, an integrated API (application programming interface) solution that permits survey software tool providers to integrate their software giving direct access to their users to millions of DOI panelists for online market research from Cint’s OpinionHUB marketplace.
Cint empowers access to opinions by innovating the way market insight is gathered. Founded in 1998, Cint launched the industry’s first and only online panel marketplace platform in 2005 initially called CPX and now named OpinionHUB.
Cint works with more than 650 clients in over 40 countries, including the most widely recognized market research agencies, media, PR and digital end clients.
On average we run over 6,000 online projects every year with our clients and deploy sample in more than 55 different markets around the world. In addition to the major markets, we also deliver services and projects to clients in markets that are still considered to be underdeveloped or emerging markets for online research, including South Africa, the Baltics and Latin America.
Through our innovative technology and solutions we have enabled clients to tap into niche audience groups not commonly found within online panels, like gamers panels, sports panels, mothers of babies, different B2B audiences, professional medical, and many, many others.
Our scope of work is broad, and we run research for leading brands, retailers, CPG/FCMG firms, and companies from many different sectors in partnership with our market research agency clients. This includes a significant proportion of tracking work that has been running continuously over five years.
Sample Sources and Recruitment
2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
Panel owners in OpinionHUB source panels through the following types of recruitment:
- Active recruitment through telephone
- Active recruitment through face-to-face (F2F) interactions
- Online recruitment using banners on different portals and websites
- Online recruitment through the owner’s portal using editorial space (media and publisher sources)
- Email recruitment through the panel owner’s newsletters
- Specific invitations sent to the panel owner’s database
- Email recruitment by sending invitations to a permission-based database
- Social networks
- Affiliate traffic
Recruitment into Cint OpinionHUB typically takes place through a link to a specific panel registration page either through an iframe or directly to a white-labeled, Cintbuilt landing page in owner’s brand template or “look & feel.” After a panelist completes the entry of their contact details, they are thanked and informed that they will receive an email with their account login details. Once this email is received, panel members ‘double opt- in’ by logging into their account to activate it. A new panelist is given the chance to verify their personal information, set the preferred frequency of surveys and view any other information provided by the panel owner. If a panel owner has an active panel on their own system and decides to move it to the OpinionHUB platform these existing panelists are imported into the Cint platform. If a potential panel owner has a list of contacts who have expressed interest in joining a panel, but haven’t been recruited as such, they are re-validated via email. The e-mail message contains their new panel account and login details as well as the link to their panel account page. To become or to continue being panel members they are requested to login and activate their account. Panelists are required to complete a double opt-in process by clicking on the panel link and then logging into their account.
3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
Cint OpinionHUB ensures an optimal blend if the needed target group exists in more than one of the available panels. The sample blend is controlled by a number of rules to ensure both source bias and panel usage risks are mitigated. The same algorithm is used for every sample and in addition, sources can be freely chosen by the user. Cint uses a suite of technologies and controls that ensure that duplicates are not present in any online sample. Cint supports both RelevantID and TrueSample.
4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
Panels within OpinionHUB are only used for market research. When members register, they are informed that their account is to be used exclusively for research. Every panel in the online exchange adheres to ESOMAR Esomar 28 Questions v.1 – page 3of8 – 20th Nov 2013 standards relating to the mandated ethics for the execution of research. Direct users of OpinionHUB (i.e. self-service or managed-service customers) are also required to sign an agreement that limits their use of sample and demographic information solely for market research purposes.
OpinionHUB has a large collection of panels in one place, recruited using different methods, from different sources, both online and offline. All panels (supply) are built by partnerships with a third party, whether they are an MR agency, media owner, publisher or digital company with a website looking to monetize their traffic and data. This allows Cint unique reach into niche audience groups such as specialist patient or professional medical databases, mothers, as well as consumers. Panelists are profiled when they join a panel, and continuously over time, so specific target groups can be identified. As Cint Access permits sourcing from multiple panels, it makes it possible to find harder-toreach groups in suitable sizes to run surveys as you are reaching into a larger pool of respondents from many different sources, rather than a traditional online (singlesource) panel. We also encourage panel owners to meet the demand generated in Cint OpinionHUB, by recruiting lessrepresented groups. Cint collaborates closely with panel partners to continually build the total sourcing pool, building the leading marketplace available for research sample.
6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider?
Cint’s transparent model allows clients to see exactly where respondents originate from. All panel sources are presented to the client should they wish to see them, and when clients self-serve (DIY) they see every panel down to the panel brand, survey invite, incentive model and ‘blend’ of the sample. Cint lets clients have the choice of how they want to sample, and from which unique panels, allowing for single sourcing or even panel blending at a project level. This can include brokering third-party sample from outside the panel marketplace. In cases where Cint sources panelists from third-party sources (outside vendors), this process is performed in compliance with our ISO processes as well as our policy on transparency. In a case where a partner is used, clients are always notified at the bidding stage and again when a job is commissioned. Cint discloses which subcontractors it uses, and why.
Cint retains an accredited list of subcontractor partners that have been vetted and form an approved network. Subcontractors are chosen based on their capabilities and quality control measures, and their adherence to Cint’s ISO-based processes.
Sampling and Project Management
Cint gives the opportunity to select ‘census representativity’ on both survey send-outs and incoming completes. The platform provides the correct allocation to achieve the proper distribution based on census data. A client can also choose to create their own spread by adding quotas or send-outs to specific target groups.
Deployed within OpinionHUB, Cint has an integrated survey router and dynamic profiler called Cint Optimizer. Cint Optimizer increases panel reach and sample efficiency by addressing deficiencies in first generation stand-alone router systems, which can often result in poor panelist experience and higher panelist churn. Benefits include:
- Increased permission-based profiling depth as Cint can easily store routing qualification data in a panel member’s profile. This results in the ability to better target surveys to panel members;
- A superior optimization and routing experience for panel members as Cint Optimizer leverages existing panel member profile information when matching a panelist to a survey, minimizing the number of qualifying questions and the time in the optimization and routing process;
- Keeping panel members incentivized, happy and continuing to take surveys.
The router can be controlled on both a source and project basis. Cint Optimizer use by panel partners and clients is voluntary.
9. If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys?
Every respondent experience with Cint begins with a direct email invitation. If the respondent does not qualify for the intended survey (screened-out, quota full, etc.), Cint Optimizer will attempt to match the respondent to Esomar 28 Questions v.1 – page 4of8 – 20th Nov 2013 another open study by matching the stored profile of the respondent to open studies. If there is a match, there are no qualifying questions asked. If there is a partial match, the panellist will be asked the remaining qualifying questions. The respondent can opt-out at any time during the Cint Optimizer experience. Survey allocation is randomized although the match rate between respondent and open studies is considered.
10. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias?
We only use Cint Optimizer on studies that are not likely to be affected by or likely will not negatively affect the available routed traffic. . With high volume and a wide mix of target populations, we mitigate the risk of router bias. Cint does not utilize routing methods that can create systematic prioritization of certain categories or studies which would create bias. The randomization element used in the routing algorithm is key. In the event a client wants measurement and reporting, we make available a full dataset of the respondent’s entire pre-profiling process in the router.
11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
Project managers decide whether a project will utilize routing. All routing parameters are set globally by a centralized operations team. A carefully selected and limited number of administrators have access to view or change these parameters. Project managers can neither view nor change the global routing preferences. Cint does not offer the router on self-service (DIY) work.
12. What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?
Standard profile data stored includes: email address, name, address, phone numbers, gender, postal code, year of birth, education level, occupation and all panelspecific profiling. Cint also has a number of standard ‘global questions’ (90 questions with sub questions) which are used across all panels and countries in OpinionHUB to obtain detailed standardized data on panelists. These ‘global questions” include questions pertaining to: automotive preferences, healthcare, employment, travel, etc. This data is updated every time panelists make changes to the profile information in their account, either proactively or through panel communications (profile surveys, newsletters, etc). Also, after responding to a survey (as a complete, quota-full or screened-out participant), panelists are presented with the opportunity to complete additional questions from the global questions list, to ensure their profile is kept up to date.
13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of emails.
Panel owners set up their own invitations in the Cint platform using the panel management dashboard controls. Content varies by panel owner, however, with all items required by ISO 20252 are included in email invitations:
- a general description of the purpose of the project
- the estimated length of interview
- a statement of the confidentiality and anonymity of each respondent’s responses
- the closing date for completed responses (if applicable)
- access to full disclosure of incentive terms and conditions applying to the project
- an explanation if the invitation is sent out on behalf of another research service provider
- for panel members, the opportunity to unsubscribe or opt out of future research
There are also clear instructions within the invite if the respondent is required to undertake a particular task, or is required to have a specific software or capability on their PC, laptop, mobile or other device. For those who do not wish to take part in the survey, there is a ‘decline’ option that is embedded in the email. The invite also includes a support email address for any queries relating to the survey.
14. Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
The Cint OpinionHUB platform utilizes a reward system based on marketplace points. The number of points awarded is driven by the length of interview (LOI). On reaching a redemption level set by the panel owner, panelists can redeem their rewards through different online payment partners linked to OpinionHUB. The size of the redemption is based on the number of points earned. Panelists can choose to receive their rewards in cash sent to their bank accounts (e.g. via PayPal) or they can shop online with online merchants. Some Esomar 28 Questions v.1 – page 5of8 – 20th Nov 2013 panels include an option to make payments to a charity. Alternatively, the panel owner can opt to provide the panelist rewards directly. Incentive levels have been set to encourage long-term participation and to discourage professional respondents who seek to take surveys only to obtain payment. A key feature that drives long-term participation is that panel owners choose an incentive model that works best for their members.
15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
- The demographics being sought (age, gender, regions, etc.)
- Any non-demographic targeting or behavioral criteria necessary to qualify
- Any quotas or sub-quotas which need to be achieved
- Additional sampling (deployment) criteria if applicable (i.e. census representative deployment or balanced send-outs to initiate the survey)
Cint has deployed a panelist/respondent satisfaction program. Every respondent that completes a questionnaire has the option to voice their opinion. They are asked set questions on survey length, logic, language and are also able to give feedback in an open text box. This information is available to clients in the insight exchange.
Cint also undertakes its own internal ‘research-onresearch’ which covers respondent engagement, satisfaction, fatigue drivers and the associated solutions. This research is shared with panel owners and utilized in Cint’s internal quality programme.
For each project, we provide standard information on number of responses, status of responses (i.e. completes, screen outs, quota full and drop outs) as well as average length of interview and incidence rate.
Data Quality and Validation
18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviors, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
Cint has a dedicated quality assurance team, which isresponsible for maintenance and advancement of the quality of Cint’s products and services. Through its ISO 20252-based quality program, Cint deploys project and company-wide procedures that result in products and services of the highest quality.
In general, Cint does not host the questionnaires for data collection. Accordingly, Cint mostly works with clients to have them deploy appropriate validation checks, including but not limited to: analysis of questionnaire completion time, data outliers, unanswered questions and patterned responses. We further encourage our clients to add straight-lining, red herring questions and other data quality checks to their surveys. Respondents who do not pass these checks do not qualify as completes and do not qualify for an incentive. If a client reports cheaters in a survey, Cint project managers remove these from the survey through OpinionHUB and alert the Panel Operations department. They will then alert the panel owner (with the list of impacted panelist IDs). Cint also maintains a record of these IDs internally. After a panelist receives “three strikes,” they are taken out of the panel marketplace platform (Cint OpinionHUB).
Every respondent invited to a study is put into a quarantine system. This ensures a panelist cannot be invited to a study beyond the frequency level set forth in the agreement between Cint and the panel owner.
19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
Cint monitors the frequency of participation of all panel members in OpinionHUB. Every panelist is also given a score based on their responsiveness.
OpinionHUB holds detailed data for each panelist on which survey they have taken part in; therefore, a panelist can easily be included or excluded in another survey. Each panel owner can choose their desired ‘quarantine period’ (i.e. the time between survey invitation mailings) to ensure panelists do not receive too many invitations.
Regardless of sample source, Cint policy allows for one invite and one reminder to a single survey opportunity. Typical lock-out periods for inviting to a new survey opportunity is seven days, but this can be vary depending Esomar 28 Questions v.1 – page 6of8 – 20th Nov 2013 on the country and panel.
20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
Depending on the quarantine settings on both the panel and the panelist, this ranges from once a month to three times per week.
Panelist may only complete the same survey once. Deduplication, re-use and the lock-out period for quotafulls or terminates is set on a per-project basis based on client-preference. The default allows no re-entry once an official status (complete, terminate, etc.) is achieved. For tracking surveys, de-duplication, re-use or lock-out period for subsequent waves is set according to client preference.
21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data?
Cint OpinionHUB saves detailed panelist participation data which includes: panelist join date, last participation date, transaction history on all surveys, redemptions, reward points transactions, etc. As Cint strictly adheres to all applicable privacy law and regulation, personally identifiable information is never made available.
Cint OpinionHUB includes an automated system that ensures all panelists are ‘quarantined’ after participating in a survey, meaning they cannot be sent another survey for a certain amount of time.. The standard quarantine period in OpinionHUB is seven days; however, this can vary by panelist and panel owner.
Cint utilizes an indexing system which gives each panelist a score depending on how active they are with respect to survey participation. A participation history is recorded and additional exclusion can be applied on a subject category.
Cint OpinionHUB has an advanced subject-category quarantine capabilities which is available to those clients who need it. Respondents can also be excluded from future surveys by excluding previous projects when selecting sample in OpinionHUB.
Upon request Cint can provide participation history reports to clients on request.
22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any?
Cint has a range of features to deal with professional/ duplicate respondents. The system includes stratified sampling to get various types of respondents, including active and less-active panelists. The length of the survey is evaluated and compared against individual answers. OpinionHUB identifies and excludes multiple panel respondents both through email address and by name. Postal address and bank data is also investigated if the payment method supports this.
Cint has a panel quality team that continuously analyzes panel data to identify ‘fraudulent’ or ‘inattentive’ panelists. Panelist status is updated in cooperation with the panel owner.
As all research stakeholders would agree, quality of response is also affected by the quality of the questionnaire. Cint project managers provide feedback to clients of all potential issues which could affect fieldwork, both before project launch as well as on project completion.
Policies and Compliance
At the time of panel recruitment, panelists are made aware of the purpose of the panel (i.e. to take part in market research). Each panelist is double-opted in. When invites are sent out to panelists, they are notified of the confidential nature of their responses and given the opportunity to ‘opt out’ of the panel. As part of the registration process, panelists are asked to fill out several key profiling questions, including a confirmation of their consent to join the market research panel. Upon confirmation, the panelist will receive an automatic email sent directly to the email address they provided in the registration process, allowing them to double opt-in to the panel. The email contains a link within it, as well as a user name and password. The link will direct them back to the panelist site, hosted on Cint’s servers where the panelist signs in with their username and password.
Cint and its clients are all required to adhere to local data Esomar 28 Questions v.1 – page 7of8 – 20th Nov 2013 protection laws and legislation. This is reflected in Cint’s panel owner agreements. All information supplied to Cint by its clients is treated in the strictest of confidence and follows all applicable national law and regulation, ISO documentation requirements.
If any research results or confidential information is provided by a client, it is not used in any other research project unless the client authorizes it.
Other security measures in place include:
- Access to project information is only granted to the user who creates each project.
- Users are automatically logged off after a given period of non-activity.
- Users must sign in using a username and password.
- Respondents reach their surveys by GUIDS (globally unique identifiers).
Panel management security:
- Access panel and panelist information is only granted to the company who owns the panel. This does not include any personally identifiable data.
- Users are automatically logged off after a given period of non-activity.
Security Assessments, the Cint SaaS Environment:
- The Cint SaaS environment has been designed with security, high-availability and performance in mind.
- All servers, services and networks are monitored 24/7 by both Cint and our hosting partner with operation teams on continuous availability.
- Cint OpinionHUB uses Secure Sockets Layers (SSL) for all sampling, panel management and critical panelist information.
26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
Cint performs a quality check on every survey before it is made available to panelists.
Project managers thoroughly test survey links for:
- Correct implementation of re-directs
- Appropriate survey logic
- Appropriate language
- Accurate quota set up
- Appropriate interview length
- Whether or not there is any personal, sensitive or prohibited questions being asked
- Whether or not panelists can choose a ‘prefer not to answer’ option with any income or ethnicity questions
- Overall quality.
If the survey contains any sensitive material or content, that is clearly cited in the survey invite and, where appropriate within the survey, giving panelists the opportunity to opt-out. As required, Cint exercises NonDisclosure Agreements (NDAs) with clients and panelists, highlighting the sensitivity of the material. These NDAs mandate that all information within the survey must be kept confidential and any breach of proprietary information may be cause for legal action.
Cint maintains a detailed inventory of all relevant national privacy law and regulations to ensure up-to-date compliance.
If any item in the questionnaire or its fielding are not to compliance with Cint quality standards, the project manager will promptly communicate with the client making recommendations for any necessary changes. All such interactions are logged.
As stated in Question 18, Cint is certified to ISO 20252 and complies with ISO 26362 where possible. Cint has a quality team that works with all departments to ensure that requirements are met. Accordingly, Cint can ensure that all controls and procedures for project execution meet the requirements of ISO 20252 and where possible, ISO 26326.
- Organisational structure and responsibilities
- Confidentiality of research
- Research documents and records management
- Outsourcing of services
- Project management
- Recruitment measures
- Access panel structure and size
- Access panel management
- Access panel usage
- Client reporting
- Data collection, validation, management and processing
- Professional rules of conduct
All panel owners in Cint OpinionHUB are required to sign an agreement that states they will comply with relevant Swedish legislation (where Cint AB is headquartered) with respect to the protection of personal information. In addition, Cint complies with European Data Protection legislation. In addition, panel owners are asked to Esomar 28 Questions v.1 – page 8of8 – 20th Nov 2013 Cint Sveavägen 20 11157 Stockholm Sweden Tel: +46 854 638 300 www.cint.com work comply with ESOMAR and Cint’s ISO quality program. Panel owners are required to inform panelists that any personal information gathered is kept strictly confidential, is never passed onto a third party without the panelist’s permission, all information is used for market research only and panelists can opt-out at any point.
Cint also recognizes and adheres to CASRO, MRA and applicable national MR association guidelines and standards. Further, Cint has put in place procedures to ensure high quality services from our partners or subcontractors.
28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?
Cint follows all applicable national law and regulation for conducting surveys with children, only performing online surveys with children and young people after receiving parental consent.
All surveys with children are addressed to parents who are asked to grant permission and make the survey available to their child. Survey invites inform the panelist (parent) of the length of interview, survey topic and encourages parents to sit with their child as they fill-out the survey.