The primary objective in executing any survey research project is to most accurately measure the population under study. A key to that success is minimizing bias. Researchers take great care to design samples to obtain data from their target population – using quotas to prevent over/under-representation of certain groups. Delivering quotas manages observable characteristics (e.g., demographics), but managing unseen, latent characteristics has been problematic – if not impossible in many cases. Thanks to investments in quality, Lucid is uniquely positioned for success here.
Even when demographics are matched precisely using quotas, latent attitudinal characteristics can influence data outcomes, as two demographically similar people might think and act totally differently. The degree to which these items influence research outcomes can be intensified by survey subject matter. For example, a study for a technology product that is overly populated with tech early adopters (relative to the population) could lead a brand to believe they are in a better position than they actually are – leading to bad business decisions.
Using the Quality Program to Classify Suppliers
Through ongoing measurement in our Quality Program, Lucid is able to classify suppliers based on different dimensions (latent characteristics) that can influence research outcomes. These characteristics include Brand Affinity, Value Seeking Behavior, Technology Adoption, Media Usage, and Gaming Proclivity. The levels of these characteristics vary by provider, which are driven by differences in recruitment strategies, sourcing, tenure, and incentivization. And for any given provider, the levels of these characteristics also change over time.
Having collected 900,000+ interviews over 23 quarters across 15 countries so far, Lucid has uncommon context and an ongoing ability to measure providers and recalibrate benchmarks for these characteristics. This leads to assessments of how our world is changing, and how different providers’ samples change over time – quantifying the degrees to which they deviate from typical, population standards. These analyses are extremely beneficial to client and project management over time – in both anticipating potential problems and managing sample stability.
The Sample Blending Calculator: How it Works
To achieve sample stability, Lucid uses this data in a proprietary Sample Blending Calculator to establish “compositionally neutral” sample designs that mitigate the risk of obtaining a faulty insight. By balancing the five latent characteristics while matching key demographics (such as HHI, Education, and Presence of Kids) to census targets we can deliver an optimal sample. The combination of quality program data and the intelligence of the blending tool uniquely positions Lucid to deliver the most representative sample – going way beyond simple demographic balancing.
Alternatively, the sample blending calculator can be used to match the sample characteristics of an incumbent provider. Clients may start from that point to first establish consistency, then slowly shift the sample design to one that is compositionally neutral and “future-proofed” should the characteristics of providers shift. Whatever the goals, the tools exist to achieve them.
Ongoing management of a sample blend for a project (or client holistically) ensures sample consistency long-term, delivering sample replicability and sustainability from the vast resources of the Lucid Marketplace. Armed with Quality Program data and the Sample Blending Calculator, the depth and breadth of the marketplace can be fully leveraged – providing capacity benefits unseen by any single provider (or small group of providers). Lucid’s investments in quality and blending tools produce sample consistency that benefits all types of research – creating a unique capability to ensure the best possible research outcomes.
The Methodology behind Substituting Blends
Providers are compared to characteristic benchmarks, derived from a 9-month rolling average within a country to establish the norm for a particular score. These measures look very similar quarter to quarter, but shift subtly over time (e.g., Technology Usage).
When a provider’s scores are compared to the benchmark, we calculate the difference in standard deviations. An ideal blend of providers has a target score of zero standard deviations across all measures – a blend in which all providers balance to neutrality.
Then, once the deviations are calculated, scoring can be used to find like-providers and create substitutes for a particular provider. For more information on our blending practices, feel free to contact Lucid or reach out to your company representative.