By Chuck Miller (DM2), with Courtney Williams (Lucid) contributing
Inevitably, business conditions change or evolve. As a result, researchers are required to rethink their sample providers used in the execution of a survey. Sometimes this is prompted by marketplace forces (respondent recruitment challenges), and other times, a provider’s business may be undergoing change (acquisition, merger, etc.). Regardless of the reason, any time we switch sample sourcing on a project, uncertainty abounds – particularly with tracking research.
Impact on Data Consistency
Researchers know that changes in sample sourcing have the potential to undermine data consistency as much as changing a question. Even when demographics are matched precisely, latent or unseen characteristics can influence data outcomes – with different degrees of influence across different survey subject matter.
Knowing this, there are important factors to consider when examining sample sourcing and assessing the accuracy of your tracker:
- What types of sample characteristics could unwittingly skew data given the respondent composition and survey subject matter? (e.g. too many early adopters in a tech brand tracker)
- In the face of business change, to what degree can a provider maintain sample consistency via stable sourcing and manage the latent, underlying characteristics? (And do they measure and manage them in the first place?)
Why Lucid Excels at Accuracy
Through ongoing measurement in our Supplier Quality Program, Lucid is able to classify suppliers on different dimensions (latent characteristics) that can influence research outcomes – items such as Brand Affinity, Value Seeking Behavior, Technology Adoption, Media Usage, and Gaming Proclivity. The levels of these characteristics vary by provider, which is driven by differences in recruitment strategies and sourcing.
Having collected 500,000+ interviews over 15 quarters, in five countries, Lucid is uniquely positioned to establish baselines for these characteristics and assess if a project’s sample deviates from typical standards. This analysis is extremely beneficial in managing change and transition.
The Art of Transitioning a Tracker
Often, researchers ask if a new provider can ‘mirror’ the sample characteristics of present supplier(s) if they transition tracking research to a new supplier or blend of suppliers. While it is possible and tempting to consider, the moment of change of supply on a tracker is the ideal time to assess whether the blend of sample coming into the tracker is optimal in the first place – instead of trying to mirror what has been done in the past.
Consider for a moment when Nielsen transitioned television audience measurement from paper diaries to passive ‘people meter’ data collection in the late 80’s. The quality of the data collected was more reliable than in the past, but critics contended that the new technique must be challenged because the data changed. The change in the data was not a sample or methodology issue, it was an insight resulting from more accurate and technologically advanced methods.
So, when you think about change in sample provision to your tracker, ask yourself if this is also an opportunity to strive for further reduction in bias within your sample, so that your insights are more reliable and factual – not simply the same as the past.
Lucid’s Tools to Manage Change
Having successfully migrated hundreds of trackers over the years, Lucid has both knowledge and tools to ensure smooth transitions. Leveraging both analyses and experience, Lucid typically employs one of two methodologies for transitions depending upon on the situation. The typical timeframe for a transition ranges from a few weeks to a few months. A common component across both methods is the use of sample blending following baseline measurement.
Sample composition information for the incumbent supplier(s) is used to construct targets for the compositional items. From there, a sample blend using multiple Lucid providers is designed – balancing on the five compositional characteristics (noted above) with allowances for demographic composition. All of this is done using the Lucid Sample Blending Calculator that leverages the most recent Quality Program data.
Alternatively, the calculator can be used to slowly shift the sample design to one that is “compositionally neutral” on the latent characteristics, which then minimizes potential biases that would lead to a faulty research outcome. Decisions can be made on whether to exactly mirror the incumbent, or move over time to a more neutral/representative sample to eliminate compositional biases. Whatever the goal, the tools exist to achieve it.
Additionally, ongoing management of a sample blend for a project ensures consistency can be delivered over the long-term – creating sample replicability and sustainability from the vast resources of the Lucid Marketplace. The depth and breadth of the marketplace can be leveraged to deliver the same blend over time, providing capacity benefits unseen by any single provider (or small group of providers). Together, capacity and consistency deliver sample sustainability – ensuring the best possible research outcomes.
Given the unique nature of every tracker and business situation, there is no one “silver bullet” to apply to transitions. To determine if Lucid’s sample, tools, and methodologies are right for you, please contact us for a consultation.
“In the beginning of this year, our team was chosen to run brand tracking for one of our highest-profile clients, and we are extremely happy we trusted Lucid to take on the job! We made some changes to our client’s previous survey including some fairly complicated custom programming, and the team was able to deliver exactly the dynamic and engaging program we were looking for. Throughout fieldwork, their team has been responsive and flexible, and their attention to detail has yielded high quality data and allowed us to calibrate the data back to previous years seamlessly.”
-Hannah Crippen, Director of Research, Sparketing