By Patrick Comer
Ladies and gentlemen, our industry just won BIG last night. I’m not talking about the democratic victory or the political impact of the exit polls. I’m talking about how, at the end of the day, data science won out over punditry. Whether or not you like Nate Silver, with 538, or his obvious lean towards Obama, he was right… and not just a little right, exactly right:
(via Michael Cosentino @Cosentino)
But it’s not just that he was right that matters. What’s most important is the methodology he employed: Modeling, historical polling data, weighting the data based on results, and a huge dose of objectivity.
Compare this to the predictions of Karl Rove or unskewedpolls.com, who didn’t like the projected outcomes of the polls and re-weighted them to suit their political agendas.
The crucible moment of the evening, in the battle between data and punditry, came from Karl Rove himself. After ~70% of the Ohio vote was in, with a differential of about 5,000 votes, all the networks called Ohio in favor of Obama, and thus, re-election. However, Rove still tried to lay out a thought process on how Ohio could swing back Republican. I guess the Fox producers decided this was news and sent Megyn Kelly down to the Decision Desk in the basement to talk to the data analysts.
In an effort to investigate how certain they were of the Ohio call, she asked these data analysts: “Percent certainty?”
Data Wonk: [pause] … “99.95%”
Yes, he knew exactly the confidence interval and their level of significance associated with the analysis.
Why is this important to market research? Because it reinforces the value of the work that we do. It’s not good enough just to toss a survey up, get 600 interviews, and expect the answer to be actionable or meaningful. The consistency and replication of the sample frame by various polling organizations ended up being far more important than a single poll was in a specific moment in time. Isn’t this what we all preach about regarding the value of tracking data?
The headlines reflect the same mood:
So regardless of your mood about today’s results, know that what you do is significant, from sampling to data collection to data processing, coding and analysis. Tell your clients this lesson: consistency and replication of the sample frame really really matters.
And now, we finally have a market research super-hero:
(via Jay Smooth @jsmooth995)