Asking People the Right Questions with Jonathan Knight of Qmee
In this episode of Through Your Looking Glass, Patrick Comer and Jonathan Knight talk about machine learning platforms’ benefits and challenges. Jonathan is the founder and CEO of Qmee — a survey platform that helps people make buying decisions through numerous questions.
Jonathan discusses the importance of proper data collection and ways of utilizing it. He notes that it is crucial to divide fraudsters and real people before using data. He explains how they deal with frauds and what it takes to gather valuable and relevant data.
He thinks that machine learning is the future, but it still needs to be driven by humans. People need to help the machine while learning by optimizing it regularly. In the end, Jonathan talks about the stack they use at Qmee and how they’ve created a vast database with high-quality surveys.
Name: Jonathan Knight
What he does: Jonathan has a master’s in Computer Science. He is a Founder and CEO of Qmee, an app that helps buyers with survey data. Jonathan has built and run high-performing teams in both the financial industry and in smaller high-growth companies. He is able to combine insightful business strategy with deep technology understanding to create significant business value.
Key Quote: “We don’t want ten times the volume with the same number of people doing surveys. We need to spread the population of people who are happy to come into the survey market and spend that time doing this. The only way to do that is to make the processes frictionless and fun.”
- The goal of Qmee is to find the right people and ask them the right questions. According to Jonathan, building technology allows them to recruit new members to Qmee and put the proper survey in front of the right person at the right time. ”We all strive to be incredibly efficient about matching users to survey opportunities, making sure they are real people, and doing all of that in a 100% automated fashion. We are doing millions of survey completes, and we have a staff of 22 people. It has been COVID, so we’ve not been in the office, but even when we’re in the office, it’s fairly quiet. The computers are doing all the old work. We don’t do anything directly, manually.”
- Frauds often happen when it comes to machine learning. As Jonathan says, the fraud problem evolves constantly, and they invest a lot of time building anti-fraud systems using machine learning. ”We spend a little bit of time every day checking the people that it’s labeled either as good or bad. Just confirming in our minds that we think that is true. And obviously, we take feedback back from the buyers and the platforms back in. The job becomes more about labeling the outputs.”
- Qmee achieved collecting a lot of data. Besides building a giant data source, Jonathan says, they are successful thanks to their data scientists. ”We have click data, hardware data over the different things; we have sign-up data. We have timestamps on everything and a huge amount of inventory data. We have a strong group of data scientists, and I guess that’s the other piece of this. You need the people to be able to do it.”
“The more automated you’re able to be, the more frictionless the market can be, volumes can increase.”
Types of learnings in automation
“I guess there are two main learnings. Number one was the automation that drives huge volume increases. If you look back at financial markets, particularly in the late 1990s, early 2000s, they went through a wave of automation, and we could copy that playbook.
The more automated you’re able to be, the more frictionless the market can be, volumes can increase. Not just linearly. They can increase exponentially as these things get better.
We’ve seen that play out in many ways. What’s probably more exciting is, we’re going to see that play out even more in the future. The automation is not complete yet through the stack. There’s plenty of opportunities. We’re planning on volumes being ten times as big as they are now. We always have that plan in our minds.
The second part is how we price opportunities. And for us, that’s a balance between us being able to reach the right person in the world, the surveys buyer wants to talk to, and engaging them enough in the process that they’re happy to do the survey and complete it and give good answers.”
Creating good surveys to find valuable answers
“We use machine learning to do an analysis of the questions and the answers, and we can, often the questions are very similar, crush those down into a much smaller subset of things that are being asked. We can use the natural language in the questions to map back to what our users have told us about themselves. When we understand a bit of a user, we can predict quite a lot about the rest of their profile.
We don’t have to ask them 200 questions. We might only have to ask them 25 questions, and that gives us a lot of information. We don’t want it to fail. We want them to come in straight away to the right survey, with a smile on their face, happy to answer the questions that are being presented to them. If we can make it as frictionless as possible for that event to happen, you have a better population of users coming into your questions.”
Frauds are an ongoing challenge
Having a lot of data can cause a lot of problems if not taken with precautions. “The fraud problem evolves almost on an hourly basis, or certainly on a daily basis. The people get blocked, and then they try new things. That’s changing all the time. It’s also something you have to react to very rapidly because for many of the people doing this, if they can take a couple of dollars out of the system, that’s a win, and they’ll come back tomorrow and do it again. Those are all problems that you can’t do manually. It just doesn’t work because you can’t apply the same resources to the thousands of thousands of people coming in fraudulently or badly into the platform.”
Data collection obstacles and solutions
“We want good users to stay on our platform. If we’re making life so hard for regular users by throwing a huge number of obstacles in their way and questions, the only people left will be the hardcore people or the fraudsters. They’re the only ones who have the patience to fight through these multiple extra questions. It’s really important that the good people get a smooth ride. We need to give them an easy ride and the fraudsters the hard ride. That’s basically how our system works. It’s updating itself every hour and changing every hour. It’s not something that you could do by hand at all. It wouldn’t work.”
Machine learning stack
“We use an awful lot of different components on Amazon Web Services (AWS). It’s a complex system to set up in the first place. It’s quite a high barrier to learning, but once you get over that barrier, it’s amazing the power that you can get for really amazingly small amounts of money. We run our machine learning $4 a day. It’s quite incredible, given all the data points coming in and what we’re doing. We went from much more bespoke to start with to actually just using the different components in AWS and just plugging them all together.”