Get a mop and wipe it up!

Survey Companies Cheating On Participants

, posted: 19-Sep-2012 19:12

There are various internet survey companies who pay their panels of members at below minimum wage to fill in online surveys.

The attraction to the public is a little pocket money. The survey company is attracted by the low cost -- far cheaper than paying staff to phone random telephone numbers. So a WIN-WIN situation.

Or is it?

For any survey, the company needs to know if it has reached its quota for each demographic sector, normally split by gender, age bracket, and location, and sometimes by other criteria, such as ethnic group, income, and survey-specific factors.

So, how are the companies cheating?

By asking actual survey questions, disguised as questions to filter out participants, before eliminating potential respondents. For example, I'd regard it as a reasonable filter question to ask about broadband v dialup internet. But to then ask which broadband company and which mobile phone company a person uses before the door is slammed shut...
Thank you for your interest.
We have reached our target number of participants and the survey is now closed.
There is normally no indication when the Actual survey questions begin. Nor any opportunity to comment on the filtering questions.

How is this cheating?

Because the survey company is gathering all the answers to these questions for free. There is no compensation for the time taken by participants to reach that point.

It is time for smarter survey companies.

Some companies maintain a detailed profile of participants to pre-filter surveys to better match their panelists. This is to be commended as it wastes less time. Companies should establish a reputation system to determine which participants on their panels are the most valuable -- across such metrics as Honesty (via consistency of answers), Availability (via speed of response), Intelligence (via comprehension of trickier concepts), and Suitability (as a measure of broadness of surveys for which they meet the filter conditions). I'm sure there are others.

With such an infrastructure, the survey companies would reject less participants, and gather higher quality data faster. They could afford to pay their quality panelists higher, attracting more people with quicker surveys (less mind-numbing filter questions) and more interesting surveys (aimed at a higher intelligence level).

Who will do it?



Other related posts:
Powershop Marketing promotes mass murderer
Best public service advertisement?
Six Things TradeMe Could Do to Improve Business

Comment by Linuxluver, on 19-Sep-2012 20:31

You're describing CB Clique, aren't you.

This filtering is you get points for completing the survey. After you're been filtered a few times you get a feel for where things are headed and you could - if you weren't honest - give them the answers they are looking for...complete the survey...and get your points.

Author's note by dmw, on 19-Sep-2012 21:54

No, I was actually annoyed by my latest experience with Valued Opinions for a survey about Utilities for which they'd pay me $0.50 if I completed it.

Looks like I shall skip have to skip everything under $1 in future.

But yes, it is easy to guess the answers they want in order to qualify to continue. That is why I suggest the rating system -- like TradeMe has -- to encourage the behaviour you wish to see.

Comment by Kyanar, on 20-Sep-2012 11:14

@Linuxluver - Colmar Brunton are probably one of the more reputable - generally speaking 5 surveys will get you enough for a $20 voucher, so you're getting about $4 for each one - a focus group will generally net you about $50 straight up.  Their Flybuys scheme is actually better value again, with a 15 minute survey giving you about 20 Flybuys points (which is equivalent to spending about $500 in a store) or 100 points for a focus group. $0.50 for a survey I wouldn't even bother with.  That's as ridiculous as SmileCity.

Comment by Russ Benk, on 28-Jul-2014 16:08

Yeah, I have noticed this and thought before about how much information the 'filter' questions are requesting. As of yet this problem has been minimal for me and hasn't put me off of online surveys (yet).

Add a comment

Please note: comments that are inappropriate or promotional in nature will be deleted. E-mail addresses are not displayed, but you must enter a valid e-mail address to confirm your comments.

Are you a registered Geekzone user? Login to have the fields below automatically filled in for you and to enable links in comments. If you have (or qualify to have) a Geekzone Blog then your comment will be automatically confirmed and placed in the moderation queue for the blog owner's approval.

Your name:

Your e-mail:

Your webpage:

dmw's profile

David White
New Zealand

Goon fan, .NET developer, contrarian seeker of truth