How Not to Do a Survey

I was on a Delta flight the other day (don't get me started on how bad it was, beginning to end) and decided to take the nifty touch screen survey on the screen on the back of the seat since it was the only part of the entertainment system that WAS working.

I thought it was going to be about customer satisfaction because at this point suffice it to say I was not satisfied. This has nothing to do with the slouched over guy with gross bare feet two rows up, nor the guy eating one of the enormous sandwiches you have to buy which from what I could tell had a small slab of roast beef with no less than an inch of mayo. And no it wasn't the fact that the seat in front of me was reclined so far into my lap that I couldn't open my laptop...nope that wasn't it at all.

So I get to the survey and each question is rated on a scale, 5 points down to 1 from Excellent, Very Good, Good, Poor to Very Poor. Each question had it's own screen. Fine so far.

Here's where it gets bad:

1. Each question had PRE-SELECTED "Excellent"! Worse than that it was so subtle that you were onto the next new question after you hit "continue" that you just keep pressing continue (or Excellent) until you're done with the survey! I ask you, what's the point?
2. Then they asked about the condition, functionality, and cleanliness of my tray table, my seat cover, the floor around me, my window, my seat pocket, the carpet, my seat cover (and it's decorative trim) and the functionality of my seat. (Nowhere could I tell them about the fact I couldn't open my laptop)
3. There were no open ended questions - They even have a keyboard on another part of the survey.

What about the entertainment system, many times the only thing that gets one through a flight (and the fact that it wasn't working), or the quality of the mayonnaise sandwiches with a side of roast beef slice? What about the food service you pay for where they ran out of most food by aisle 34 and there were 44 aisles on the plane? What about the disposition of the flight attendants, which I actually would have rated them high on with hell breaking loose?

If they wanted to know how to redecorate the plane, they should have started off the survey with "Flight Redecoration Survey". If you're going to do a customer satisfaction survey, especially in the middle of an experience, make sure you ask the tough questions. Knowing the answers will only make your business better.

Comments

[Sheila Hibbard]

I found your customer sat survey experience very interesting. It reminded me of how many clients over a 30 year marketing research career I had to educate about the science behind questionnaire design. Why it was important to include open-ends even if the responses would require time consuming coding. Why the order of questions was important so as not to bias the respondent's future responses. Why it was important to include all aspects of whatever was being measured to ensure nothing was being overlooked or assumed. Often the hardest part was to get clients to think about what they were going to do with the information once they received it.

Unfortunately, experiences like yours, only go to reinforce the negatives of a company. The company, conversely, thinks they're doing a 'good' thing.

It is interesting in this era of technology that almost anyone who can type thinks they're capable of designing a good study or a good questionnaire. Just type in one of the hundreds of the internet survey tools and they too can claim they did 'research'. The rule still applies...garbage in, garbage out.

[Miguel]

It's a natural human tendency to "kick the dog" after having an unsatisfactory customer experience. Yet, in my business I find many of my customer's expectations way out of line with what they are willing to pay. They assume they have the right to be "competely satisfied - or their money back". This notion has become a myth in today's business environment. Although survey's are usally a feeble attempt to gain a relationship with customers, they are by no means the culprit of poor execution on the company's part OR elevated expectations on the customer's part. If you are a frequent flyer my suggestion is to entertain yourself during your next flight and not expect the airlines to make you a happy camper. You could have purchased books, games, knitted a sweater, put an audio book in your iPOD and the problems of dissatisfaction would evaporate in seconds. I do agree with the premise that an airlines shouldn't try to "fake-out" building relationships with surveys, it still would be remise on their part if they did not offer them. Morale of the story: Don't expect jumbo companies to make you happy, take on some that responsibility for yourself.

[Travis]

Honestly what does it matter. I have never in my life seen a company respond or change due to a survey. In my opinion the only reason for a survey is to make the customer feel better by creating the illusion that they were actually heard.

[michelle]

All too often, companies launch these surveys without giving it proper thought. And surprisingly, it's often the large corporations (that surely have it within their budget to do sufficient research) that make this totally avoidable & stupid mistake.

Open-ended questions are a must! You cannot predict every possible scenario that a customer will experience. Simply adding an "other" or "comments" field allows for more valuable feedback. At least they gave you a 5 point scale - I've seen surveys that offer 4 options .. so you either are "for" or "against" essentially. This ensures that response data is skewed.

As for the pre-selected options, well that's a bit optimistic & conceited to say the least. But it could be over-ridden, correct?

You're right though - asking the tough questions is crucial. How can your business improve if you don't know where you are going wrong?