Marketing “research”

We commissioned third-party firm contracted to do some marketing “research” for us. (And for those of you who are new, these: “” are air quotes.) The group has been disappointingly disorganized.  We agree to schedule a meeting to discuss their results at 3pm Monday, and by 3:05pm Monday, we’re calling their answering service trying to hunt them down. “No, we did not receive the document. Did you actually hit Control-Enter. You know, send it?”  Or we can hear the guy paging through them, much like a general practitioner does just before he (or she) walks in to administer the physical.

Today, we went over the results, and the presentation wasn’t particularly helpful.  For example, suppose they asked “What is your favorite color?“  The marketing “researcher’s” results look like this:

What is your favorite color of toilet?
a) Red - 12%
b) Green - 9%
c) Blue - 6%
d) Other - 73%

The problem?  The “other’ responses are a huge chunk of the survey results.  From this scant information, we don’t know if this represents people naming all of the paint chips in Home Depot (e.g., seashell pink - 0.1% chartreuse - 0.07%, etc) or, more likely, there is an effed up choice selection, missing something obvious (e.g., White: 40%, Ivory: 33%, Pepto-Bismol pink: 0.000000007%)

Another component of the “research” delivered is metrics of customer satisfaction and perceived importance of several features.  This type of question is a two-parter where the “ideal” score is when a customer’s satisfaction equals their perceived importance.  For example, if we were selling sinks, “technical support” might yield an importance of 1 out of 5.  Receiving a 5 out of 5 on technical support suggests we  may be spending too much energy into that aspect of our solution.  Conversely, not leaking might be a 5 out of 5 importance.  We’d better hit that.

Now, suppose the results look like this:

Sink Quality	Rank	Importance	Satisfaction	Delta
Does not leak	1	5.0	4.6	-0.4
Stain-resistant finish	2	4.5	4.2	-0.3
Connections for hot and cold water	3	3.8	3.9	+0.1
Technical support	4	1.5	1.9	+0.4
…	…	…	…	…
Large enough to wash a dog	10	0.5	0.3	-0.2

I’d conclude we should be focusing our resources on ensuring the “sinks do not leak” and “stain resistant finish.”  They’re the most important and exhibit the biggest gap in satisfaction. The market researchers see it differently, recommending we should focus on making sinks “Large enough to wash a dog.”  When pressed to explain, they trot out several statistical metrics to nine digits of precision: Cramer’s V, Wilks’ Lambda, chi-square, df, eigenvalue and canonical correction.   I don’t know what most of these are, nor should I need to.  The meeting kept getting hung up on questions like:

Us: Why is the “Smells of bacon (-0.440)” is ranked between the “2-inch drainpipe (0.512)” and “Magneto-optical soap holder (0.393)?”

They say: “Gaussian Globorfrotz is absolute value.”

Me: “Well, why didn’t you just take the sign off?  [Secretly wanting to ask: Did you guys make up Globorfrotz?  It gives me flashbacks to El Jefe."]

The raw data I received is separated into one file per question.  It’ll take me about four hours to assemble, but the results will be more useful.