The Landmark Blog

Mobile Market Research: Love or Hate the Box?

Do you remember a time when you or your children were young and you or they received a gift...a really big gift...a gift that was big enough that it came in a huge box?  Or, maybe you've purchased a new refrigerator, or a TV, something contained in a really big box to avoid damage.  What impact did the box have on you as a child, or your children if you have them?  I could be a little left of center (very probable) but often times the box holds more entertainment value than the gift or purchase itself. 

Let me give you a couple of examples.  Recently, my family purchased a new TV.  My youngest son took the box it came in, threw a sleeping bag in it, and slept in it for three nights in a row.  A little strange.  When I tried to throw the box out, he raised royal @#$@, enough that the box remained in his room for about a month.  The thing just sat there and took up space in his room.  In the background of my mind, maybe I actually understand this more than I'd like to admit.

I remember a time growing up when my parents had to buy a new refrigerator.  Along with the refrigerator came the ultimate, the magnificent, the grand of grand, biggest box ever.  We lived in a rambler house with two levels.  We found many amazing uses for this box (breakdance pad for one).  Eventually though, we placed it on the stairs and used the box as a sled.  We started at the top of the stairs and slid down the stairs on top of the box.  After some time doing this, we realized that if we got a couch cushion, and opened up the box as big as it would go, we could surf down the stairs on the cushion and the box.  It was quite an adventure.  

Do you love or hate the box?  I loved it as a child.  As an adult, I don't like it so much.  As a child, a box felt warm, comforting, and downright fuzzy and enjoyable.  Now, as an adult, when I climb into a box it feels...cold, smelly, lonely, and claustrophobic?  Something has fundamentally changed...I guess I could get all Freudian on this, but I don't think that's what it is. 

The box has taken on a new meaning for me as an adult.  The box now refers to "THE BOX".  Like...are you in the box...the figurative box not the physical box (I'm a little too big...as in large...to fit in most physical boxes now days).  Are you in the box?

I've been doing a lot of work recently on Mobile Market Research.  I'm finding that this part of our research industry is really underdeveloped and is poised to make some pretty significant shifts and changes in the next few years.  My goal with this work has been to do two things...

1.  Figure out the best solution for this type of research (oh yeah...and implement a solution).

2.  Establish a policy and best practices for Discovery Research Group, my organization, so that we're positioned to be a leader in this area moving forward.

In the process, I've experienced a couple of eye openers that I'm not so sure about.

1.  There are some current thought leaders in this area that may be trying to force mobile research into a box that is a little to constraining for my taste.

2.  This forced boxdom (boxdom is probably not a word...but makes sense for this post), is largely dependent on the software these thought leaders sell, have on hand, or are most comfortable using.

There are emerging conferences that focus on conducting Mobile Market Research.  I believe that what they are doing is extremely important as they are bringing together organizations that have an invested interest in doing this type of market research correctly, accurately, and with as few quality concerns as possible.  In fact, later this year, the Merlien Institute will be holding a conference titled "Market Research in the Mobile World 2010: The Next Frontier".  My challenge is not with these conferences or what they are promoting, but with organizations that specifically "boxify" (again not a word) the Mobile Market Research market. 

Two different camps exist in this niche.  The first is filled with traditional vendors that define mobile market research as simply "dialing mobile phones from your market research telephone center," while the second camp snubs this group entirely, warns against them, and defines mobile market research as "conducting survey research using SMS, MMS technology" and even warns users of simplifying smartphone or mobile phone market research to only online surveys using mobile devices.  While a quieter third group focuses specifically on their proprietary mobile panels as the best or only "good" solution.

This is where my aversion to the box comes in.  I propose a mixing of these camps to one that focuses on using internet capable mobile devices to collect market research, regardless of the method of collection, in whatever way best suites the needs of the respondent...and to box it up as Mobile Market Research...because that is the device the respondent used to participate.  The brilliance and importance of mobile devices is their immediacy, their "in the moment" access, and their ability to use multiple technologies, telephone, IVR, internet capability, SMS, MMS, picture, video, etc. to accomplish a wider spectrum of research that has potential to consolidate both Qualitative and Quantitative methods.

I guess I just don't like the box...

• What do you think?  Too grand an aspiration, too simplistic, maybe too cynical?  Interested in your thoughts.

Don't Lose Your Patients: Healthcare and Patient Satisfaction Surveys

My first real career goal, the first one I really worked at, the first... "When I grow up I want to be..." was a stunt man.  I had a fascination with Evel Knievel (had his action figure and motorcycle...it's not a doll), as a teenager I loved Super Dave Osborne, and in my youth, I idolized Lee Majors in "The Fall Guy."  If a kid in elementary school could really work toward a career, it was me, I was serious about it.

I had a best friend who egged me on as frequently as he could.  He was my "Agent and PR man" (we were in grade school).  We spent many a day on his Idaho farm practicing for my stunt man career.  I'd dive headfirst off his fifteen foot high haystack to the ground and land in a small pile of loose hay.  Under his watchful eye, I learned how to gracefully "roll out" of falls from high places, backflip out of swings, fall out of trees, and ride pigs.  I even crashed his motorcycle and lived to tell the tale (it was the first time I had ever ridden one.)  Stuntmen need to be able to ride horses, motorcycles, and snowmobiles.  I learned how to do those things from my "Agent".

We often got into trouble, but along with the trouble came plenty of crashes, burns, scrapes, sprains, bruises, and at one time, a broken bone (or two).  The home I lived in while growing up was in the country and had a revolving outside laundry line.  You'd place your wet laundry on the lines, as the wind blew (the wind always blows in Idaho), the lines would spin around in circles, and your clothes would dry faster...ingenious.  My parents had a clothes dryer so the revolving laundry line was unnecessary for its original intent.  As youth looking for an adrenalin rush, my friends and I took down the long extensions (keeping the stronger short "arm" that spun) and tied a rope to the apparatus.  We also tied an old inner-tube to the rope and would sit in the contraption.  Someone would push, while the rider sat in the inner-tube and spun.  You'd swing around in circles a million miles an hour horizontal to the ground and get dizzier than you could imagine.  I'm sure you see where this is going.

One morning while waiting for the school bus, a friend and I were messing around on the "swing".  My friend was a year older than me and kind of a big guy.  He was able to really get that thing rotating very quickly.  In the corner of my yard was an old tractor tire that my mom had converted into a plant box to grow strawberries.  That morning, we swung especially fast, the rope broke, I went airborne, and when I landed my arm slammed into the tractor tire.  My arm hurt pretty badly so I went in the house to tell my mom what happened and that my arm hurt.  When she took off my coat, you could tell my arm was broken because it was shaped like the tread on the tractor tire.  We headed to the doctor's office and found it was broken in two places through both bones.

Each year, hundreds of thousands (millions) of healthcare patient satisfaction surveys are completed ranging from patients that have broken bones or been to the doctor for a cough, to long-term home health and hospital stays.  Patients with terminal illnesses are surveyed and in some cases surveys are done with the families of those that have passed.  Discovery Research Group conducts a large number of these healthcare related satisfaction surveys, health services satisfaction, or patient satisfaction surveys each year.  We work with some of the largest healthcare research organizations to conduct this work for hospitals and health plans, and work directly with some of the small or middle sized clinics and local companies who do not have the research budgets to implement the full scale healthcare research initiatives that these large healthcare research organizations provide.

I was in a conference session several months back with a number of researchers who were interested in healthcare research.  When the speaker surveyed the audience about how many were doing healthcare work on a large scale there were very few of us.  I was a little surprised that our expertise in this patient satisfaction survey arena was as unique as it was.  Along those lines, here are ten suggestions for surveying healthcare respondents (especially patients and participants) the next time your research calls for it:

• Be sensitive to the patient being surveyed.  But, focus on being unbiased in the process.  It's a fine line.
• Be painfully aware of HIPAA and privacy related issues.  And, monitor the entire process from front to back (interviewers, online methods, IVR, etc.) for compliance to law, rules, and training agendas.  Include the sponsor of the survey in the compliance plan.
• Find an effective way to juggle the need for good response rates with the need to be sensitive to respondent’s ailments.  It can be difficult.
• Consider multiple methods of collection. Multiple survey methods will help maximize response rates and provides the patient with different ways to vocalize their experience.
• Avoid barriers in survey programming that cause a choppy interviewing process (too many screens before "actual interview", surveys that are too long, survey process that doesn’t make sense, not enough information, too much information, etc.).
• Track your patient sample closely, down to the respondent level.  Healthcare research is very rigorous and a reporting at the patient level should be available for every sample record, without compromising the patient's personal information.
• Provide detailed instruction to technical and interviewing staff on the information that can and cannot be disclosed to the household while getting patient on the phone, while speaking with family members, while interview is completed, etc.
• Clear answers to frequently asked questions (FAQs) are very important.  Provide information on how to answer the difficult questions that will inevitably come up (Where did you get my name?  How do I reach someone at the hospital?  Who should I talk to about…? Why are you interviewing me when I’m sick?  I'm feeling like...should I go to the doctor?).  There is a lot of sensitivity surrounding healthcare related issues and the probability of escalation increases when staff is unable to answer patient questions clearly and concisely.
• Help lines should be established for verifying the legitimacy of survey calls.  Also, make a decision in advance on how to approach the "feedback" you’re provided that is directed to the practitioner, the survey sponsor, the research company, etc.
• A lot of patient and healthcare survey work is done using telephone methodologies for privacy related issues.  Be careful of the dialing times.  Don't dial too late or too early in case the respondent is still recovering.  Also, try dialing at different times during the day and on weekends.  With recovering patients this is critical as they are often in appointments, home during the day, and sleeping earlier and longer.

This is NOT an inclusive list for interviewing patients.  Feel free to add additional recommendations to this post.

Practices from the Field: Telephone Surveys and Mobile Phones

I'm dating myself here, but do any of you remember party lines?  And no, I'm not referring to dialing up your friend and organizing a party by calling everyone that you know...and that your friend knows...and that his friend knows...etc.?  Though social meanings change drastically in short periods of time, I'm talking about something else.  If you do an internet search on "party line," the first response actually generates an adult content web site.  Look past that to the Wikipedia posting of "party line" for the "traditional" meaning of the word. 

Party lines date back to the days when underground lines (Local Loops) weren't available to everyone, and households had to share telephone lines rather than being directly connected to digital lines in a million different possible ways.  A party line was when two or more households shared one telephone line.  In a party line, different houses had unique rings (now that's an old-school ringtone) that identified which household was being dialed.  For instance household #1 would have two short rings, while household #2 would have a long and a short ring, household #3 would have two long rings, etc.  Though almost obsolete now, party lines held out in the "country" much longer than they did in urban areas and may still exist in some very rural areas that aren't easily connected directly to the "local loop."

The real disadvantage to a party line is the ability to eavesdrop on your neighbor (some would call that an advantage).  If you were quiet enough about it, you could listen in on any conversation you wanted (take that all you folks whose entire jobs are aimed at protecting PII).  Remember when your younger brother or sister, or your parents would listen in on your phone conversation (before you had a cell phone), party lines took "listening in" to a whole new level. 

Aside from the eavesdropping aspect there were other disadvantages.  For instance, in the early days of internet...could you imagine what your dial-up would have done to the party line?  For that matter, a long conversation (a simple phone interview) could tie up the other household's line for what would seem like years.  In the heydays of party lines, early market research interviewing was mostly conducted door-to-door (face-to-face) and the market research sampling was address based.  As telephone technology evolved, better digital capabilities emerged, and households were able to connect directly to a line without sharing it with their neighbors, RDD Sampling was a more feasible option and telephone interviewing took off.

As a market research industry, we're back to a tipping point for survey interviewing and collection.  Online interview growth has started to level off (an exciting development for those of us that have huge banks of phones for telephone interviewing).  But, wireless households are growing very quickly.  The last CDC measurement of wireless only households fell out at 25%, while wireless "mostly" households add another 16%, totaling 41% of households who are wireless dominant.  Contrary to the last few years that they tracked this, the Pew Research Center has started to see some coverage response bias related to the data they collect, though still relatively minor depending on the demographic characteristics of those being interviewed.  For phone interviewing to stay relevant to the future of our industry, we must deal with cell phone interviewing and representation NOW.  It's our responsibility as providers of these services to provide solutions and best practices for cell phone interviewing.

Under that assumption, here are a few sampling solutions that are gaining traction in this arena:

• A return to the address-based sampling methods of old.  With the ability to port numbers, eliminate landlines, etc. address based sampling is emerging as a very good option for telephone interviewing and is obtainable through many sample providers.
• Supplementing RDD samples with Cell Phone samples is a good idea to consider.  Although, it's worth mentioning that just because a number is a cell phone, it doesn't necessarily mean that it comes from a cell-only or cell-phone dominant household.
• Client lists that include cell phone sample, seem to be more prevalent than in previous years and a much larger part of the industry.  This solution though used more and more, doesn't address market research projects whose methodologies rely on RDD based sampling frames.

Assuming that we will see an increase in cell phones in our market research samples, here are some suggestions for telephone interviewing cell phones moving forward:

• Ensure TCPA compliance by running samples up against cell-phone lists like Neustar (ported numbers from home to cell) and Telcordia (provides information on cell phones and allows you to identify cell phones against your sample list).
• Hand dial records that are identified as cell phones through these sources.
• Provide multiple methods for completing surveys (telephone, online, IVR, inbound, etc.).  This will help you reduce costs associated with telephone interviewer-only methods.
• Consider incentives for samples that are heavily aimed at cell phones.

And, here are some evolving suggestions for telephone interviewing as a result of this trend toward increasing cell-phone representation:

• Limit the length of the interview.  You've heard it before, you'll hear it again.  The shorter the interview, the less it will cost to complete for you, your supplier, and the respondent.  
• Screen for respondent safety.  Ask a quick question about whether the respondent is in a safe place where they are free to talk.  If they are driving or are operating heavy machinery, terminate the interview.
• Be sensitive to subject matter.  Respondents who are participating in surveys over cell phones are more likely to be in public places than when using landlines.  Be considerate of their location when doing surveys where PII (personal identifiable information) is a concern.
• Screen for age.  Many cell phones are owned or dominated by under-age users.  Consider the ethical issues associated with interviewing respondents who are under 18 years of age.  Ask the respondent before you consider the interview.  
• Consider interviewing during "free-periods"...nights and weekends.
• Provide local numbers.  Many forward thinking telephone long distance providers are now supplying the option to provide a local number on caller id.  It's worth considering a local number as opposed to a toll-free 800 based number.  They're both free to the respondent if called back and the local one is more likely to be answered.

Surprisingly, in the study linked to above, Pew is finding that response rates for cell samples are very similar to the response rates for landline samples.  In addition, they are finding that in some cases, cell samples are more cooperative than are their landline counterparts.  Cell phone interviewing can be done as effectively as "standard" historical land-line interviewing.  It will take some thought, some evolution, and some commitment to doing it right.  What's new?

I'm sure I've missed many suggestions.  Please provide them if you'd like to add to this list.

Robots and the Uses for IVR in Market Research Surveys

Did you ever used to watch the TV show "Mystery Science Theater 3000"?  The show was extremely funny and very creative.  The basic premise was to take a bunch of old, corny, black and white science fiction and horror movies, play them in the background on a large screen while a couple of "robots" and a human make fun of them.  Sounds like a strange premise, but it works.  If you've never seen it, make the jump (after you read this blog) to their website and check it out.  Very funny stuff.  

The thing that every person knows about robots is that, like Mystery Science Theater 3000, all good robots should have the ability to speak.  Market Research has speaking robots of their own.  If you're involved in doing telephone market research, you've heard of "robo-dialing" or "auto-dialing" and "interactive voice response - IVR".  Take a look at this picture...now you know who's making these automated research calls.

It's been a while since I really geeked out and posted an article that was extremely focused on methods of collecting market research data.  This one has been rolling around in my brain for some time.  For awhile now, I've wondered why more people don't use IVR with more regularity as a method for data collection.  I've come to the conclusion that researchers may be less familiar with the process, probably don't completely understand how it operates, and don't really make the connection to its benefits.

IVR or Interactive Voice Response has been around for quite some time.  About once a week, I receive a phone call from one of my children's schools telling me that they were absent from a class (most times we knew about it).  Candidates in the political arena often use the technology for "Get out the Vote" campaigns. Though it can't be used for research purposes, basic IVR phone technology is built into the structure of most company's telephone systems (ex. when the "phone tree" asks for the extension, your voicemail, etc.)  

The 2009 version of Rockhopper's "Research Industry Trends" report shows us that only 11% of companies actually use this market research data collection method.  I really wonder why the technology is used so infrequently, especially given that it has many of the same advantages as online research, and overcomes some of the barriers.  Here are some advantages to using IVR as a method of collection: 

• IVR deals very effectively with the self selected sampling methods comparable to what you see online (inbound calls from "current customers").  This is the IVR use that most people are familiar with. 
• Most IVR systems have the ability to make outbound calls.  IVR is a capable method for Random Digit Dialed (RDD) or client listed outbound studies.
• Because IVR is automated, the labor associated with the job is minimized.  The net result is a cost per interview that is much less expensive than "typical" phone studies and similar to online costs.
• Similar to online research, the turnaround time on IVR studies can be much faster for outbound dialed projects, with many of the same benefits of phone dialing and fewer of the "risks" associated with online research.
• IVR is particularly capable of generating qualified responses on low incidence studies.  Most systems can be set up to dial through the records and giving the respondent the choice between being transferred to a telephone interviewer (called a Warm Transfer) or completing the interview with the automated system.
• Many IVR systems store data in databases which are easily plugged in to data or research dashboards.

What are your thoughts.  Why don't more people use IVR in their arsenal of market research methods of collecting market research data?  Are there uses for IVR that I've missed?  What about barriers I've overlooked?  I look forward to your responses.