Otter ResearchBranding and Marketing Research FirmLos Angeles California USA +1 310 937 3434Home | Contact | Culture | Experience | Expertise | Partners | Perspectives | Resources | Roles | Blog |
|
Resources | FAQ
Five Questions To Ask Your Research ConsultantUltimately, we want your business. However, given our academic backgrounds, we also want to educate consumers and companies alike on how to discriminate good research from bad. In the end, we believe that when you understand some of the basics of what constitutes proper study design and data collection, analysis, and presentation, you will make more informed decisions as to who to hire for your research needs.1. What Range Of Projects Constitutes Your Experience?Some research consulting firms specialize in serving commercial business with typical marketing issues. Some specialize in internal auditing. Others are best-suited for working with government agencies. We believe that we are one of the few who have a truly wider range of experience.To look at our past and current clients is to see an incredibly diverse range of interests. We have worked with and for academic institutions, civic institutions, non-profits, city governments, entertainment groups, private individuals, commercial developers and the list goes on. The point is that we are very likely to have prior experience with the type of help you need and we know how to deliver it in efficient and effective ways. And as value added, lessons learned from our vast experience enhances the quality of what we bring to any project. 2. How will the analytic sample be determined?Who do you care about? What is your target audience or consumer? Who do you want to become a client that isn't currently? These are all questions that should determine who you will want to include as research participants and from whom conclusions will be drawn.Much of commercial and retail research draws on samples of convenience, and are not drawn in a systematic way, meaning that conclusions you draw from that data set may be limited in their generalizability to the audience you care about. You may only be interested in young adults, or senior citizens, or only college-educated women or only people who already have experience with your product or service, or only exclusively those who have not. The point is that these decisions must be made and we can help you make them to best suit your needs, within your constraints of cost and time. Sampling decisions ultimately determine what you will and what you will not be able to learn from your research. This is often a critically overlooked and poorly thought out step. Poor decisions here will derail your study before it even starts. We will help you think through this important first step so you fully understand where you research will be headed. 3. Will a mixed methods design be incorporated?Perhaps the best standard -bearing quote to which all research marches came to us from no less an authority than Galileo Galilei, "Measure what is measurable, and make measurable what is not so." And in more recent times Thomas D. Cook, who literally wrote the book on research design, tells us that, "All methods are flawed." So what to do?Some parameters come to us in a form that is easily measurable using standard statistics and mathematics. We call these types of data quantitative. Anything that in its basic form is already a number (think physical attributes, counts, frequencies) or that has been subject to commonly agreed upon scales of measurement (IQ scores, temperature, etc…) is considered quantitative. The vast majority of scientific inquiry relies on quantitative data. But there is one major problem with that. Imagine you asked a TV viewer to rate how enjoyable or watchable a certain TV ad was and you provided them a scale ranging from 1 (very enjoyable) to 5 (not enjoyable). You could get this consumer to rate dozens of ads. In fact, you could get a thousand consumers to rate dozens of ads. What I have just described is standard operating procedure for many marketing and advertising firms. Aggregating average scores across these respondents and perhaps breaking the results down by age, gender and ethnicity would pass as pretty good research in these fields. But here's the problem. You are no closer to understanding why a certain ad is enjoyable or not than you were before you began. You have no idea whether Person A who is a 40 year old, Chinese American male rated the ad a "1" for the same reason that the Person B, a 30 year old African American female who also rated it a "1" did. Could you follow up each rating question with a series of additional questions trying to pinpoint the basis for each rating? Yes, but that leads to another problem. How do you know which characteristics or attributes of the ad to ask about? Unless you are prepared to ask an infinite number of questions, it is near impossible to guess what questions will uncover the "why" of each viewer. The solution? Let the viewer tell you what why they rated the ad the way they did. In other words, you need open-ended questions that allow a participant to use their own words to explain. This is an example of qualitative data. Qualitative data most commonly takes the form of rich, verbal or written material, as you would generate in an interview or open-ended questionnaires, or from field notes that someone would take when making observations, or from focus group discussions. In all these cases, what you end up with is something that gives you a much deeper insight into people's thoughts and opinions. The problem is how do you take all that stuff and apply systematic analytic techniques that allow you to make reliable and valid conclusions from it. This is when you call in experts like us who spent half our lives getting Ph.D.s in research psychology, anthropology or sociology. There are specific, well-established protocols for how to best get and utilize qualitative data. However, we take it even one step further. The ideal is to combine both quantitative and qualitative approaches, giving you the best of both worlds, specifically the rigor, objectivity and statistical allowances of quantitative designs and data and the richer, more descriptive and personalized attributes of qualitative data. There are very few people who know how to properly design, implement and analyze mixed methods approaches and data. This is what we specialize in and when possible, prefer to conduct all our research using mixed methods designs. We speak both languages, quantitative and qualitative. This is fundamental to a researcher's ability to appreciate the strengths and weaknesses of each and how to draw on them to maximize the effectiveness for any single study. Further, research design decisions and findings must ultimately be communicated. Having practiced these languages in many situations, we are primed to communicate with you and your target audiences in ways that will engage and inform. Much of our academic work is dedicated to the innovation and application of mixed methods toward capitalizing on the strengths available and fitting them to the nuances of particular projects. As professionals, we invent this stuff, we write about it, we do it, AND we can do it for you! 4. How will the reliability of the research surveys be determined?Anyone can create a survey that on its face, appears to ask the relevant questions. The real trick is to design a survey or any research instrument that is also reliable. Reliability in the research sciences refers to a specific mathematical relationship of the participant's responses across survey questions. Why is it important?Imagine a bathroom weight scale that due to age has a faulty spring that is thrown off by changes in humidity and temperature. Sometimes the scale is off by as little as 2 lbs, sometimes by as much as 10. How useful would this scale be, day in day out, if you were trying to manage your weight and diet with it? Not much value at all because this scale is unreliable. Its inconsistent readings interfere with your ability to determine your "true" weight. Surveys suffer from the same potential problem. Some items because of the way they are worded or because of the type of response scale used can introduce random error into your data, making it difficult to impossible to determine the true scores. As a matter of course, we always subject our scales and instruments to psychometric validation procedures to determine the reliability of the instruments we use. We will never deploy a scale that does not meet widely accepted reliability standards. The same must also be said for qualitative methods. Far less well understood by many researchers, even in academia, systematic procedures for assessing and assuring reliability with qualitative methods are available-and we ALWAYS use them and we can do the same for you. One way to understand this is by thinking about the difference between stories/anecdotes and evidence. How many marketing researcher reports only use interviews or focus groups to tell stories? We would venture to guess "most." We are the exception here. We know how to glean sound and systematic evidence from our qualitative data. Stories or evidence? Which would you like to be using as you make your decisions? 5. How will the results of the research and any analysis be communicated?We know how to do some pretty complex things, but we will not dumb things down when presenting them to you. We respect our clients and will take whatever time and effort is necessary to help you understand the how and why of everything we do. And perhaps as importantly, we will help you package our analyses and research in a way that will allow you to communicate the findings to whatever audience you need to, be they the public at large or to your own employees or department heads. No Wizards Behind Curtains here.In The LimelightThe Otter Research BlogOur WorldBrent ScarcliffEvocative Places Community, Destination, and Tourism Brand Consultants Extra Meaning Brand Communications, Product Placement, and Public Relations Consultants Fresh Meaning Brand Naming Consultants Joy Salvador Otter Brand, Consumer, Culture, and Market Research Consultants Resounding Success Company and Product Naming Consultants Rich In Meaning Book, Game, Movie, Music, Show, Television, and Themed Entertainment Consultants Rich With Meaning Lifestyle and Luxury Brand Consultants Savaiki South Seas Lifestyle Company Savory Characters Food, Beverage, and Restaurant Brand Consultants Scarcliff Salvador Cultural Branding and Naming Agency The Brand New Meaning Blog The Cultural Riches World, Pop, and Brand Culture Video Blog The Earthy Language Lexicon The Genetic Linguistics Blog The Names We Love The NBA Player Nicknames Blog The Pop Language Blog The Richly Evocative Thesaurus of Semantic Spheres The Scarcliff Dictionary of Branding The Scarcliff Directory of Branding The Take My Meaning Dictionary of Aphorisms, Proverbs, Quips, and Quotations The Titles We Love The Wildly Colorful Wildlife Photography Blog The Wonderfully Evocative Dictionary of Expressions, Figures of Speech, Idioms, Phrases, and Sayings The Words We Love Otter ResearchCulture and Consumer Research FirmCopyright© 2009 Scarcliff Salvador Inc. All rights reserved. All trademarks are the property of their respective owners. |