Presentation for the DocSig AMA Academic Winter Meetings 2022, Las Vegas
Linda L. Price
Thanks so much for inviting me to do this session with the wonderful AMA DocSig. So happy to see all of you here in person. I’m honored and delighted to be here. The specific title for today’s discussion was inspired by current work I’m doing with Amanda Garrison, a 4th year PhD student at University of Wyoming. More generally, it was inspired by many years of working with PhD students, struggles with getting my own research published in top journals, and my experience as an Editor, Associate Editor and reviewer for many of our top consumer research and marketing journals.
As a forewarning, Amanda’s dissertation focuses on adult amateur boxers, who in their 30s and 40s decide that what they really want to do is get into a boxing ring and get hit and hit someone else in the face—it’s one-on-one intentional, corporeal violence. And, surprisingly (at least to me), it’s a rapidly growing leisure activity (300% in 5 years). To qualify to get into the boxing ring, wanna-be boxers must pass through many milestones, including sparring matches, that test their strengths and pinpoint their weaknesses. Many wanna-be boxers realize at the stage of sparring that it’s not for them—when the first friendly blow lands on their face and really hurts, or they are expected to defend their stance by fighting back with skill, agility and no fear.
I don’t want you to think of submitting to our top journals as entering corporeal combat, but I do think that the metaphor of preparing for a decisive match, by making sure you’ve successfully attained other milestones is critical. And I wouldn’t be lying if I said that sometimes the review process feels more like a boxing match than many of us would like to admit. Throughout this presentation, I will refer to milestones along the research journey to publication, and I will also broadly reference the activity of sparring. By milestones, I mean markers that when attained reassure travelers that the proper path is being followed. Attainment of these milestones signals distance travelled and the remaining distance to a destination. Sparring is generally, but not always, linked to combat sports. Sparring is intended to educate the participants and builds skills of improvisation while identifying strengths and weaknesses in a variety of circumstances. Mostly, what I’m going to talk about is Research Questions as a challenging and significant milestone in the journey to publication.
Many of you may know, from reading or seeing one or more of my presentations related to “Thinking Theoretically” or how to carve out an academic career, that I have long touted the importance of embracing your “burning questions,” as a key step in the research journey and toward a long and satisfying academic career. I talked about this in my 2014 ACR Presidential address, observing that people such as Peter Wright and Elizabeth Hirschman were important early role models for me on my journey because they always let their own vexing questions be their guides (Price 2015). I too, always hope, that the answers that come out of my research, will be useful for me in my own life. In my ACR Presidential address I suggested choosing co-authors, not just or even primarily for their expertise, but rather based on their passion, determination, and persistence for finding the answer to a shared “burning question.” I also tried to assure the audience to hang in there, even when others don’t share their own burning questions, and gave several examples of my own consequential research that at the time was considered irrelevant by many of my well-meaning colleagues.
In my 2019 ACR Fellows address, I doubled down on the perspective of pursuing your burning questions with willing, fellow travelers (Price 2020). I identified the quest as the most important part of research, a quest being “a long search for something that is difficult to find.” Whether studying fruit flies, indigenous peoples or family heirlooms, underlying research quests are the burning research questions that fuel them. They are the belief that if you had an answer to a particular question it would make a big difference in your beliefs, behaviors, and outcomes and perhaps the beliefs, behaviors and outcomes of other people. I gave as one of my examples, Connie Pechmann, who has spent her entire career trying to answer the burning questions, “How can I keep people from smoking, or help them stop?” In the ACR Fellows address I unfolded more about how I think we find burning questions, noting that burning questions rarely come from observing thousands of data points, but “from the careful observation of just one thing—a person, an event, an experience, an object, a story—a single leaf, or a chance encounter with a total stranger,” (Price 2020). I draw on several examples in my ACR Fellows address, but a favorite is Susan Fiske, the social psychologist whose seminal work on stereotypes, prejudice, power, gender, social cognition and dehumanization started when she moved to Boston for college. I’ve cited her many, many times. She describes:
I was struck by an absence that took me awhile to place. Although there was the right amount of fluffy white stuff, the people were far too white. The lack of ethnic variety in the Boston I encountered—the result of heavy de facto segregation—seemed odd to me. Probably primed by my mother’s interest in communities, I couldn’t figure out why people would want to live that way. Probably primed by my father’s orientation to research, I realized there must be empirical answers, (Fiske, 2004, 71).
I still believe the first milestone in the research journey and the journey to become a happy and productive academic scholar is to find a burning question. A burning question provides direction, it motivates and accelerates action to reach a goal. It helps the quest endure when you are hungry, tired, cold and feeling defeated. With the push for increased relevance in research from all of our major journals, accrediting bodies, and academic institutions, combined with the dire condition of our planet, many scholars are increasingly turning attention to individual and shared burning questions—how can we reduce waste, how can we reduce and/or manage violence, how can we get people to stop driving their cars so much and/or get them to buy electric cars, how can we address racial and economic inequity and so on.
Unfortunately, burning questions rarely come in the form of a research question. Burning questions are more personal—How can I understand XYZ, and/or Why is XYZ happening? Or, they are more action oriented and managerial—how can I stop, prevent, increase, start or in some way change XYZ? Moreover, burning questions don’t imply a theory or a method, they could draw on many theories and engage various empirical methods. The path from a burning question to the next milestone—a research question, is fraught with unexpected obstacles, danger, and detours. When I met Amanda last year, she had her own burning question, and she had incredible access to as much data as anyone could desire, but she was really struggling with how to identify what theoretical conversation she could meaningfully contribute to. I would hazard a guess that, although we are both doing a lot of research question sparring, that struggle is not over yet. It made salient how difficult it is to get from a burning question to a research question.
Whereas we receive quite a bit of guidance on many aspects of the research process, we receive little guidance on how to get from a burning question to a research question even though everyone agrees that a good research question is essential. That is not to say that we don’t have classes on how to write hypotheses, but in my experience the chasm between the hypotheses young scholars test in their first experimental design class, and their own burning questions is often formidable. To be honest, even with many experimental design classes under my belt, lots of published ethnographies, and numerous years of research experience, I still find it incredibly difficult to get from a burning question to a good research question.
A useful source I uncovered a number of years ago for thinking about research questions, and one I regularly use in my PhD Theory Construction seminar is Robert Alford’s book The Craft of Inquiry (1998). He writes about the difficulty of getting from a problem (a burning question) to a research question, observing:
Formulating research questions commits you to a lengthy process of intellectual work. Since a research project may last for several years, it is easy to flounder, go down dead ends, and become paralyzed with self doubt. …The process of defining research questions is essential to learning how to think critically about the research literature in relation to your own work (Alford 1998, p. 25).
Alford provides the best definition of a research question that I’ve run across to date, noting:
A research question is literally a sentence that ends in a question mark and in which every word counts, one that points in two directions—toward the theoretical framework that justifies the question and toward the empirical evidence that will answer it. A research question is a commitment to a way of framing the problem for the research to begin or continue, since research questions usually change in the course of the project (Alford 1998, p. 25).
Critically, the research question articulates the particular theoretical conversation that is joined and the empirical evidence that will be used to answer the question. The framing of the research question tells us the focal theory or theories as well as the empirical approach. Let’s consider some examples. Consider the research question for a recent paper published in JCR (Van Laer et al 2019):
How does the degree of narrativity of a consumer review affect consumers’ narrative transportation and persuasion?
Based on the research question alone, we can infer quite a lot about the theoretical conversation and the empirical strategy that will be employed. This question implies many strategic researcher choices. We know that this will not be qualitative research, because the research question is of the form how does x affect y, and we know that three constructs are key to the conversation joined: degree of narrativity, narrative transportation and persuasion. Depending on our familiarity with this research stream, we might also guess that the big take away is connecting a new measure of “degree of narrativity” to more familiar research constructs of narrative transportation and persuasion, and doing that in an important emerging substantive domain—online consumer reviews. It’s important to remember that concepts are historical and their meanings can’t be arbitrarily changed by the next user. So, while the authors are elaborating and extending narrative theory, they are also drawing on rich historical traditions of meanings given to narrative transportation and persuasion by prior research within consumer research and literary theory.
As another example, consider this research question from a recent JCR article (Goor et al 2020):
Can luxury consumption, alongside providing desired status benefits, also make consumers feel inauthentic, producing what we call the impostor syndrome from luxury consumption?
Again this research question flags lots of strategic researcher choices. We see the basic form of can X lead to Y, and so we might anticipate that the research is not going to be qualitative, although it is less clear in this case than in the prior case. The question merely poses that this can sometimes happen, along with other outcomes, so qualitative research could also be used to answer this research question. The research is located in relationship to two conversations—a substantive one surrounding luxury consumption, and a theoretical conversation about inauthenticity. As many of you know, the authenticity literature is a huge literature, so again, we see a narrowing around self-authenticity, and in particular, the imposter syndrome. Not revealed in this research question is that the researchers will also investigate whether psychological entitlement moderates the relationship between X and Y, hypothesizing that it will. This moderator is backgrounded for good reason. While the authors find evidence for it, the more persuasive and interesting finding is that sometimes people shun “authentic” luxury goods because it makes them feel “inauthentic.”
A JCR article that many of you may be familiar with posed the following research question (Preece, Kerrigan and O’Reilly 2018):
How do serial brands attain longevity within evolving sociocultural contexts?
This research question doesn’t reveal the theoretical frame, but the very next sentence does. Specifically, “We adopt an assemblage approach to unpack this delicate balancing act between continuity and change,” (p. 330). The title also flags the joining of a substantive question, “how do serial brands survive in changing sociocultural contexts?” with the theoretical lens of dynamic brand assemblages, specifically, “License to Assemble: Theorizing Brand Longevity.” The authors combine this with an evocative serial brand—The James Bond movie franchise—as their focal brand assemblage. From the research question we infer that the research will be qualitative and process focused. We can predict that the authors will examine one or more serial brands that have survived across multiple changing sociocultural contexts in order to uncover how the brands survived. The focal question is how. Of course, with just a few different words, this question could be modified to change either the theoretical lens or the empirical strategy.
We could go through many other examples, but my hope is that these few examples illustrate how important the research question can be in framing the contours of an empirical strategy and theoretical contribution. The examples I’ve drawn on are now published, so we don’t know how many revisions these questions went through before their final form. There may have been numerous iterations before data collection was completed, before there was a working paper, and before a first submission. Through the review process, there’s a good chance the research question evolved as well. As Alford notes, research questions are always “successive approximations, as you learn more about the phenomenon being analyzed,” (1998, p. 27).
Researchers can enter the framing of the research question either from an empirical entry point or from a theoretical entry point. But then there’s a rolling process in which the researcher narrows iteratively both theoretical and methodological choices—exorcising particular aspects of the problem that the researcher doesn’t have the time or resources to deal with in order to focus on aspects of the problem that can be handled and finished. I could provide numerous examples from my own research of how the research question is narrowed around what is empirically and pragmatically possible. Referring to work by Jack Katz studying legal aid lawyers, Alford provides a succinct example that shows this exorcising and shifting of a research question to make it tractable (Alford 1998, p. 139, see also Katz 1983).
Katz first asked the question “Was there, I wondered, a common process of leaving the institution, or ‘burning out,’ as the lawyers put it?” He says that “it quickly became apparent that I could not hope to explain the difference between those who did and did not stay more than two years…So I changed, the definition of the explanandum to ‘desireing to stay two years’” (p. 131). In effect, he reversed the research question from studying why people left to why they stayed.
The practice of tackling your research question again and again, is one of the best ways to insure that you have a succinct, compelling story. We might metaphorically call this sparring with your research question—testing its strengths and weaknesses. Discovering if it can stand up as a contribution against prior research; putting it up against different scenarios and participants to test and improve its improvisational strengths. Testing it against different theoretical arguments for that evidence. Defining the research question requires mapping the intellectual territory, tracking through the jungle of relevant books and articles and formulating various research questions to think through both the theoretical and empirical implications until you land on a question where you can handle and finish the empirical portion, and it will contribute to the theoretical conversation you are joining in a meaningful way. In going through this process remember, in the words of Alford (p. 29):
- Rich data and rigorous evidence cannot replace a coherent theoretical argument, and
- Brilliant, logically consistent theoretical claims cannot substitute for evidence.
Although there is always slippage between the theory and the data, finding that near optimal match between the theoretical argument and the evidence is a significant part of what makes the researcher’s story come together. Obviously, the more you can put your research question up against all your own arguments and against all the arguments of worthy and willing opponents the more likely that reviewers of that first journal submission will be persuaded by your story. Sometimes, that’s worked out pretty well for me. For example, I went back through my records for the 2014 Journal of Marketing publication that I worked on with Amber Epp and Hope Schau. It was not published on the first round, but the research question stayed the same, as we tried on a subsequent round to better unfold the theoretical lens’ and better articulate the managerial implications. Moreover, the research question even maps to early presentations of the research well in advance of submission. In this case the research question also stands in as a title, if phrased as a question, e.g., What are the roles of brands and mediating technologies in assembling long-distance family practices? I have a few other projects where the research question evolved quickly and was robust through sparring and even in the journal review boxing ring.
By contrast, other projects I’ve worked on have gone into the first review with a research question that has been through lots of practice sparring, but still gets knocked out of the ring on the first round; is revamped with lots more practice sparring, and in its new and improved form, gets knocked out of the ring again in the second round (Godfrey, Price and Lusch, online 2021). I know from experience this can happen again and again. Thinking about my own failed efforts and those of other manuscripts I’ve reviewed, some common mistakes include:
- Great theoretical argument but the evidence doesn’t fit the theoretical lens. Sometimes this can easily be resolved by going through the data more carefully to close more of the slippage between the theory and data, and sometimes it requires additional data collection. Sometimes, despite a great theoretical argument the evidence would be too hard or costly to obtain, and the researcher needs to search for a different theoretical argument using a different theoretical lens. Sometimes the theoretical argument can be written as a conceptual paper, where evidence can be more loosely fitted and where it’s easier to comingle types of evidence and theory.
- Rich data is loosely connected to too many theoretical conversations. To resolve this requires asking again and again with each possible theoretical lens up against the evidence: ”what’s interesting,” “what’s important,” “what do we learn that we didn’t know before.” When joining a conversation where there’s a lot of extant research, it’s important not to give up too soon on that theoretical lens, but instead to try to narrow the lens to a very particular part of the conversation. We see this exemplified in the research question examples I’ve already discussed.
- Theoretical argument is linked to evidence, but it’s not interesting. This is one of the most common problems in the papers I review. Essentially, the theoretical contribution and substantive importance of the findings is not clear. Sometimes, this can be corrected by more carefully focusing the theoretical lens to foreground where the contribution lies. For example, this might include identifying new useful processes, new components, new mediators or moderators for an already known relationship. Sometimes, this can be corrected by showing the importance of how that known lens informs evidence in a very different poorly understood domain. In this case the emphasis is not on a theoretical contribution but on how the enabling lens of that theory dramatically changes how we see a particular domain. One of my favorite general sources for trying to avoid this problem is Murray Davis’ paper on how to ask interesting questions (1971).
- The theoretical argument is too big and/or too elaborate to parsimoniously evidence. This can happen in various ways. Sometimes the authors are simply trying to tell a story that is too big to put in an article. Sometimes the theoretical argument is so complicated that it’s impossible for readers to follow the story line, or match pieces of evidence to parts of the story. Sometimes, the theoretical argument seems to rest on so many different elements being in just the right place that the argument itself seems contrived or trivial. Regardless of the reason the story needs to be simplified. Sometimes that means carving away the obvious parts of a theoretical argument with prior literature or pretests in order to just speak to the aspects of the theoretical argument where the research intends to contribute.
- The theoretical argument is too narrowly defined around the evidence. This happens pretty often in experimental research I’ve reviewed, but can also happen in grounded theory (or theory-in-use) papers. For experiments this takes the form of hypotheses defined around variables instead of constructs, where the relationship between variables and meaningful constructs is unclear. A great recent paper to read on this subject and why it’s so important to keep the theoretical argument and evidence separate is a 2021 JCP by Calder et al. This is also a great paper for a PhD seminar. For qualitative papers, this often takes the form of themes pulled from evidence, but then not pulled up from there to be built into theoretical arguments.
Generating this list of common problems, I’m inclined to just keep going, because it’s definitely not a complete list. Nevertheless, I hope this partial list resonates with your own experience reviewing and makes salient some of the common problems we all confront as we attempt to attain the important research milestone of sparring with our research question enough times that we feel comfortable saying, this research question is ready—send it into the ring.
The journey to publication is a long and arduous one as many of you know from experience. I’ve only talked about what I see as two crucial milestones along that path—a burning question and a research question that is the result of a lengthy process of intellectual work to articulate the theoretical contribution and analytic strategy. Nevertheless, whenever I get stuck, it helps me to go back to the research question—does it need to be tightened, can my evidence support it, is it still interesting, have I covered the literature and carefully defined the constructs that are implicit in that research question, can it be understood or is it too complicated and convoluted. Then I can redraw my models, revisit my evidence, rewrite my contribution statement and climb back into the ring.
Thanks so much for listening and thanks again for the opportunity to be here!
References
Alford, Robert R. (1998), The craft of inquiry: Methods, theories and evidence. Oxford, UK: Oxford University Press.
Calder, Bobby J., C. Miguel Brendl, Alice M. Tybout, and Brian Sternthal (2021), “Distinguishing Constructs From Variables In Designing Research.” Journal of Consumer Psychology 31 (1), 188-208.
Davis, Murray S. “That’s interesting! Towards a phenomenology of sociology and a sociology of phenomenology.” Philosophy of the social sciences 1, no. 2 (1971): 309-344.
Epp, Amber M. and Cele C. Otnes (2021), “High-Quality qualitative Research: Getting Into Gear,” Journal of Service Research, 24 (2), 163-67.
Epp, Amber M., Hope Jensen Schau, and Linda L. Price (2014), “The Role of Brands and Mediating Technologies in Assembling Long-Distance Family Practices,” Journal of Marketing, 78 (3), 81-101.
Godfrey, D. Matthew, Linda L. Price, and Robert F. Lusch. “Repair, Consumption, and Sustainability: Fixing Fragile Objects and Maintaining Consumer Practices.” Journal of Consumer Research (2021).
Goor, Dafna, Nailya Ordabayeva, Anat Keinan, Sandrine Crener, The Impostor Syndrome from Luxury Consumption, Journal of Consumer Research, Volume 46, Issue 6, April 2020, Pages 1031–1051, https://doi.org/10.1093/jcr/ucz044.
Pechmann, Cornelia, and Susan J. Knight (2002), “An Experimental Investigation of the Joint Effects of Advertising and Peers on Adolescents’ Beliefs and Intentions about Cigarette Consumption.” Journal of Consumer Research 29 (1), 5-19.
Preece, Chloe, Finola Kerrigan, and Daragh O’reilly. “License to assemble: Theorizing brand longevity.” Journal of Consumer Research 46, no. 2 (2019): 330-350.
Price, Linda L. (2020), “ACR Fellows Address: The Fellowship and The Quest,” Advances in Consumer Research.
Price, Linda L. (2015), “Presidential Address: Obliquity, Wonderment and The Grand Adventure of Doing Consumer Research,” Advances in Consumer Research.
Thomas, Tandy Chalmers, Linda L. Price, and Hope Jensen Schau (2013), “When Differences Unite: Resource Dependence in Heterogeneous Consumption Communities,” Journal of Consumer Research, 39 (5), 1010-33.
Van Laer, Tom, Jennifer Edson Escalas, Stephan Ludwig, and Ellis A. Van Den Hende. “What happens in Vegas stays on TripAdvisor? A theory and technique to understand narrativity in consumer reviews.” Journal of Consumer Research 46, no. 2 (2019): 267-285.
Zhao, Guangzhi, and Cornelia Pechmann (2007), “The Impact of Regulatory Focus on Adolescents’ Response to Antismoking Advertising Campaigns.” Journal of Marketing Research, 44, (4) 671-687.