As concerns over travel and inter-personal contact intensify, interest in online and mobile qualitative research continues to grow. But at what cost? What do you gain and what do you lose when you move from face-to-face to online qualitative research?
I’m still surprised by the dichotomy that splits many qualitative researchers; there are those who believe in the power of online and those who stubbornly reject it. Nothing is black and white. If you are an outright rejecter, you need to confront your biases and consider how your excellence in offline qual will make you excellent at online qual too (as long as it’s designed well and proper time is invested in the additional analysis required).
I’m a practitioner (and trainer) of both face-to-face and online / mobile qual (that is ‘asynchronous’ rather than ‘real-time’ online qual, as explained here). So I come to this with a balanced view (and if you disagree please feel free to share your views in the comments!).
Even before my days as Director of Innovation at Hall and Partners over a decade ago, I’ve been a big advocate of the power of digital qualitative research. Yes, there’s a trade-off. But overall I think the benefits of online, actually out-weigh the losses from not doing face-to-face research. I don’t just see online as a ‘good-enough’ substitute. I see it as a ‘superior’ method to offline (on balance). That’s why my consultancy Feeling Mutual, specialises in online and mobile qual.
Here’s the main gains and losses as I see them (NB. I’m focusing on face-to-face focus group discussions (not-depths) vs asynchronous text and mobile online qual here (not real-time text chat or video-calls). Please feel free to add anything I’ve missed!
The top 4 losses when switching from off, to online qual
1) You can’t feel the ethereal in online qual
The best researchers, pick up on the energy in the room, the unspoken emotions and other insightful, non-verbal forms of communication, that give them an instinctive sense of what people really think and feel.
Online, you get less access to non-verbal cues. This reduces the power of your gut to guide you. With less non-verbal information, the researcher encodes feedback on a less deep level (making recall and interpretation harder).
2) Clients are less engaged with online qual
A huge reason for doing research is to inspire clients to empathise with consumers. When clients turn up for group discussions (even if a fair few emails are being sent during them!) they are listening, watching and learning who their customer is. And the conversations (and agency-client / client-client relationships) that develop in the backroom, can be central to future decisions.
Online, you get less client engagement during fieldwork, as it’s all too easy to not-log in and view what is being said online.
3) The moderator cedes some control in online qual
The moderator has less control over participants. They are less able to use charm (or authority) to gee them up and keep them focused. It’s harder to stop individual participants from derailing the conversation or dominating it.
Online, the moderator’s personality has less power, as they speak in written words rather than physical presence or body language.
4) Participants interact less online
There is a social energy in-person, that inspires participants to bounce ideas off each other. The desire to agree or disagree (to like and be liked) creates a creative energy which can be harnessed to develop ideas and explore them from multiple angles.
Online, people are less naturally motivated to build on what others say. And so there’s less opportunity to explore social dynamics between people.
But are these losses from doing online instead of offline, more significant than the gains? I don’t think so.
I accept the above trade-offs. But online qual is different and its uniqueness creates big benefits. Moreover, some of the weaknesses of online listed above simply don’t matter. Here’s why…
The top 10 gains when switching from off, to online qual
1) Online creates more opportunity to go into detail, scrutinize in-depth and interact
The art of conversation is not just in the first question you ask, it’s in the ones you follow up with to truly understand them. In face-to-face groups, time is your enemy. Bear with me on the maths here. You typically talk to 8 people, for 2hrs; so even without factoring in a late start and the time it takes moderators to talk, you actually only get 1/8 of the 120mins each person gives you (which is 15mins per person on average). You can’t keep probing the same person to go in deeper and deeper. You have to move on to the next question and involve others.
Online lets you go into more depth, with more levels of interaction, creating more chances to scrutinize what people say. This means you can really get under the skin of what people mean. Not just a bit. But a lot. Asynchronous online qual is more like a series of depths because it’s not in real time. So each person gives you a full 2hrs of their time feeding back, because they share their feedback in parallel (and don’t have to wait for the other to stop talking). So that’s more than 8 times more feedback for the same incentive than in face to face.
So while you might lose ‘ethereal’ information from being online, you gain a lot of depth. You have instant access to everything everyone said as a transcript, which means you capture more detail (without having to go through transcripts or tapes days later).
2) Online elicits feedback on their terms, in their natural contexts
Be honest. If I said: ‘let’s have a chat. And by the way, others are listening-in behind that mirror and via the camera’. Do you really think that would not influence your subsequent behaviour? Viewing facilities are great for clients (because they can view) but not so good for getting feedback in a comfortable, natural environment.
Moreover, behavioural science tells us context is everything. If you want to understand about shopping habits, why not get them to feedback while they’re shopping? They will tell you more ‘in the moment’ than you could possibly share from their (often false) memories of the experience.
Online and mobile qual gets you closer to their real lives, makes them feel more comfortable and therefore more likely to express their true feelings openly.
3) Online gives you more vivid outputs that tell a more compelling story
If pictures are worth a thousand words, a mobile video must be millions. While you might engage a stakeholder less in online research (because they don’t follow it so closely), you can inspire more stakeholders by creating vivid multi-media outputs.
4) Online encourages more emotional disclosures
People posture. They care what others think. If others are in the room when you ask them something personal or emotionally sensitive, they are more likely to posture and factor in what other people will think. Online, people feel less judged, less aware of others’ views and so more likely to disclose personal, emotional insights.
5) Online feedback is more pure, and less prone to group and researcher effects
We all know that a dominant (or particularly intelligent, attractive or charming) participant can profoundly influence what others say. Private responses in face to face groups (and good moderation) can help mitigate this. But they do not remove the risk.
Most decent online qualitative platforms let you ask questions in a private, one-on-one mode, so there is less influence from one person over another. Reducing the influence of others, and helping participants share their own individual views is an important part of moderation. Online facilitates this to perfection.
While you might get less interaction between people. It is still easy to achieve. You can open up conversations and set tasks which explicitly encourage people to interact. And they are interacting based on what people say (not on how dominant, attractive or charming they are). I think online qual is more pure.
So although the moderator has less control online - they don’t need as much, because there’s less group dynamics to manage. And if someone fails to engage you can easily drop a new participant in.
6) Online methods lend themselves to more agile collaboration
As feedback comes through each day, the moderator can pick out the best insights and quotes and the agency or client teams can comment on that analysis. It creates a far more collaborative way of working. The researcher learns from the client and agency view and vice-versa.
Of course the researcher has to stay independent (as any decent consultant can and will). I believe our interpretations are stronger when we take on-board (and if required reject) the observations of the client and agency.
7) Online reduces the logistical, financial (and health) costs associated with face-to-face
You don’t have to travel to the market or city (or eat from a shared plate of biscuits!). This helps save money, the environment and potentially the health of all involved in this current climate.
8) Online gives moderators more time to develop their thinking
Moderating face-to-face groups is intense and complex. You have to keep an eye on time, think about what the client and agency who are viewing are thinking, manage the group dynamics, manage the stimulus and manage your own fatigue after hours of moderation. Face-to-face does not lend itself to developing clever, insightful hypotheses about what you are hearing, while you are hearing it.
Asynchronous online qual is calm and does. They answer each day at their convenience. You moderate each day at your convenience. Online gives you more time to breathe, to think about what they’re saying, to probe in depth and to develop and test your hypotheses and story as you go.
How many brilliant thoughts have you had when moderating a face to face group which you then forget because you’re worried about stopping Gavin from talking about Brexit again?
9) Online makes moderation less about how it seems and more about what they say
Face to face moderation skills takes years to develop (not least because of the challenges listed above). Face to face moderation is in many ways a performance (to ensure the client and agency feel you have given the ideas being tested and the questions being asked, a fair hearing). Moderators are often judged on subtle dimensions: their energy, the way they handle the dynamics and so on. You often hear moderators being blamed for the poor reception of the ideas being tested.
Online, this issue is largely resolved. Online is less about how able you are to manage group dynamics, watch the clock or move through a large volume of questions. It’s more about what you probe and when, and how you design and interpret the study.
10) Online gives you your evenings back
Oh, and you get your life back! You can be home for bath-time or kick-off, if it’s happening.
Irrespective of what happens with the Coronavirus, the qualities of online will only get better, as technology evolves. It’s time to stop dismissing online as a poor substitute for face-to-face, and time to start recognising its unique value.
Who we are
Feeling Mutual is a network of specialist, senior, online qualitative and mobile researchers around the world. This means we work in every market, across every category and on every type of qualitative brief. We bring in the best experts and use the right technologies for the job at hand.
If you are a client and need a proposal for online and mobile qualitative research, an agency that wants support in designing or running projects, training for your research teams or even an independent consultant with online expertise that wants to work with us - please do get in touch with Tom@feelingmutual.com.
And if you think I’ve missed something in my analysis above, I’d genuinely love to hear it. It’s an important time for the industry to have this discussion. So please do share a comment.
(Just to clarify, I’m not saying face-to-face qual is not hugely valuable. I believe in it. I just think online and mobile has been ignored for too long. And it’s time to recognise their strengths, like they deserve).