Overview

As part of the financial services domain, we aim at simplifying our customers' lives when managing their finances. As such, in order to design the right thing(s), it is imperative that we develop a deep fundamental knowledge regarding the pains, needs, and the goals customers have, such as when managing their expenses, financing their businesses, and in other comparable processes.

In Q2 2022, Mollie was motivated to learn more about some of these matters, particularly our customer's experience with managing their expenses and overall payment instrument usage (particularly credit cards).

This page summarises my responsibilities in this project. Due to the recency of this initiative and the sensitive nature of the insights gathered, I'll only focus on the process that I followed and the key impacts of my work.

Team

•  1 Senior User Researcher (yours truly)
•  1 Marketing Researcher
• 1 Product Manager

My Contributions

•  Lead User Research, across all stages, such as
- Planning
- Running
- Analysing & Sharing

Timeline

•  Q2 2022

The Process

For this research project, I followed a 6-stage process: Kickoff, Planning, Preparing, Running, Analysing, and Sharing. In the next sections, I'll provide a summary of each stage. Please note that, despite their apparent sequence, some of these stages are iterative in nature and/or can be run in parallel.

Kickoff

During the kickoff stage (i.e. usually a meeting), we identified the following:

• Stakeholders - who must be involved and/or notified?
• Responsibilities (related to the above) - who does what?
• What do we want to know? - e.g. Are we gathering foundational knowledge? or evaluating existing features?
• Why does that matter?
• What will we do with the information?
• Deadlines - how much time do we have?
• Target audience - who do we want to gather information from?
• Hypothesis / Assumptions - what do we think we know?

Planning

Based on the information gathered during the Kickoff phase, I proceeded to create a Research Plan to address our needs. This plan includes:

• Research Questions
• Methodology (including recruitment strategies)
• Timeline
• Potential Future Research

Once approved by all relevant stakeholders, we moved to the next stages.

Research Questions

For this project, our research questions addressed 3 main topics:

Costs

• What type of costs do customers usually have? How do they pay for them? What pain points do they have (if any)? (...)

(Credit) Cards

• Do customers use c.cards to pay for their expenses? What do they use them for? What motivates them to acquire them? (...)

Mollie's role

• Is Mollie part of our customers' financial managing processes? What type of features are relevant? Do they work as expected? (...)

Note: the above questions are NOT the exact research questions, but rather approximations ...

Methodology

As this project aimed at gathering foundational knowledge about the aforementioned topics, it was clear that this was going to be an Exploratory Research project. To answer our research questions, I proposed a methodology based on two stages:

Stage 1: Understanding the problem space

By running in-depth user interviews with customers, we aimed at inquiring about their experiences, challenges, interests, and expectations regarding the aforementioned topics.

Stage 2: Quantifying insights

By running a survey, we aimed at quantifying the insights gathered in Stage 1, and understand how these are distributed across our customer base.

Preparing

During this stage, we created all the resources needed to run the accepted research methodology. For this project, these included:

Stage 1: Session Structure (aka the interview script)

• How to address the participants?
• What questions to ask them? - including what type of follow-up questions to ask, or what questions to skip, depending on a variety of reasons;
• How much time to spend per question (or topic)?
(...)

Communication Resources

This includes the default email communications to be sent out to participants, as well as all meeting links necessary (more details in the Running section). For the emails, the following use cases were addressed:

• Invitation for user interviews with a voucher incentive (and reminder)
• No Show - i.e. if the participant did not show up to the interview
• Invitation for survey participation (with a voucher draw incentive)
• Thanking the participant for their collaboration, in either stage of the project (and awarding them the promised incentive)

Stage 2: Survey (Questions & Logic)

Naturally, this could only be achieved after the end of the first stage.

Running

As the name suggests, this phase involves the running of the user interviews sessions and, later, the publishing of the survey.

Stage 1: User Interviews

A total of 10 customers agreed to talk to us. Participants were sampled from Mollie's research panel (the Insights Community). All participants were from different types of businesses (and other different internal categories) and were all located in the Netherlands (our target audience for this project).

All interviews were run online, at a time selected by the participants. Each session took, approximately, 45 minutes. Besides the participant, at a minimum, one moderator (yours truly) was in the session; when possible, other stakeholders were able to join as observers and/or notetakers. As a token of our appreciation, all participants were awarded a discount voucher.

Stage 2: Survey

Nearly 700 customers agreed to participate in our survey - nearly double the size required to ensure a representative sampling with a confidence interval and margin of error according to standard best practices (95% and 5% respectively).

The survey took approximately 5-10 minutes to complete. As an incentive, by completing the survey, participants joined a voucher draw.

Tools

To run these research stages, we primarily used the following tools:

• Calendly - for scheduling interviews with customers;
• Userzoom Go - to run and record the interviews, take notes, and transcriptions of the sessions;
• Momentive (Survey Monkey) - to design the quantitative survey and collect responses;

Analysing

During this stage, we created all the resources needed to run the accepted research methodology. For this project, these included:

Stage 1: User Interviews

This stage was run almost in parallel to the Running stage - after all, analysis begins at the end of the first session.

With permission from participants, every session was recorded. Combining the notes taken per session and the transcriptions from the same, I analysed the data through a combination of Thematic Analysis and Affinity Diagraming to gather insights regarding the most common patterns in participants' responses. At first, this was used to help design the survey in Stage 2; After that, the insights were compared to the results of the survey, so to understand how representative they are within our customer base.

Stage 2: Survey

The analysis of the survey followed two main approaches. The primary one, a Statistical Analysis of the results - ranging from standard calculations around the most common options selected, to more complex correlation and statistical significance tests, comparing different variables, such as the level of agreement with specific sentences and the respondents' business type/vertical, among others.

As our survey had some open-ended questions, I had also a considerable amount of qualitative data to use. By using a lite-thematic analysis of those responses, I was able to identify some of the most common patterns and, given the benefit of the sample size, was then able to apply similar statistical tests to them, to evaluate their frequency and compare them with other variables.

Tools

The primary tools used to analyse all information were:

• EnjoyHQ (later replaced by Dovetail) - for the qualitative analysis of the interviews;
• Google Sheets and RStudio - for general calculations, visualisation, and statistical evaluation of the survey results;

Sharing

It is important to mention that, despite this stage being listed last, I make sure to inform stakeholders of the results throughout the entire process. On one hand, by encouraging them to participate in the user interviews, as note-takers or observers. Should that not be possible, by using debrief meetings to discuss the events of the sessions. On the other hand, by sharing with all stakeholders detailed summaries of each interview and, later on, the survey responses. While these do not replace the role of a full report, they allow stakeholders to be on-the-loop and, if necessary, have something to share with the leadership teams.

In any case, the results of this project were shared via a final report, published in our research repository (Dovetail), composed of a full summary of the results as well as dedicated insights (in short stories) for a faster and more precise review. Accompanying this, stakeholder presentations were held to socialize the results and encourage the reading of the report.

Outcomes

The results of this research allowed us to validate some of our hypotheses, but more importantly, they allowed us to identify a significant mismatch between the reality of our customers and our initial assumptions.

By doing this, a clearer understanding of our customers' problems was found, which ultimately led to a significant shift in our product strategy (for the following/upcoming year). However, this also gave us the first glimpses of what type of solution Mollie could provide.

Thank you

Recommended Next:

OR
See all related projects below: