Product Lanuch Research
Exploring Mixed-Methods Research to Minimizing Risk and Guiding Product Migration
6 weeks | 
Diary Survey, User Interview | 
Sample Size: 5 | 
Qualtrics, Miro, Excel
The in-store product team plans to migrate a legacy merchant and customer-facing product (touchscreen device) to an updated web codebase.
This update will unlock new features that can generate more business opportunities.
Challenge
The migration will significantly alter current features and user flows. The team is concerned that these changes could result in dissatisfied users and potential churn.
How might we execute a seamless migration with minimal disruption and deliver an improved user experience that caters to the users’ needs?
Solution
We outlined three research projects to achieve this goal. This page focuses on the 3rd project, we leverage dairy survey and user interview to collect user feedback on the overall user flow and key feature changes to address issues or friction before the general migration.
Methodology
Based on our previous research on feature priorities and usability testing, our team has created a minimum viable product(MVP) to gather real-time user feedback. This feedback will be used to inform the overal strategy for the broader product migration.
Objective
-
Evaluate the overall user experience with the upgraded user flow compared to the legacy flow.
-
Identify and address red flags before shipping the product.
-
Provide insights for future improvement opportunities and broader product migration strategy.
Methodology
-
Diary survey
-
Product feedback user interview
Diary Survey
Over a three-week period, testing participants were given a 2-minutes short survey every day to track and evaluate their satisfaction, perceived speed, and ease of use.
Monitoring Daily Response
I set up a Qualtrics report-sharing link to track real-time user feedback. This allows stakeholders, including the product manager to evaluate and monitor UX metrics on a daily basis. It enables them to quickly identify issues and bugs in real-time rather than waiting until after the testing period
User Interview
After three weeks of using our alpha release product, I interviewed our testing participants to gather feedback on their overall experience and uncover areas for improvement.
Click 
to view examples of interview questions
Overall Feedback
How would you describe your overall experience with the upgrading system? How does the new system compare to the previous one you were using?
Ease of Use
Did you experience any confusion while using the new system?
Where in the flow does this typically happen?
How did you handle the issue?
User Flow Changes
How do you feel about allowing customers to choose their own rewards?
How did your customers respond?
Was there any confusion or difficulty in understanding the new changes?
Perceived Speed
Does this new change impact your checkout and transaction speed?
How will it impact your interactions with customers?
Customer Feedback
How have your customers been reacting to the new system?
Are there any questions, confusion, or signs of liking/disliking it?
Interview questions base on the survey
Additionally, I utilized the responses gathered from the diary survey to create tailored interview questions. By taking into account the feedback received in the survey, we were able to delve deeper and uncover underlying factors that influenced user experiences.
This allowed us to identify any issues encountered during earlier testing phases that may have been overlooked during previous interviews conducted weeks later. Furthermore, it served as an opportunity to explore discrepancies or additional insights that may have arisen after since the completion of the diary survey.
Analysis
I introduced collaborative analysis sessions with stakeholders to expedite the analysis process and engage them in discussions.
-
After each user interview, a 15-minute debrief session was scheduled where prompts and guiding questions were provided on a Miro board.
-
During these sessions, I provided prompts and guiding questions on the Miro board to facilitate discussions and help them recall the initial impressions, key insights, and memorable quotes from the interview.
This approach saved the research time on the interview transcript analysis and promoted alignment with stakeholders.
Presentation
The presentation was divided into three main sections for clarity and to provide actionable insights.
Executive Summary
This section provide stakeholders with a concise summary of the research findings without going into excessive detail. It includes an overview of the research background, including the number of participants and methodologies used.
Quantitative Feedback from Survey
In this section, I focused on the quantitative aspects of the research. I began by presenting key UX metrics obtained from the diary survey results, such as satisfaction, ease of use, and perceived speed scores. The survey findings were then analyzed in detail for each metric and participant to identify emerging trends and patterns through visualization using charts.
In addition, I identified and discussed the outliers and unusual data points to help stakeholders gain deeper insights into user behavior.
Qualitative Feedback from Interviews
I highlighted the positive feedback received from participants, emphasizing what they loved about our upgraded user flow. I included direct quotes and stories to make the feedback relatable.
Additionally, I used this section to discuss the dissatisfied feedback received from participants and their suggestions for improvement, providing the reasoning behind their behavior and its impact on the identified metrics.
Recommendation
In the final section, I summarized the key takeaways from the research and offered stakeholders practical recommendations to enhance product development. Emphasizing key areas for improvement, I suggested actionable steps to address identified concerns and optimize user experience.
Impact
The research findings have been instrumental in guiding our decision-making process and helped the product and development team in prioritizing the next steps to improve the MVP.
-
Positive feedback on the upgraded MVP has boosted our confidence in taking action to migrate our product.
-
Identification of critical bugs and issues has allowed the product manager to prioritize and address them in our plan for product improvement.
-
In response to minor UX improvement feedback, our designers are already planning to explore alternative designs in the next iteration of product development, based on the insights gained from the research.
These valuable insights have increased team confidence in the upgraded product, reducing potential friction and disruptions during the migration process.
The research findings have had a significant impact on our product development strategy and migration plan. We are using the insights to improve the user experience, ensure a seamless transition for existing users, and enhance overall product quality.
Reflection
This research project, along with previous feature priority discovery research and prototype usability research, represents the culmination of our efforts in the product migration project.
To enhance stakeholder engagement, I introduced debrief sessions 📝 after interviews. This practice expedited ⏩ the analysis process and reduced time constraints.
Furthermore, to facilitate real-time tracking 🕒 of user feedback 🗣️, I implemented a Qualtrics-based report-sharing link 🔗. This empowered stakeholders to promptly identify and address issues or bugs.
Drawing from past successes, this third research project 📚 has equipped both stakeholders and myself with well-established processes and methodologies that will better prepare us for future collaborations 🤝.