When I first started working in user experience, I was eager to quantify every insight I gathered. It seemed logical: if something is important, it should be measurable. But over time, I realized that not every UX insight fits neatly into a metric. Some insights lose their meaning or even mislead when forced into numbers. This post shares why some UX insights should remain qualitative and how turning them into metrics can sometimes do more harm than good. Why Some UX Insights Should Never Become Metrics The Temptation to Quantify Everything In product teams, there’s often pressure to prove the value of UX work through numbers. Metrics like task completion rates, time on task, or Net Promoter Score are common. These numbers help stakeholders understand progress and justify investments. But this focus on metrics can push teams to convert every insight into a number, even when it doesn’t fit. For example, during a usability test, a participant might express frustration about a feature feeling “confusing” or “unfriendly.” While you can count how many users said this, the real value lies in understanding why they feel that way. Reducing this rich feedback to a percentage misses the nuance and can lead to superficial fixes. When Metrics Oversimplify Complex User Feelings User experience is about human emotions, motivations, and behaviors. These are complex and often contradictory. Metrics tend to simplify this complexity into a single number, which can hide important details. Take the example of a mobile app’s onboarding process. A metric might show that 80% of users complete onboarding. That sounds good, but it doesn’t reveal if users felt overwhelmed or rushed. Maybe they completed it because they had no choice, but their frustration could lead to churn later. Without qualitative insights, this metric alone gives a false sense of success. Examples of UX Insights That Should Stay Qualitative Emotional reactions Users’ feelings like frustration, delight, or confusion are hard to measure accurately. Instead, capturing stories or quotes provides richer context. Contextual behaviors How users interact with a product depends on their environment, mood, or goals. These factors are difficult to quantify but critical to understand. Unspoken needs Sometimes users don’t articulate their needs directly. Observing body language or hesitation during testing reveals insights that numbers can’t capture. Ideas for innovation Creative suggestions or “wish lists” from users often don’t translate into metrics but inspire new directions. How Forcing Metrics Can Lead to Wrong Decisions I once worked on a project where the team focused heavily on reducing the average time users spent on a checkout page. The metric suggested faster checkout was better. But qualitative feedback revealed users wanted more reassurance and information before buying. By rushing them, the team actually increased anxiety and abandoned carts. This shows how relying solely on metrics can lead to decisions that harm the user experience. Metrics should support, not replace, qualitative insights. UX researcher observing user interacting with prototype on tablet Balancing Metrics and Qualitative Insights The best UX research combines both numbers and stories. Here are some tips to keep insights meaningful: Use metrics to track clear, objective outcomes like error rates or task success. Reserve qualitative methods like interviews and observations to explore user feelings and motivations. Share user quotes and stories alongside metrics to provide context. Avoid creating metrics for insights that are inherently subjective or complex. Regularly revisit metrics to ensure they still reflect real user needs. Final Thoughts Not every UX insight should become a metric. Some insights lose their power when reduced to numbers. By respecting the qualitative nature of many user experiences, we can design products that truly meet users’ needs. Metrics are useful tools, but they work best when paired with rich, human stories. If you’re a UX professional, try to balance your data with empathy. Focus on understanding users deeply, not just counting their actions. This approach leads to better products and happier users.

When I first started working on user experience (UX) design, I quickly realized that delivering a seamless experience across multiple channels and platforms is one of the toughest challenges. Users expect to switch from a mobile app to a website or even a physical kiosk without feeling lost or confused. But how do you measure if your UX is truly consistent? Over time, I developed a practical approach to evaluate UX consistency that I want to share with you. Measuring UX Consistency Across Channels and Platforms Why UX Consistency Matters Consistency in UX means users can predict how things work no matter where they interact with your brand. This predictability builds trust and reduces frustration. For example, if a checkout button looks and behaves differently on your website compared to your mobile app, users might hesitate or abandon the process. Inconsistent UX can lead to: Confusion and errors Increased support requests Lower customer satisfaction Reduced brand loyalty Measuring UX consistency helps identify gaps and prioritize improvements that make the experience feel unified. Defining What to Measure Before jumping into metrics, I focus on defining what consistency means for the product. This involves: Visual consistency : Are colors, fonts, icons, and layouts similar across platforms? Functional consistency : Do features work the same way everywhere? Content consistency : Is the language, tone, and information aligned? Interaction consistency : Are gestures, clicks, and navigation patterns familiar? For example, a banking app should use the same terminology for account types on both mobile and web. Buttons for transferring money should be in similar locations and behave the same way. Methods to Measure UX Consistency 1. Heuristic Evaluation Across Platforms I start by conducting heuristic evaluations on each platform using a checklist focused on consistency. This includes: Checking if UI elements follow the same design system Verifying that workflows match Ensuring error messages and help content are consistent This method quickly highlights obvious mismatches. 2. User Testing with Cross-Platform Tasks Next, I run user tests where participants complete the same tasks on different platforms. I observe: Time taken to complete tasks Errors made User comments on differences or confusion For example, I asked users to update their profile on both the mobile app and desktop site. When users struggled more on one platform, it indicated inconsistency. 3. Analytics and Behavior Tracking Analyzing user behavior data can reveal inconsistencies indirectly. Metrics to watch include: Drop-off rates at similar steps on different platforms Frequency of feature use Navigation paths If users frequently abandon a process on one platform but not another, it suggests a UX gap. 4. Visual and Interaction Audits Using design tools, I compare screenshots and interaction flows side by side. This helps spot: Differences in button sizes or colors Variations in spacing and alignment Inconsistent iconography Tools like Zeplin or Figma make this easier by showing design specs clearly. Real-World Example: Retail App and Website In one project, I worked with a retail brand that had a mobile app and a website. Customers complained about confusion when switching between the two. We measured UX consistency by: Mapping user journeys on both platforms Running user tests with 20 participants performing shopping tasks Conducting a visual audit of UI elements We found the mobile app used a different checkout flow and terminology. After aligning these elements, the checkout abandonment rate dropped by 15% on mobile. Tablet showing consistent shopping cart design Tips to Maintain Consistency Over Time Create and maintain a design system that includes UI components, colors, typography, and interaction patterns. Document UX guidelines clearly for all teams involved in product development. Regularly test new features on all platforms before release. Use shared tools for design and development to reduce discrepancies. Gather user feedback continuously to catch emerging inconsistencies. Final Thoughts Measuring UX consistency is not a one-time task but an ongoing process. By focusing on visual, functional, content, and interaction consistency, you can create a smoother experience that keeps users engaged and satisfied. Start by defining what consistency means for your product, then use a mix of evaluations, user testing, and data analysis to find gaps. Finally, build strong processes and tools to keep your UX aligned as your product grows.

When I first joined a product team, I quickly realized we lacked a clear way to measure how users experienced our product. We relied on gut feelings and anecdotal feedback, which made it hard to know what truly worked or where to improve. Building a UX measurement framework changed everything. It gave us clear data to guide decisions and improve user satisfaction. If you want to create a similar system in your organization, here’s how to start. UX Measurement Framework: Understand What You Want to Measure Before collecting any data, you need to define what aspects of user experience matter most to your organization. UX is broad, covering usability, satisfaction, engagement, and more. Focus on specific goals that align with your product and business needs. For example, if you run an e-commerce site, measuring how easily users find products and complete purchases is crucial. For a SaaS tool, tracking task success rates and user satisfaction with features might be more relevant. Ask yourself: What user behaviors or feelings indicate success? Which parts of the experience cause frustration or drop-off? What business outcomes depend on good UX? Clear goals help you choose the right metrics and avoid collecting irrelevant data. Choose the Right Metrics Once you know what to measure, pick metrics that provide meaningful insights. UX metrics generally fall into three categories: Behavioral metrics : Show what users do, such as task completion rate, error rate, or time on task. Attitudinal metrics : Reflect how users feel, like satisfaction scores or Net Promoter Score (NPS). Qualitative feedback : Includes user comments, interviews, or usability test observations. For example, to measure ease of use, you might track task success rate and time on task. To understand satisfaction, use surveys with questions about overall experience or specific features. Avoid using too many metrics at once. Focus on a few key indicators that directly relate to your goals. This keeps analysis manageable and actionable. How to Build a UX Measurement Framework for Your Organization Collect Data Consistently A measurement framework only works if you gather data regularly and consistently. Set up processes to collect metrics at key points in the user journey. This might include: Usability testing sessions during product development In-app surveys after completing tasks Analytics tracking user behavior on your site or app Use tools that fit your needs and budget. Google Analytics, Hotjar, or Mixpanel can track behavioral data. Survey tools like Typeform or Qualtrics help gather attitudinal feedback. Make sure your team understands when and how to collect data. Consistency ensures you can compare results over time and spot trends. Analyze and Share Insights Collecting data is only useful if you analyze it and share findings with your team. Look for patterns that reveal strengths and pain points in the user experience. For example, if task completion rates drop after a new feature launch, investigate what’s causing confusion. If satisfaction scores improve after a redesign, highlight what worked well. Use visuals like charts or dashboards to make data easy to understand. Regularly share reports with product managers, designers, and developers to keep everyone aligned. Iterate Based on Feedback A UX measurement framework is not a one-time project. Use the insights you gather to make improvements, then measure again to see the impact. For instance, after identifying a confusing checkout process, you might simplify the steps and test again. Tracking changes over time helps you learn what drives better user experiences. Encourage a culture where data guides decisions but also leaves room for creativity and experimentation. Whiteboard showing UX measurement framework and user journey mapping Start Small and Scale Up When I built my first UX measurement framework, I started with just two metrics: task success rate and user satisfaction. This simple approach gave us valuable insights without overwhelming the team. As you gain experience, expand your framework to include more metrics or deeper analysis. Tailor it to your organization’s size, product complexity, and resources. Remember, the goal is to create a system that helps you understand and improve user experience, not to collect data for data’s sake.

Using research effectively can transform how designers and product managers (PMs) make decisions. Yet, many teams struggle to integrate research into their daily workflows. I’ve seen firsthand how coaching can bridge this gap, helping teams not only understand research but also apply it in ways that improve products and user experiences. In this post, I’ll share practical strategies I’ve used to coach designers and PMs to get more value from research. These approaches focus on building habits, fostering collaboration, and making research a natural part of the product process. Designer and PM collaborating on research insights Understanding the Challenges Teams Face Before coaching, I always start by identifying why research isn’t being used effectively. Common challenges include: Research insights feel too abstract or disconnected from daily tasks. Teams receive research late, after key decisions are made. Designers and PMs lack confidence in interpreting research data. Communication gaps between researchers and product teams. For example, I worked with a product team that often ignored user feedback because they found reports too long and full of jargon. The designers felt overwhelmed, and PMs didn’t see how research could help prioritize features. Recognizing these pain points helps tailor coaching to the team’s specific needs. Building Research Habits Through Coaching One of the most effective ways to improve research use is to build simple, repeatable habits. I encourage teams to: Start meetings with a research insight: Begin sprint planning or design reviews by sharing a relevant user finding. This keeps research top of mind. Create quick research summaries: Teach designers and PMs to write brief, clear summaries of research reports. This makes insights easier to digest. Ask research-driven questions: Encourage the team to frame problems and ideas based on user needs and data, not assumptions. In one coaching session, I introduced a “research moment” at the start of every team meeting. Over a few weeks, this habit helped the team shift their mindset. They began to ask, “What does the research say about this?” before making decisions. Encouraging Collaboration Between Designers, PMs, and Researchers Research becomes more useful when teams work closely with researchers. I coach teams to: Involve researchers early in the product cycle. Invite researchers to design critiques and planning sessions. Use collaborative tools to share research findings in real time. For example, a PM I coached started inviting the user researcher to weekly check-ins. This allowed the PM to clarify findings and get advice on how to apply insights. Designers also began requesting quick user feedback during prototyping, speeding up iteration. This collaboration breaks down silos and makes research a shared responsibility. User researcher sharing insights with product team Making Research Actionable and Relevant Research can feel overwhelming if it’s not directly tied to decisions. I help teams translate findings into clear actions by: Mapping insights to specific product goals or features. Prioritizing research based on impact and feasibility. Creating visual tools like journey maps or personas that designers and PMs can reference easily. For instance, I worked with a team that struggled to prioritize features. We created a simple matrix linking user pain points from research to potential features. This visual helped the team focus on what mattered most to users and justify decisions with data. Coaching Through Real Projects The best coaching happens in the flow of work. I join teams during real projects to: Observe how they use research. Provide immediate feedback. Model how to ask research questions and interpret data. This hands-on approach builds confidence and shows the practical value of research. One designer I coached said, “Seeing how to connect research to my sketches made me rethink my whole process.” Tips for Sustaining Research Use Over Time To keep research use strong, I recommend: Regularly revisiting research habits and adjusting them. Celebrating wins where research improved outcomes. Sharing success stories within the team to build enthusiasm. Providing ongoing support and refresher coaching sessions. Research is not a one-time event but a continuous practice. Teams that treat it as part of their culture see better results.

When I first stepped into a UX research leadership role, I quickly realized that guiding a team through complex projects was far more challenging than I expected. Mistakes happen, but some can slow down progress or even damage the credibility of UX research within an organization. Over time, I learned which pitfalls to watch for and how to steer clear of them. If you lead a UX research team or aspire to, this post will help you avoid common errors and build stronger, more effective research practices. The Biggest Mistakes UX Research Leaders Make — And How to Avoid Them Mistake 1: Neglecting Clear Communication With Stakeholders One of the biggest errors I made early on was assuming stakeholders understood the value and process of UX research without clear communication. When research findings are unclear or poorly presented, stakeholders may ignore or misunderstand them, which reduces the impact of your work. How to avoid this: Share research goals and methods upfront to set expectations. Use simple language and visuals to explain findings. Regularly update stakeholders with concise reports or presentations. Invite feedback and questions to ensure alignment. For example, in one project, I created a one-page summary with key insights and recommendations. This helped executives quickly grasp the research value and made it easier to get buy-in for design changes. Mistake 2: Overloading the Team With Too Many Projects In my early leadership days, I often accepted every research request that came my way. This led to burnout and lower quality work because the team was stretched too thin. How to avoid this: Prioritize projects based on business impact and feasibility. Set realistic timelines and resource limits. Push back when necessary and explain why some requests must wait. Encourage your team to focus on depth rather than quantity. A good practice is to hold quarterly planning sessions with product managers to align research priorities with company goals. This keeps the team focused and productive. Mistake 3: Ignoring the Importance of Team Development I once underestimated how crucial it is to invest in my team’s growth. Without ongoing training and mentorship, skills stagnate, and motivation drops. How to avoid this: Schedule regular one-on-one meetings to discuss career goals. Provide access to workshops, conferences, and online courses. Encourage knowledge sharing within the team. Recognize achievements and celebrate progress. For instance, I introduced monthly “research show-and-tell” sessions where team members present recent work and lessons learned. This boosted confidence and fostered collaboration. UX research team collaborating on user journey maps on a whiteboard Mistake 4: Failing to Integrate Research Early in the Product Process Waiting too long to involve UX research can lead to costly redesigns or missed opportunities. Early research helps identify user needs and potential problems before development begins. How to avoid this: Advocate for research involvement during product discovery and planning. Collaborate closely with product managers and designers from the start. Use quick, iterative research methods to inform early decisions. Share findings promptly to influence design direction. In one project, early usability testing uncovered a confusing navigation flow. Addressing this before development saved weeks of rework and improved user satisfaction. Mistake 5: Relying Too Much on Quantitative Data Alone Numbers are valuable, but they don’t tell the whole story. I once focused heavily on survey results and analytics, missing the deeper insights that come from qualitative research like interviews and observations. How to avoid this: Combine quantitative and qualitative methods for a fuller picture. Use interviews, diary studies, or field visits to understand user motivations. Look beyond metrics to explore why users behave a certain way. Share stories and quotes to bring data to life. For example, pairing analytics with user interviews helped my team uncover emotional pain points that numbers alone couldn’t reveal. Mistake 6: Not Advocating for UX Research Within the Organization UX research can be undervalued if leaders don’t actively promote its benefits. Early in my career, I assumed good work would speak for itself, but that’s rarely enough. How to avoid this: Share success stories and case studies with leadership. Educate teams about how research improves products and customer satisfaction. Build relationships with other departments to increase collaboration. Show how research reduces risks and saves money. By regularly presenting research impact, I helped secure more budget and support for my team. Avoiding these mistakes requires ongoing attention and effort, but the payoff is a stronger UX research practice that drives better products and happier users. If you lead a UX research team, take time to reflect on your current approach and identify areas for improvement. Your team and your users will thank you.

When I first started working in UX research, the process often felt slow and repetitive. Gathering user feedback, analyzing data, and generating insights took weeks, sometimes months. I knew there had to be a better way. That’s when I began exploring agentic AI—intelligent systems that can act autonomously to support and improve UX research workflows. Since then, I’ve seen firsthand how this technology can transform the way we work, making research faster, more accurate, and more insightful. Agentic AI i What Agentic AI Means for UX Research Agentic AI refers to artificial intelligence systems designed to perform tasks independently, making decisions and taking actions without constant human input. In UX research, this means AI can: Collect and organize user data automatically Identify patterns and trends in user behavior Generate hypotheses and suggest next steps Automate routine tasks like transcription and coding This level of autonomy frees researchers from manual work and lets them focus on interpreting results and designing better experiences. How Agentic AI Improves Research Speed and Quality One of the biggest challenges in UX research is managing large volumes of data. Traditional methods require hours of manual effort to transcribe interviews, tag responses, and analyze feedback. Agentic AI can handle these tasks in minutes. For example, AI-powered transcription tools convert audio to text instantly, while natural language processing algorithms categorize responses by theme or sentiment. In my experience, using agentic AI tools cut the time spent on data preparation by over 50%. This speed doesn’t come at the cost of quality. AI systems can detect subtle patterns that humans might miss, such as emerging user frustrations or unexpected behavior trends. These insights help teams make informed design decisions faster. Real-World Examples of Agentic AI in UX Research Several companies have started integrating agentic AI into their UX workflows with impressive results: UserTesting uses AI to analyze video feedback, automatically highlighting key moments and emotional reactions. This helps researchers focus on the most relevant parts of user sessions. Lookback.io employs AI to transcribe and tag user interviews, speeding up the analysis phase and improving accuracy. Dovetail integrates AI to summarize qualitative data, making it easier to share findings with stakeholders. These tools show how agentic AI can handle repetitive tasks while enhancing the depth of insights. Researcher reviewing AI-generated UX insights Challenges and Considerations When Using Agentic AI Despite its benefits, agentic AI is not a magic solution. It requires careful implementation and ongoing oversight. Here are some points I learned along the way: Bias in AI models : AI systems learn from existing data, which can contain biases. Researchers must validate AI outputs to avoid skewed conclusions. Transparency : Understanding how AI arrives at its suggestions is crucial. Black-box models can reduce trust in findings. Human judgment remains essential : AI supports but does not replace human intuition and creativity. Researchers still need to interpret results and make final decisions. Data privacy : Handling user data responsibly is critical, especially when AI processes sensitive information. Balancing these factors ensures agentic AI enhances rather than hinders UX research. What the Future Holds for UX Research Workflows Agentic AI will continue to evolve, becoming more capable and integrated into everyday research tools. I expect to see: Smarter AI assistants that proactively suggest research questions or methods based on project goals Real-time analysis during user testing sessions, enabling immediate adjustments Greater collaboration between AI and human researchers, combining speed with empathy and context These advances will make UX research more agile and responsive, helping teams create user experiences that truly meet people’s needs.

By Philip Burgess | UX Research Leader When AI tools deliver insights at lightning speed, it feels like decision-making should become faster too. Yet, I’ve noticed the opposite happening in many cases: quicker AI insights sometimes slow down the actual decisions. This might sound counterintuitive, but the reality is more complex. I want to share what I’ve learned about why faster AI insights don’t always translate into faster decisions and what we can do about it. Faster AI Insights Slower Decisions The Promise of Speedy AI Insights AI has transformed how we gather and analyze data. Instead of waiting days or weeks for reports, AI can now scan mountains of information and deliver insights in seconds. This speed promises to help businesses, researchers, and individuals make quicker, smarter choices. For example, in retail, AI can instantly analyze customer behavior and suggest product adjustments. In healthcare, AI can quickly flag potential diagnoses from medical images. These rapid insights seem like a clear advantage. But speed alone doesn’t guarantee faster decisions. Why Faster Insights Can Slow Down Decisions 1. Overload of Information When AI delivers insights rapidly, it often produces a flood of data points and recommendations. Instead of a few clear takeaways, decision-makers face a complex web of information to sift through. This overload can cause analysis paralysis. I remember working with a team that used an AI tool to monitor social media trends. The tool generated dozens of insights every hour. The team spent more time debating which insights mattered than actually acting on them. 2. Increased Complexity of Insights AI models can uncover subtle patterns and correlations that humans might miss. While this is valuable, it also means insights become more complex and harder to interpret. Decision-makers may need extra time to understand the implications fully. For instance, a financial analyst using AI to predict market trends might get nuanced forecasts with multiple scenarios. Evaluating these scenarios carefully takes time, especially when stakes are high. 3. Need for Human Judgment AI insights are just one part of decision-making. Humans must weigh these insights against experience, values, and context. When AI insights come quickly, people may feel pressured to slow down and double-check before committing to a decision. In my experience, leaders often pause to validate AI findings with other sources or team discussions. This step adds time but reduces risk. 4. Trust and Verification Rapid AI insights can raise doubts about accuracy. If decision-makers don’t fully trust the AI, they will spend more time verifying results. This verification process can delay decisions. A marketing team I worked with initially distrusted their AI tool’s customer segmentation. They ran multiple tests and cross-checked with manual analysis before acting, which slowed their campaign launch. How to Balance Speed and Decision-Making Focus on Actionable Insights Not all AI insights are equally useful. Prioritize those that clearly support a decision or action. Filtering out noise helps reduce overload and speeds up the process. Simplify Presentation Present AI insights in clear, concise formats. Visual summaries, key metrics, and straightforward recommendations help decision-makers grasp information quickly. Build Trust Through Transparency Understanding how AI generates insights builds confidence. When possible, use AI tools that explain their reasoning or provide confidence scores. Combine AI with Human Expertise Use AI as a support tool, not a replacement for judgment. Encourage collaboration between AI outputs and human experience to reach balanced decisions faster. Person reviewing AI-generated charts on a tablet Real-World Example: AI in Healthcare In healthcare, AI can analyze patient data quickly to suggest diagnoses or treatment options. However, doctors often take extra time reviewing AI recommendations. They consider patient history, symptoms, and their own expertise before deciding. This process can slow decisions but improves outcomes. The goal is not just speed but accuracy and safety. Faster AI insights help doctors by providing more information, but the final decision requires careful thought.

By Philip Burgess | UX Research Leader Artificial intelligence research moves fast. Every week, new papers, models, and breakthroughs appear, promising to change how we live and work. But I’ve learned that taking AI research outputs at face value can lead to misunderstandings, misplaced trust, and even setbacks. In this post, I want to share my experience and insights on why it’s crucial to look beyond the headlines and dig deeper into what AI research really means. What happens when AI Research outputs are taken at face value The Allure of AI Breakthroughs When I first started following AI research, I was amazed by the bold claims. Papers would announce new models that “outperform humans” or “solve complex problems.” The excitement is understandable. AI has the potential to transform industries like healthcare, transportation, and education. But I quickly noticed a pattern. Many research outputs are presented with impressive metrics or flashy demos, yet they often come with caveats buried deep in the text. For example, a model might perform well on a specific dataset but fail in real-world scenarios. Or the training data might be limited or biased, affecting the model’s fairness. This gap between research claims and practical reality can cause problems when people take outputs at face value. Why Taking AI Research Outputs at Face Value Is Risky Overestimating Capabilities One common issue is overestimating what AI can do. I remember reading about a language model that scored high on a benchmark test. The headlines suggested it could understand and generate human-like text flawlessly. But in practice, the model sometimes produced nonsensical or biased responses. This happens because benchmarks often simplify complex tasks. They don’t capture the nuances of real-world use. When organizations adopt AI tools based solely on research results, they risk deploying systems that don’t meet expectations. Ignoring Limitations and Biases AI models learn from data, and data reflects human biases. Research papers sometimes acknowledge this, but the warnings don’t always reach the wider audience. I’ve seen cases where AI systems unintentionally reinforce stereotypes or exclude certain groups. Taking outputs at face value means missing these important limitations. It can lead to ethical issues and harm the people the AI is meant to serve. Misunderstanding the Context Research outputs are often tested in controlled environments. For example, a computer vision model might excel at recognizing objects in clean, well-lit images. But in real life, lighting conditions, angles, and backgrounds vary widely. I recall a project where a facial recognition system performed well in the lab but struggled outdoors. The team had to spend months adapting the model to handle real-world conditions. This shows that research results don’t always translate directly to practical applications. Close-up of AI research data visualizations on a computer screen How to Approach AI Research Outputs More Wisely Read Beyond the Abstract The abstract and headlines often highlight the best results. I learned to read the full paper, especially the sections on limitations, data, and methodology. This helps me understand the context and constraints of the research. Look for Independent Evaluations Research from a single team can be promising but may not tell the whole story. I look for independent evaluations or replication studies that test the model under different conditions. This gives a clearer picture of its strengths and weaknesses. Consider the Data Data quality and diversity matter a lot. I pay attention to the datasets used for training and testing. Are they large enough? Do they represent diverse populations or scenarios? This affects how well the AI will perform outside the lab. Test in Realistic Settings Before trusting AI outputs, I recommend testing models in environments that mimic real use cases. This can reveal unexpected issues and help improve the system before deployment. Real-World Example: AI in Medical Imaging A few years ago, I followed a study where researchers developed an AI model to detect cancer in medical images. The paper reported high accuracy, and many hoped it would revolutionize diagnostics. But when hospitals tried to use the model, they found it struggled with images from different machines or patient groups. The initial research had used a limited dataset from one hospital. This taught me that even promising AI research needs thorough validation before it can be trusted in critical fields like healthcare. Final Thoughts AI research outputs offer exciting possibilities, but they are not the final word. Taking them at face value can lead to overconfidence, ethical risks, and practical failures. Instead, we should approach AI research with curiosity and caution, digging deeper into the details and testing models thoroughly.

By Philip Burgess | UX Research Leader When I first started writing UX research case studies, I thought the story was simple: show the problem, explain the research, and present the solution. But over time, I realized many case studies miss a crucial part — demonstrating the real impact of the research. This gap often leaves readers wondering if the research made any difference at all. In this post, I want to share what I’ve learned about why most UX research case studies fail to show impact and how you can avoid these pitfalls. Why Most UX Research Case Studies Fail to Demonstrate Impact The Problem with Impact in UX Case Studies Many UX case studies focus heavily on the process: the methods used, the number of participants, or the tools applied. While these details are important, they often overshadow the results that matter most. Without clear evidence of impact, the case study feels incomplete and less convincing. For example, I once read a case study that described a detailed usability test with 20 participants. The study explained the tasks and the issues found but didn’t show how the findings influenced the product or business. The reader was left asking: Did the product improve? Did user satisfaction increase? Did the changes lead to better business outcomes? These questions remained unanswered. Why Impact Is Often Missing Lack of Clear Goals One common reason case studies fail to show impact is the absence of clear goals from the start. If the research doesn’t begin with measurable objectives, it becomes difficult to prove success later. Goals like increasing task completion rates, reducing errors, or improving user satisfaction provide a benchmark to compare before and after results. Focusing on Outputs Instead of Outcomes UX research outputs include reports, personas, or journey maps. These are valuable tools but only part of the story. Outcomes are the changes that happen because of those outputs — like improved user engagement or higher conversion rates. Many case studies stop at outputs without connecting them to outcomes. Poor Data Collection on Impact Sometimes, researchers don’t collect or track data that shows impact. For instance, if a usability test identifies issues but no follow-up metrics are gathered after changes are made, it’s impossible to prove the research made a difference. Tracking key performance indicators (KPIs) over time is essential. How to Show Impact Effectively Set Clear, Measurable Goals Start every project by defining what success looks like. For example: Increase checkout completion rate by 15% Reduce average task time by 30 seconds Improve user satisfaction score by 10 points These goals give you a target and a way to measure impact. Link Research Findings to Business or User Outcomes When presenting your case study, explain how your research influenced decisions and what changed as a result. For example: “Based on our usability test, we simplified the checkout process, which led to a 20% increase in completed purchases.” “Our interviews revealed confusion around navigation, prompting a redesign that reduced support calls by 25%.” This connection makes the case study more persuasive. Close-up of a redesigned user interface with clearer navigation Use Quantitative and Qualitative Data Combine numbers with stories. Quantitative data like conversion rates or error counts provide hard evidence. Qualitative feedback from users adds context and emotion. Together, they create a fuller picture of impact. Follow Up and Track Changes Over Time Impact doesn’t always show immediately. Plan to revisit your research findings after changes are implemented. Collect data at multiple points to demonstrate sustained improvements or identify areas needing further work. Personal Experience: Turning a Case Study Around I once worked on a project where the initial case study was all about the research process. It described interviews and usability tests but didn’t show what happened next. The team wasn’t sure if the research made a difference. To fix this, I helped set clear goals before the next round of research. We tracked metrics like task success rates and user satisfaction before and after changes. We also gathered user quotes to highlight improvements. When we rewrote the case study, it told a complete story: the problem, the research, the changes made, and the measurable impact. The case study became a powerful tool for the team to advocate for UX research in future projects. Final Thoughts Showing impact in UX research case studies is not just about bragging rights. It’s about proving that research drives real improvements for users and businesses. By setting clear goals, linking findings to outcomes, using solid data, and following up over time, you can create case studies that truly demonstrate value. If you want your UX research to make a difference, start by telling the whole story — from problem to impact. Your readers will thank you, and your work will have a stronger voice.

By Philip Burgess | UX Research Leader When I first started working in the UX field, I often found myself confused by the terms "UX research case study" and "UX design case study." They seemed similar on the surface, but as I gained experience, I realized they serve very different purposes and highlight distinct parts of the product development process. Understanding these differences can help you communicate your work more clearly and showcase your skills effectively. A UX researcher reviewing user feedback notes during a case study What Is a UX Research Case Study? A UX research case study focuses on the process of gathering and analyzing data about users. It tells the story of how you identified user needs, behaviors, and pain points through various research methods. The goal is to provide evidence that supports design decisions or product improvements. Key Elements of UX Research Case Studies Research Goals Clearly state what you wanted to learn. For example, understanding why users abandon a checkout process or how they navigate a mobile app. Methodology Describe the research methods used, such as interviews, surveys, usability testing, or field studies. Explain why you chose these methods. Participant Details Share who the participants were, how many, and how you recruited them. This adds credibility and context. Findings Present the data collected, including quotes, statistics, or observed behaviors. Use visuals like charts or heatmaps to make insights clear. Recommendations Suggest actionable changes based on the research. This might include redesigning a feature, simplifying navigation, or adding new functionality. Example In one project, I conducted usability tests on a fitness app. The research revealed users struggled to find the workout history section. Based on this, I recommended moving the history to a more prominent spot on the home screen. This change improved user satisfaction scores by 20% in follow-up tests. What Is a UX Design Case Study? A UX design case study highlights the creative and problem-solving aspects of designing a product or feature. It shows how you translated research insights into wireframes, prototypes, and final designs. The focus is on your design process, decisions, and the impact of your work. Key Elements of UX Design Case Studies Problem Statement Define the design challenge you aimed to solve. For example, improving the onboarding experience for new users. Research Summary Briefly mention relevant research findings that informed your design choices. Design Process Walk through your steps: sketching, wireframing, prototyping, and testing. Include iterations and explain why you made changes. Visuals Show your design work with images of wireframes, mockups, or interactive prototypes. Outcome Share results such as increased engagement, reduced errors, or positive user feedback. Example For a travel booking website, I redesigned the search results page after research showed users felt overwhelmed by too many options. I simplified the layout, added filters, and improved the visual hierarchy. Post-launch analytics showed a 15% increase in bookings from the search page. A UX designer creating wireframe sketches during a design case study How These Case Studies Complement Each Other While UX research and design case studies focus on different parts of the process, they are deeply connected. Research provides the foundation for design decisions, and design tests the hypotheses generated by research. Together, they tell a complete story of how a product evolves to meet user needs. When to Use Each Case Study Use a UX research case study when you want to highlight your skills in understanding users and generating insights. Use a UX design case study when you want to showcase your creativity, problem-solving, and design execution. If you have experience in both areas, consider combining elements from each to demonstrate your full range of abilities. Tips for Writing Effective Case Studies Be clear and concise Avoid jargon and explain your process in simple terms. Use visuals wisely Images and charts help readers grasp complex information quickly. Tell a story Frame your case study as a journey from problem to solution. Highlight impact Show measurable results whenever possible. Reflect on challenges Share what you learned and how you adapted.

By Philip Burgess | UX Research Leader Starting a ResearchOps function can feel overwhelming. When I first faced this challenge, I didn’t have a clear roadmap. I knew research was crucial for product success, but organizing it efficiently was a puzzle. Over time, I learned that building ResearchOps is about creating a system that supports research activities, making them smoother and more impactful. If you’re ready to build your ResearchOps function from scratch, I’ll share practical steps and lessons from my experience. Organized research workspace with documents and tools Understand What ResearchOps Means for Your Team ResearchOps is the set of practices, tools, and processes that support research teams. It’s not just about managing studies but about creating an environment where research can thrive. Before you start, clarify what your team needs: How many researchers are there? What types of research do you conduct (qualitative, quantitative, mixed)? What tools do you currently use, and what gaps exist? How do stakeholders access and use research findings? Answering these questions helps you tailor your ResearchOps function to your team’s unique needs. Start Small with Clear Priorities When I began, I focused on a few key areas instead of trying to solve everything at once. Here are some priorities that made a big difference early on: Participant recruitment and management: Create a database or system to track participants, their profiles, and availability. This saves time and improves recruitment quality. Research repository: Build a central place where all research reports, notes, and recordings live. This makes it easier for everyone to find and reuse insights. Scheduling and coordination: Use shared calendars and tools to streamline study planning and avoid conflicts. By focusing on these areas first, I reduced friction and freed up researchers to focus on their work. Choose Tools That Fit Your Workflow Selecting the right tools is critical. I recommend starting with tools your team already knows or that integrate well with your existing systems. For example: Use spreadsheets or simple databases for participant tracking. Use cloud storage or knowledge management platforms for research repositories. Use calendar apps with shared access for scheduling. Avoid investing in complex software before you understand your team’s workflow. You can always upgrade tools later as your ResearchOps matures. Define Clear Roles and Responsibilities Even if you’re a small team, defining who handles what keeps things running smoothly. In my experience, having a dedicated ResearchOps lead or coordinator helps maintain focus. This person can: Manage participant recruitment Maintain the research repository Coordinate study schedules Train new team members on processes If you don’t have a dedicated person, distribute these tasks clearly among team members to avoid confusion. Whiteboard showing a ResearchOps workflow with tasks and responsibilities Build Processes That Scale Processes should be simple but scalable. For example, create templates for: Consent forms and participant communication Research plans and reports Data storage and privacy compliance Standardizing these documents saves time and ensures consistency. As your team grows, these processes will help onboard new members quickly and maintain quality. Foster Collaboration and Communication ResearchOps is not just about tools and processes; it’s about people working together. Encourage regular check-ins and knowledge sharing. I found that weekly syncs where researchers and ResearchOps coordinators discuss upcoming studies and challenges build trust and improve coordination. Also, make research findings accessible to the whole organization. Use newsletters, presentations, or internal websites to share insights. This increases the impact of research and helps justify the investment in ResearchOps. Measure and Improve Continuously Once your ResearchOps function is running, track what works and what doesn’t. Collect feedback from researchers and stakeholders regularly. For example: Are participant recruitment times improving? Is the research repository easy to use? Are studies running on schedule? Use this feedback to refine processes and tools. Continuous improvement keeps ResearchOps aligned with your team’s evolving needs.

By Philip Burgess | UX Research Leader When I first joined a product team as a UX researcher, I quickly realized that not all research efforts were pulling in the same direction. Some projects focused heavily on usability testing, while others prioritized market trends or stakeholder opinions. At first, I thought this diversity was a strength. But soon, I noticed a pattern: when UX research priorities were misaligned, the product suffered in ways that were hard to measure but very real. This post explores the hidden costs that come with misaligned UX research priorities. I’ll share practical examples from my experience, explain why alignment matters, and offer tips to keep your UX research focused and effective. Why UX Research Priorities Often Get Misaligned In many organizations, UX research sits at the intersection of multiple teams: product management, design, engineering, marketing, and sometimes even sales. Each group has its own goals and expectations, which can pull research in different directions. For example, product managers might want quick validation of features to meet deadlines, while designers seek deep insights into user behavior to inform the next iteration. Marketing teams may push for research that highlights competitive advantages or customer personas. Without a shared understanding of what the research should achieve, priorities can clash. This misalignment often happens because: Lack of clear research goals: Teams don’t agree on what questions the research should answer. Conflicting timelines: Different teams want results at different speeds. Limited resources: When time and budget are tight, teams compete for research attention. Poor communication: Teams don’t share findings or adjust priorities based on new information. Conflicting UX research priorities causing confusion Conflicting UX research priorities can create confusion and reduce the impact of findings. The Real Costs of Misaligned UX Research When UX research priorities don’t align, the consequences go beyond missed deadlines or frustrated teams. Here are some hidden costs I’ve seen firsthand: 1. Wasted Time and Effort Teams may duplicate research efforts or conduct studies that don’t answer the most important questions. For example, I once worked on a project where two teams ran separate usability tests on the same feature but with different goals. One focused on navigation flow, the other on visual design. Both tests produced valuable insights, but the overlap meant wasted time and resources. 2. Confusing or Contradictory Insights When research focuses scatter, findings can conflict. One team might highlight a usability issue, while another emphasizes user satisfaction with the same feature. This makes it hard for decision-makers to know which insights to trust or prioritize. 3. Poor Product Decisions Misaligned research can lead to decisions that don’t reflect user needs. For example, a product team might prioritize a feature based on stakeholder enthusiasm rather than user pain points uncovered by research. This can result in features that users ignore or find frustrating. 4. Lower Team Morale When teams feel their research needs aren’t heard or valued, motivation drops. I’ve seen designers become disengaged when their user insights were overshadowed by business-driven research priorities. This can hurt collaboration and creativity. How to Align UX Research Priorities Effectively Based on my experience, aligning UX research priorities requires intentional effort and clear communication. Here are some steps that helped my teams stay on the same page: Establish Shared Research Goals Start by defining what the research should achieve. Bring together stakeholders from product, design, marketing, and other teams to agree on key questions. For example: What user problems are we trying to solve? Which features need validation? What metrics will define success? Having a shared goal helps focus research efforts and avoid duplication. Create a Research Roadmap A roadmap outlines when and how research will happen. It balances the needs of different teams and sets realistic timelines. For example, schedule exploratory research early in the product cycle and usability testing closer to release. This helps manage expectations and resource allocation. Communicate Regularly and Transparently Keep all teams informed about research plans, progress, and findings. Use shared documents, presentations, or regular meetings to update stakeholders. Transparency builds trust and allows teams to adjust priorities based on new insights. Prioritize Based on Impact and Feasibility Not all research questions are equally important or easy to answer. Use criteria like potential user impact, business value, and resource availability to prioritize studies. This ensures the most critical questions get attention first. Focused UX research analysis to align priorities Focused UX research analysis helps teams align priorities and make informed decisions. Practical Example: Aligning Priorities in a Mobile App Redesign In one project, our team was redesigning a mobile app with a complex user base. Initially, product managers wanted to focus on adding new features, while designers pushed for improving navigation based on user complaints. Marketing wanted research on user demographics to tailor messaging. We held a workshop to align priorities. Together, we agreed the first research phase would focus on understanding navigation pain points, as this was the biggest barrier to user retention. We scheduled demographic research for later, after improving usability. This alignment helped us: Avoid conflicting research efforts Deliver actionable insights that improved the app’s flow Increase user retention by 15% within three months Final Thoughts on UX Research Alignment Misaligned UX research priorities can quietly drain your team’s energy, waste resources, and lead to products that miss the mark. Aligning priorities takes effort but pays off in clearer insights, better decisions, and stronger collaboration.