The economics of software creation are shifting fundamentally. AI agents compress weeks of development into hours, letting a single developer generate and deploy functional prototypes instantly. This raises a key question: How do we match rapid development with thoughtful, rapid user research?

While human teams explore limited prototypes due to capacity, AI-powered teams remove these constraints. We can shift from testing one or two to ten or more variations—but this scale of work amplifies our need for direction. Without proper research, we risk moving faster in the wrong direction.

Our human creativity and knowledge is no longer the bottle neck to creating a breadth of detailed concepts. What does the process to create innovative new products look like without this constraint?

The need to find challenges and solve them with novel solutions hasn’t been alleviated with the increased pace of development — this has only made it easier to go faster in the wrong direction. We need tools and techniques that help us highlight the paths to direct these powerful AI agent teams towards solving the challenges we specify with our human-centered insights.

The Seduction of Personal Problem-Solving

Every breakthrough starts with a moment of frustration—that instant when you encounter a problem and think, "I could build something better." AI agents have transformed these moments into immediate opportunities for innovation. No need for engineering teams or extensive resources—just you, your problem, and an AI partner ready to generate solutions.

Personal Experience: The Innovation Seed

Your own challenges provide the perfect starting point for innovation. They offer deep context, genuine understanding, and authentic motivation. However, personal experience serves best as a compass pointing toward opportunity, not a complete map of the solution space.

Think of your problem as the first data point in a broader research journey. It helps you identify who the user persona is you will interview that shares your challenge, what questions to ask them, and how to recognize patterns in their shared experiences. This is where the path splits: you can either keep your personal solution or embark on the journey to understand and solve this problem for others at scale.

The Critical Role of User Discovery

Building on personal insight, effective user research in the AI age focuses on expanding understanding through systematic discovery. We need to learn:

• How others experience and articulate the problem

• Which aspects of the challenge cause the most pain

• How different users currently navigate similar obstacles

• Where existing solutions fall short of expectations

• What success looks like across different contexts

• The unexpected ways people work around the problem

• The broader ecosystem in which the solution must function

This systematic exploration transforms personal intuition into validated insight. It reveals whether your initial hypothesis resonates with others and, more importantly, how it needs to evolve to serve your user base effectively.

Through conversations with potential users, patterns emerge that help shape the solution. Your personal experience helps you ask better questions and recognize meaningful insights, but it's the synthesis of multiple user perspectives that reveals the true scope and nature of the problem worth solving.

Launch and Learn Shortcomings

Think of product development as a funnel—start broad with research, then progressively narrow based on validated insights. Skipping steps might save time initially, but it's often why products and startups struggle to gain meaningful traction. Your personal problem might be pointing to a massive opportunity, but without proper exploration, you'll likely build something that serves too few users too narrowly.

The Art of Intentional Fidelity: From Low-Fi to AI-Fi

Deep Discussion Doesn’t Necessarily Mean High Fidelity Prototypes

The true art of product research isn't about fidelity levels—it's about directing attention to what matters most at each stage of development. AI's ability to generate high-fidelity prototypes quickly doesn't change this fundamental principle; if anything, it makes intentional fidelity choices more critical.

Crafting research stimuli with AI, allows you to choose the fidelity. We should not assume we must create software that looks like a completed software product. It is up to the imagination of the human research lead to imagine conversation drivers from software high-fidelity prototypes, lower fidelity prototypes, sorting activities, surveys, and more.

Strategic Conversations: The Power of Constraint

Early strategic discussions benefit from deliberate constraints. When exploring foundational questions like:

• Process architecture

• Organizational workflows

• Decision ownership

• Output requirements

• System boundaries

Low-fidelity tools create a focused conversation space. A simple flowchart or task map can spark deeper insights than a polished interface that accidentally anchors user research conversations to current paradigms.

The Paradox of AI-Generated Detail

While AI can generate remarkably complete prototypes, this capability presents a new challenge: attention diffusion. A fully realized prototype might actually impede strategic discussions by distracting users rather than help drive a constructive conversation:

• Drawing focus to implementation details prematurely

• Suggesting technical constraints that don't exist

• Creating conceptual anchoring to old familiar patterns

• Overwhelming participants with too many options

AI Validation Research

There is a common pitfall that a more realistic prototype yields better research conversations. This has not been my experience, early validation research conversations using rough prototypes and sort activities have been the most fruitful for me in driving strategic new product development.

AI Usability Testing

As the concept evolves usability testing to refine the product into the best configuration that is the clearest to the user becomes more important over gathering fresh opportunity areas. This is where AI agents could really shine, but we must be measured as not to A/B test our products into a consensus driven mess. It is important for the human product leads to set priorities, vision and features. Keeping this clarity of purpose will ensure insights are translated into clear features in a concise roadmap.

AI Agents Can Help Accelerate Research

The future of user research will be transformed by AI agents acting as powerful research accelerators—not replacements for human researchers, but as tools that amplify our capabilities across the research lifecycle. Here's where AI agents are poised to revolutionize our research practice:

Research Planning & Design

• Generate nuanced discussion guides aligned with research hypotheses

• Create multiple fidelity levels of research stimuli for different objectives

• Suggest research activities based on past successful methodologies

• Design flexible testing protocols that adapt to participant responses

Insight Collection & Analysis

• Real-time transcription and initial pattern identification during sessions

• Rapid synthesis across multiple interview transcripts

• Cross-referencing insights against existing research databases

• Generating insight clusters and theme identification at scale

• Transforming 25 hours of interview recordings into structured insights in minutes

Interactive Research Support

• AI chat agents conducting preliminary screening interviews

• Standardized parts of research sessions handled by AI moderators

• Follow-up probing questions generated in real-time

• Consistent data collection across multiple research streams

The key evolution here isn't just speed—it's depth and consistency. While human researchers remain essential for strategic direction, hypothesis formation, and nuanced understanding, AI agents can help us explore more broadly, synthesize more deeply, and validate more thoroughly than ever before.

The Future of AI-Powered Product Development

This human-AI collaboration could fundamentally change how we understand user needs, making comprehensive user research more accessible and actionable for teams of all sizes.

The Fundamentals Are The Same The Tools Are Changing

AI hasn't changed the fundamental principles of good research—it's given us tools to apply them quickly and effectively. AI agent tools will bend to our taste—it is up to us to art direct the right fidelity of thought in our work for the phase of research. The challenge isn't generating high-fidelity prototypes; it's choosing the right level of detail to support meaningful discovery at each stage of development.

Those Who Align Depth of Insight With Execution Win

The winners in this new era won't be those who can generate the most prototypes the fastest, but those who can gather meaningful insights to drive informed product development the fastest. Building an informed innovation practice won’t change in the AI age it will still mean:

• Asking the right questions

• Focusing research participant attention effectively

• Generating a breadth of insights

• Iterating the solution space with research insights

The tools have evolved, but the core principle remains: great products solve real human needs, and understanding those needs requires intentional, focused research—regardless of how quickly we can prototype solutions.

It has always been a bad habit to jump right to building solutions, AI agents capable of turning our very thoughts into fully functional software experiences make this even more tempting.

Join the discussion in the comments below or reach out directly—I'd love to hear your thoughts

🧲 AI Agent Magnet

Anthropic has released a new open-source standard to connect AI agents with data sources such as content repositories, business tools and dev environments. This work punctuates the importance of data to creating AI agents that can perform increasingly complex tasks. Today, the fragmented connections to data makes AI agents brittle. This is the first step towards a universal agent protocol that could allow agents to talk and coordinate. There are many safety and security concerns with such agent-to-agent communication and this is a good first step to learn as a technical community building these systems.

LangChain’s LangSmith has become where the developers are hanging out to experiment with new agentic experiences. “With nearly 30k users signing up for LangSmith every month, we’re lucky to have front row seats to what’s happening in the industry.” Platform growth has always been dependent on where the developers are choosing to give their attention and time. LangSmith is becoming that place for AI agent development. To stay on the cutting edge, in 2025 we should all be paying close attention (and our own experiments) to this platform.

OpenAI released Their o3 model as the final announcement for their 12 Days of Christmas release. o3 is capable of advanced reasoning able to reason through complex challenges. To wrap-up 2024, AI companies have been releasing a dizzying amount of announcements including o3, Gemini 2, Grok 2, Mistral 7B, and Llama 3.1

💬 Suggestion box

A newsletter exploring the principles that will help us design AI agent experiences and startups that amplify human creativity.

Subscribe to join a community of designers and developers shaping purposeful AI agents.

Until next time, keep innovating and stay curious!

Jason