The Future software design is LUX

In the rapidly evolving landscape of artificial intelligence, a new frontier is emerging: Language User Experience (LUX). As AI continues to reshape our digital interactions, the need for professionals who can craft meaningful, language-based experiences with AI agents is becoming increasingly critical.

This shift to language-based experiences demands a new breed of designers – LUX designers – who can bridge the gap between traditional UX and the unique challenges posed by AI agents. These agents are becoming both collaborative functions within applications and tools used to create next-generation experiences. The word ‘language’ in this context means speech, writing and gestures allowing for the embodiment of a range of experiences from apps to immersive experiences. LUX designers must develop new design thinking principles that consider AI agents in multiple contexts:

1. System Architecture: LUX designers integrate AI agents into the core architecture of experiences considering the information architecture of an agentic system planning the memory inputs, tools and environment the agent will access. For example, they might design an AI-powered customer service chatbot that seamlessly interfaces with a company's knowledge base and CRM systems.

2. Synthetic Collaborators: They create interfaces where AI agents work alongside human users. This could involve designing a collaborative writing tool where a host of specialized AI agent offer real-time suggestions, fact checks, research and edits as a human author writes.

3. Design of Specialized AI agents: LUX designers approach AI agents as products themselves, considering the role as a persona, data sources that support this role’s functionality, training methodologies, and fine-tuning processes. For instance, they might design the user interface for a healthcare AI agent helping detail the agent requirements documentation and then deliver to medical professionals to easily provide feedback, improving the AI's diagnostic capabilities over time.

4. Stakeholders in Service Experiences: They design agentic applications that interact with applications representing the user in a negotiation with a different businesses agentic application or human representatives. This work will involve creating an AI agent driven negotiation platform where AI agents represent different parties, requiring careful design of interaction protocols and ethical guidelines.

By addressing these multifaceted roles of AI agents, LUX designers are shaping the future of human-AI interaction across diverse applications and industries.

Reimagining design techniques and tools for non-determinism

Traditional design methodologies are struggling to keep pace with the non-deterministic nature of AI interactions. While game designers have long worked with various map structures (arena, linear, circular, branching, asymmetrical, and hub-and-spot), these concepts need to be reimagined for AI agent design.

We propose a hybrid approach that fuses elements from game design, service design, and interaction design. This new methodology introduces non-deterministic design maps that can account for the unpredictable nature of AI responses while still providing a framework for creating coherent user experiences.

Key to this approach is the ability to map out scenarios that highlight both the consistent behaviors of AI agents and areas where unexpected responses might occur. This balance between predictability and surprise is at the heart of engaging AI interactions.

Borrowing from across design disciplines for AI agent experiences

Game Design

Game design offers a wealth of principles that can be adapted for creating engaging AI agent experiences. Let's explore how these concepts can be applied in LUX design:

1. Clear Objectives for AI Interactions: Every AI interaction should have a defined goal, much like game objectives. In a customer service AI, the objective might be to resolve user queries within three interactions. LUX designers should balance concrete goals (like task completion rates) with fuzzier objectives (such as user satisfaction) to create more nuanced and realistic AI experiences.

2. Constraints in AI Capabilities: Just as game rules limit player actions, AI systems need clear boundaries. LUX designers must define what the AI can and cannot do, ensuring these constraints form a coherent user experience. For instance, a virtual assistant might be constrained to accessing only certain types of personal data, with these limitations clearly communicated to users. It is a LUX designers job to foster safe usage of AI agents by designing to minimize hallucinations and work within safety and governance guardrails.

3. Success Criteria for AI Performance: Establishing clear metrics for AI agents success is crucial. Charlie Munger, Vice Chairman of Berkshire Hathaway, said, "Show me the incentive and I'll show you the outcome." In an AI-driven language learning app, success might be measured by user progress in vocabulary acquisition or conversation fluency. Well-defined criteria help set user expectations for progression through a workflow and for developers provide benchmarks for how to improve AI agent performance.

4. Rewards in AI Interactions: Incorporate both intrinsic and extrinsic rewards in AI agent experiences. An AI fitness coach might offer intrinsic rewards through encouraging feedback, while also integrating with external reward systems like fitness tracker achievements. LUX designers should carefully balance these rewards to maintain user engagement without overshadowing the core AI interaction.

5. Playful AI Experiences: Infuse a sense of play into AI interactions to encourage user engagement, exploration and make this new technology approachable to the general public. This play could involve adding elements of discovery or surprise in AI responses, or designing interfaces that make interacting with AI agents feel more like a game than a chore. For example, an AI-powered educational tool might use playful language and interactive challenges to make learning more enjoyable.

6. Balancing Competition and Cooperation in AI Systems: Consider how competition or cooperation can enhance AI experiences. An AI writing assistant might introduce a gentle competitive element by challenging users to improve their writing scores, while also offering cooperative features like collaborative editing with the AI agent. LUX designers should be mindful of their audience, choosing competitive elements only when they enhance rather than detract from the objective of the user experience.

By adapting these game design principles, LUX designers can craft AI interactions that balance user needs with system capabilities, all while maintaining a sense of play and discovery that keeps users coming back for more.

Service Design

Let's explore how service design journey maps can be reimagined for AI interactions:

1. Holistic Experience Mapping: Service design emphasizes a holistic view of the user journey. For AI experiences, this translates to mapping not just the direct interactions with the AI, but also the broader context of the user's goals, emotions, and touchpoints with the entire system. This approach helps in designing AI agents that are more contextually aware and responsive to the user's main journey operating within the capabilities of the foundation model’s capabilities.

2. Sequencing with Non-Deterministic Branches: Service design uses sequencing to visualize the customer's journey through pre-service, during service, and after service stages. For AI experiences, we can adapt this concept to include non-deterministic branches. This allows designers to map out potential AI responses and user interactions, acknowledging that the exact path may vary based on the AI's learning and decision-making processes.

3. Evidencing for AI Transparency: The principle of evidencing in service design involves using visual aids to help team members understand the customer's stage in the journey. In AI experiences, this concept can be adapted to create transparency around the AI agent's decision-making process. Designers can incorporate elements that make the AI's "thinking" or approach more visible to users, helping to build trust and understanding.

4. Co-creative Iteration with AI: Service design emphasizes co-creation with stakeholders. In the context of AI experiences, this principle can be extended to include AI agents as a co-creator. Designers can help human workers about the techniques and methods to use these AI agent tools in their work.

5. Adaptive Touchpoint Design: Service design focuses on optimizing touchpoints throughout the user journey. For AI experiences, touchpoints need to be designed with adaptability in mind. These techniques will help LUX designers highlight happy paths, understand where systems are brittle and how to ensure AI agents fail gracefully. These types of capabilities also exist in game design and UX design techniques that should also be carried through to LUX methods.

By evolving the tools and principles of other design disciplines, like game and service design, LUX designers can create more comprehensive, transparent, and adaptive AI experiences that truly center on the human user's and AI agent’s needs and journey.

 Crafting a New Design Document for AI Agents

To address the unique challenges of AI agent design, we need to evolve our tools and mindset around our practices as designers. We propose a new Agentic Design Document (ADD) that builds upon the strengths of game design and service design tools while incorporating elements specific to agentic system design.

The ADD will serve as a guide for creating cohesive, engaging AI experiences. Teams designing and developing these systems need a single source of truth in the form of an Agentic Design Document to ensure teams are aligned on the approach to develop consistent agentic systems. It is this through the unified intent that teams will be capable of iterating on the performance of these non-deterministic systems to ensure their safe usage.

The future of AI agent design is embracing non-determinism

As we move forward, AI agent designers must become comfortable with non-deterministic outcomes. Like a tennis match where the rules and court are fixed but each game unfolds uniquely, AI interactions should offer structured unpredictability.

This approach opens up exciting possibilities for creating expectedly delightful yet surprising experiences. By embracing this philosophy, we can design agentic experiences that keep users engaged through a perfect balance of familiarity and novelty.

The Critical Role of LUX in Shaping Our AI-Driven Future

So, why does this matter? As AI becomes increasingly integrated into our daily lives, the way we interact with these intelligent systems will fundamentally shape our relationship with technology. LUX designers stand at the forefront of this revolution, tasked with creating the interfaces that will mediate our interactions with AI.

The early pioneers of this emerging field will not only design signature experiences but also create new design tools that will help pull the rest of the industry into this new paradigm. These innovators will be responsible for crafting interactions across a wide spectrum of AI-enabled devices and systems:

• Hardware: Robots, autonomous vehicles, smart home devices, augmented reality experiences

• Software: AI agent applications, AI-enabled operating systems, virtual assistants, metaverse experiences

• Services: a customer engagement process that includes every step in the journey from initial meeting to conclusion incorporating AI agents.

By developing these new methodologies and tools, LUX designers will pave the way for more intuitive, engaging, and meaningful AI interactions. They will play a crucial role in ensuring that as AI becomes more prevalent, it also becomes more accessible, understandable, and beneficial to users across all walks of life.

The field of LUX represents more than just a new design discipline; it's a bridge between human needs and AI capabilities. As we stand on the brink of an AI-driven future, the work of LUX designers will be instrumental in shaping a world where technology enhances human potential rather than replacing it.

For software designers and creative technologists aspiring to work in AI or already in the field, now is the time to embrace these new concepts by experimenting with non-deterministic design techniques to contribute to the evolution of AI interaction design.

We're currently designing the ADD framework to support strategic planning for non-deterministic software agentic applications. If you’re interested in beta testing the ADD framework please email us.

There’s a reason 400,000 professionals read this daily.

Join The AI Report, trusted by 400,000+ professionals at Google, Microsoft, and OpenAI. Get daily insights, tools, and strategies to master practical AI skills that drive results.

Sign up now for free and work smarter, not harder.

Surfing The Web

Three things i’m excited about this week: an interesting article on the open questions and themes in AI, a new agentic app and an interview of Sierra’s co-founder and product lead on making AI experiences delightful.

Suggestion Box

Forward it to a friend and have them signup here.

Until next time, keep innovating and stay curious!

Jason and Anastasia