I had some great DMs in the community this week asking for us to research examples of software that is suited for both human and ai agent interfaces.

Admittedly, this is a new are and teams are not yet designing in this fashion, but there are early clues emerging online we can take inspiration from as we begin to include this new user, the AI agent in our design work.

From last week’s post we introduced we must consider another user when designing the future of collaborative software experiences — AI agents. In this edition, we're diving deep into the exciting world of human-AI collaboration and exploring examples of software that seamlessly integrates both human and AI agents.

As AI becomes increasingly integrated into our digital tools, we need to rethink traditional usability paradigms. It's no longer enough to design solely for human users; we must consider the needs and capabilities of AI agents as well.

To address this challenge, we've developed a human-AI agent usability framework for assessing human-AI collaboration from a usability standpoint.

This framework evaluates software based on four key themes:

1. Implement clear semantic structures for both visual and programmatic interfaces: This ensures both humans and AI agents can easily understand and interact with the software.

2. Develop robust error detection and recovery systems: This minimizes disruptions and ensures smooth collaboration, even when unexpected events occur.

3. Create explicit pathways for stepwise planning and execution: This facilitates coordinated action and efficient task completion in collaborative scenarios.

4. Build interaction patterns that serve both human and synthetic users: This fosters seamless communication, mutual understanding, and effective collaboration between humans and AI agents.

A Framework for Assessing Human-AI Collaboration

To illustrate the framework's practical application, we evaluated three popular software tools using the new human-AI agent usability framework the software we reviewed to find commonalities in software use for human-ai collaboration.

These three software solutions were chosen for three reasons:

1. The infinite canvas is a powerful software paradigm for creation experiences that facilitates clear structuring of interfaces and planning for humans and AI agents alike

2. Collaboration is a first-class affordance in how these software solutions are architected and I can picture it between humans and AI agents alike working together as these agents become more capable.

3. Early agent integration is present in all three of these solutions are early adopters of this new technology as they experiment with the different paradigms for agent and human collaboration.

The tools we’ve tested represent software I believe are noteworthy for the traits of enabling human-ai collaboration. These three examples are in no way an exhaustive list of options. In the comments, please share your ideas for software that is for consideration, better yet score them with your own human-ai agent usability framework and share it with the community.

• FigJam - The online whiteboard for generating ideas, align on plans, and connect as a team online.

• TLDRaw - A free and instant collaborative whiteboarding tool with an integrated ‘computer’ mode for helping design AI automation workflows.

• Gumloop - The no-code platform to build and launch your own AI-powered business automations and tools.

This framework provides a structured approach for evaluating software usability in human-AI collaborative settings, ultimately seek to contribute to the design and development of more effective and user-friendly AI-powered applications. We encourage you to explore our framework and use it to evaluate software yourself. Here are some tips to get started:

How This Tool Fits Into Your Design Toolbox

As we begin designing for human-ai collaboration we need frameworks to recognize and critique these new experiences. Having a framework for thoughtful critique will empower teams building collaborative AI agents to gather best-in-class inspiration for their own work. The key themes for inspiration are:

• Improved User Experience: By considering the needs of both human and AI users, we can create more intuitive, efficient, and enjoyable software experiences.

• Enhanced Collaboration: Designing for human-AI collaboration can lead to more effective and productive teamwork, unlocking new possibilities for innovation and problem-solving.

• Responsible AI Development: By prioritizing usability and ethical considerations, we can ensure that AI systems are used responsibly and benefit both humans and society as a whole.

Human AI Agent Collaboration Scores

Each criterion is evaluated on a scale of 1 to 5, with 1 being the lowest and 5 being the highest.

• TLDRaw (total:4.0), emerges as the strongest contender scoring an averaged total across human and AI agent usability of 4. This emerging tool while in its infancy illustrates a sensitivity of being intuitive for human use and illustrating clear AI agent guidance and control. The software has its issues, lacking the depth of control and clear communication of features for humans and AI, but overall it has a strong start.

• FigJam (total: 3.9),the strongest human interface, at 3.9, and the weakest AI agent tools. It has a simple interface and clear interaction patterns, is highly user-friendly for humans. However, it offers less customization for AI agents compared to Gumloop, with only basic manipulatable components.

• Gumloop (total: 3.8), scored the highest for AI agent interactions with a 4.1. It is powerful for AI agent interaction due to its clear instructions and customization potential, presents a steeper learning curve for human users, scoring the lowest 3.8, due to its software complexity. The tool seeks to overcome human usability issues with a form view of agent automation views, but this does not help the humna-ai collaboration experience.

Here Is Your Template

This framework is a tool to help us be aware of what beneficial patterns help human and AI agent users collaborate. As we create new software UX/UI paradigms, we must consider how to best support this emerging form of teamwork.

What other traits or design patterns can you think of that might help promote collaboration with humans and AI agents? For example, how can we design for explainability and trust, or how can we ensure that AI agents are able to adapt to the different ways that humans work and learn?

• Familiarize yourself with the criteria and metrics: Understand what each theme and criterion measures and how it relates to human-AI collaboration.

• Gather user feedback: Collect both quantitative and qualitative data from human and AI users to gain a comprehensive understanding of the software's usability.

• Iterate and improve: Use the evaluation results to identify areas for improvement and refine the software to better support human-AI collaboration.

🧲 AI Agent Magnet

add 3x each week

The Anthropic Economic Index is a valuable resource for designers and developers of AI agents because it provides real-world data on how AI is being used in the workplace gathered from millions of conversations with Claude. The index shows that AI is primarily being used to augment human capabilities, particularly in fields like software development and technical writing . The team at Anthropic suggests that AI agents will likely play a collaborative role in the future workforce, working alongside humans to improve productivity and efficiency rather than displace these roles. I’m jot so sure, I think some jobs will be outright replaced while those that deepen their skills will be augmented as mentioned. The next link by Replit’s CEO details this point of view…

This is a thoughtful clip by Amjad (CEO Replit), If you’re in tech and want to stay relevant you have two options to hyper specialize or become a generalist. 1. to get close to the ‘bare metal’ becoming hyper-specialized, he mentions NASA won’t be using Javascript to launch rockets. I would say this is true of all the companies building embodied AI from humanoid robots to cars. 2. He mentions we need generalists like designers and product managers who are capable of building one-person enterprises. The paths to avoid are those that are well trodden today and represented in data on Github for example that all agent co-pilots are based on. The path to continued relevance is being uniquely helpful.

Lately, I’ve been experimenting with Langflow. I’m impressed with the clear interface and relationship to the code view and agent app preview. Feels related to Xcode, but instead of for iOS apps it’s an agent builder. Langflow’s clear focus on accuracy of agent work is compelling and worth keeping an eye on as this open source community evolves the tool.

💬 Suggestion box

A newsletter exploring the principles that will help us design AI agent experiences and startups that amplify human creativity.

Subscribe to join a community of designers and developers shaping purposeful AI agents.

New Referral Rewards

Until next time, keep innovating and stay curious!

Jason