Claude Code Team Shares Insights on Agent Evolution and Team Collaboration
Cat Wu and Thariq Shihipar from Anthropic's Claude Code team recently participated in a fireside chat at the AI Engineer World's Fair. The conversation covered a wide range of topics including Claude Code, Claude Tag, Fable, coding agent security, evals, tool design, and how Anthropic uses these tools internally. The full video of the session is available on YouTube.
Simon Willison, who hosted the session, noted that Claude Code launched in February of the previous year and is now under a year and a half old. It originally appeared as a bullet point in the Claude Sonnet 3.7 launch announcement.
How Daily Work Has Changed with Coding Agents
Cat Wu described the evolution of working with coding agents. When Claude Code first launched with Sonnet 3.7, users had to closely monitor every action the agent took. She recalled reading every permission prompt carefully and frequently rejecting actions. Now, with each new model generation, the experience has improved dramatically. She said the team has been able to delegate more menial implementation tasks to Claude, freeing up time for creative work like designing better user experiences. With Fable, she noted a step change improvement where many features can now be one-shot.
Thariq Shihipar shared his personal journey with Claude Code. He first heard about it from a friend who told him to try it when Opus 4 came out. After trying it, he immediately wanted to work at Anthropic. He reflected on how quickly people forget the early days when they had to approve every permission prompt. Now, he focuses on pushing himself to produce higher quality work than ever before. He has been using Claude Code to edit videos, meeting the exacting standards of the brand team in just a few hours.
Conventional Software Engineering Wisdom That No Longer Holds
Cat Wu identified a major shift in engineering skill sets. Two years ago, product managers would spend six months talking to customers, aligning cross-functional teams, and writing detailed specs before any code was written. Now, the timeline from idea to implementation has shrunk from six to twelve months down to perhaps a week. She advised engineers to develop stronger business sense and product taste to decide what is worth building. Execution skills have become less critical in most product domains, though infrastructure work still requires careful attention to detail.
Thariq Shihipar offered a different perspective. He said that rewrites are now good. The old wisdom from The Mythical Man-Month that said never rewrite is no longer valid. He believes that a codebase is a spec, and rewriting can help distill and create better versions of that spec. He mentioned that Bun was rewritten in Rust and works great, and noted that Anthropic started shipping Claude Code on Bun-in-Rust to everyone on June 17th.
Claude Tag: Multiplayer, Proactive, and Team Memory
Claude Tag launched about a week before the chat. It is a version of Claude that lives in team collaboration tools, initially within Slack. Cat Wu explained three key differences from Claude Code. First, Claude Tag is multiplayer by default. Once added to a Slack channel, team members can chime in and collaborate on pull requests. Second, it is proactive rather than reactive. Users can tell Claude Tag to monitor bug reports in a channel, create PRs to fix them, and tag the relevant engineer automatically. Third, it includes team memory. If users tell Claude Tag their preferences in natural language, it remembers them for future interactions.
Internally, Anthropic sees Claude Tag as the evolution of Claude Code. Cat Wu revealed that Claude Tag currently lands 65% of product engineering PRs at Anthropic. This represents more than half of all PRs for the product engineering team. The team splits work between Claude Code and Claude Tag based on complexity. Claude Code remains the best tool for complex tasks requiring interactive iteration, while Claude Tag handles proactive work like fixing bug reports without manual intervention.
Thariq Shihipar highlighted non-coding uses for Claude Tag. Before the talk, the team asked Claude Tag when Fable would release to align it with the announcement. Claude Tag searched Slack and identified who had been discussing the topic. He described it as a search engine for the company, with access to all product context. Users can ask metrics-related questions by connecting it to event stores. The marketing team has used it to learn about features by having Claude clone the codebase and generate recordings of the feature in action.
Multiplayer Sessions and Social Dynamics
Cat Wu noted that a large percentage of Claude Tag sessions are multiplayer. A typical workflow might involve someone suggesting a new feature, tagging Claude Tag to do a first pass, then asking it to share a recording for design review. Designers can nudge the output, and engineers take it to production. The team is still working out the social dynamics for steering the same session, but people have found it intuitive to integrate Claude Tag into their teams by observing how others use it.
Thariq Shihipar added that Claude Tag is great for teaching people and reducing slop. Because everyone sees how others use Claude, it levels up everyone's skills. He compared it to how Midjourney solved the challenge of teaching advanced image prompting by enforcing public prompting in Discord channels.
Prioritization in a World of Cheap Features
Simon Willison asked how the team decides which features are worth building when the cost of building has dropped so much. Cat Wu explained that the team dogfoods their products every day. When they encounter something they want to do but cannot, they fix the product instead of finding a workaround. They share products internally at Anthropic first, then with early customers who provide honest feedback. They have an internal bar for active users and retention that a feature must meet before it is shared publicly. If a feature is not polished, people will churn, and the team will not ship it. Using internal user retention as a decision metric helps ensure quality.
Surprising Feature: Remote Control
Cat Wu shared an example of a feature that surprised the team. Remote control allows users to connect a mobile device or web browser to a local Claude Code session running in the CLI. She initially did not understand the need, as she prefers to kick off tasks directly on mobile and run them in cloud sessions. However, after rolling out remote control, many users told her they plug their laptops into power, open multiple remote control sessions, lock the screen, and then control Claude Code from their phones on the couch. The team is now leaning into this workflow.
Code Review in the Age of Coding Agents
Simon Willison raised the topic of code review, a major theme at the conference. He asked how much attention humans pay to reviewing code written by coding agents. Thariq Shihipar said it varies by task. For important areas like the system prompt, there are code owners who must approve any changes. A GitHub bot reviews every PR, often doing the bulk of the review. For complex PRs, team members might create artifacts to explain the changes for reviewers. The team invests heavily in verification and CI/CD to catch failures.
Cat Wu explained that the team is moving toward a world where humans do not need to be in the loop for most changes. For critical changes to the core of Claude Code and other products, code owners still manually review everything. But for outer-layer changes, Claude now handles the full code review. This transition took over six months of incremental steps. The team started with human review for everything, then gradually identified files where automated review caught 100% of issues. When incidents occur, the team reviews the PRs that caused them, updates the automated review to catch those issues, and adds the PRs to an eval set to prevent regression.
Building Trust in New Models Through Evals
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
Simon Willison asked how the team builds trust in new models like Fable. Cat Wu said the main reason they build up an eval base over time is so new models can be a drop-in replacement. When a new model arrives, they run the entire eval set to ensure it is strictly better than the previous model. This gives them confidence to deploy it.
She explained that the team has both model evals for Anthropic as a whole and team-specific evals. They run code review across every repo at Anthropic. For features like auto mode, they have evals across all users at Anthropic and have commissioned multiple external testers to red team the system with prompt injections and malicious inputs.
System Prompt Evals and Behavioral Improvements
Simon Willison asked how the team knows if a system prompt improvement actually improves the product. Cat Wu acknowledged they do not have complete confidence, but they work hard to avoid regression. They start with a suite of external evals they trust, then complement it with an even larger suite of internal evals. They primarily optimize for capability: given a task definition and the full codebase, does Claude make the right decisions, fix bugs, and pass tests?
However, they also track behavioral issues that impact user experience. For example, users dislike when Claude Code says it is time to go to sleep or asks for permission to continue after completing part of a task. The team builds behavioral evals to catch these issues and prioritizes them based on user feedback. They encourage users to be vocal about their feedback so the team can rank issues and build evals for each one.
Collaboration Between Claude Code Team and Model Training Teams
Simon Willison asked about the interaction between the Claude Code team and the teams training the models. Cat Wu said they work closely together, meeting often to discuss what the next generation of models should be able to do. The research team publishes blog posts about targeting longer-horizon work and training Claude to be honest, harmless, and helpful. They also focus on aligning Claude with user intent, even when that intent is expressed vaguely.
System Prompt Reduction by 80%
Thariq Shihipar revealed that the system prompt for Claude Code has been reduced by 80% thanks to Claude Fable and Opus 4.8. The team now uses different system prompts for different models. One key insight was that they were over-constraining Claude. Earlier models like Opus 4 needed many examples, but removing examples was extremely helpful because Claude became more creative than the examples provided.
Simon Willison noted this contradicts his usual advice to give models examples. Thariq agreed and said the focus now is on the shape of what you give Claude: the tools and the system prompt. They also try to provide more context and fewer "do not do this" instructions, because those can conflict with user instructions and confuse Claude. The team aims for fewer hard constraints, more context, and fewer instructions overall.
Cat Wu added that when prompting models, users should consider edge cases. They reviewed all instructions in the Claude Code system prompt and found cases where a statement was 90% true but had a 10% exception. They did not want to constrain the model or confuse it. For example, they had instructions saying to always verify front-end changes, but if a user asks for a quick copy change, verification may not be needed. They adjusted the wording to say that for larger user experience changes, running the app locally is appropriate.
Simon Willison pointed out that this relies on the model's judgment, which is only available with frontier models like Opus and Fable. Cat Wu confirmed that the team now uses different system prompts per model. Only the most frontier models have the 80% token decrease, while older models still use the full system prompt.
Subagents and Prompting Models
Simon Willison noted that a year ago he did not trust a model to write a prompt, but now good models are excellent at prompting. He said thinking about subagents helped him accept this, because it involves one Claude model setting up a prompt for another Claude model. Thariq Shihipar added that workflows take this further, with Claude not just prompting a single subagent but orchestrating many subagents, each with a detailed prompt. He has also used Claude to prompt image generation models via the Gemini API, finding it less lazy than he is at prompting.
Cat Wu mentioned that Claude also wrote the prompt for the workflow tool. Simon Willison expressed frustration that Anthropic publishes prompts for Claude Chat but not for Claude Code tools. He said he still has to run a proxy to intercept them and would love to see the Claude Code prompts published deliberately. Cat Wu said she would write down that feature request.
Tool Design Philosophy
Simon Willison asked about the bar for introducing new tools. Thariq Shihipar said his career peaked when he introduced the ask user question tool. He described tool design as more of an art or biology than a science. The team trends toward fewer tools. The last set of tools introduced was the task tool. They try to give Claude more general versions to do things.
Cat Wu largely agreed but added that as they introduce tools, they keep the cardinality low and ensure each tool has a distinct function so Claude can easily distinguish when to call each one. For the file edit tool, they keep it because it allows them to render a dedicated UI showing when Claude makes a file change. Many new users still appreciate this experience. However, for users on auto mode, she said the file edit tool may not matter and could be removed.
Safety and Security: Auto Mode and Prompt Injection
Simon Willison raised the topic of prompt injection risks. He admitted he still mostly runs Claude Code in YOLO mode and feels guilty about it. He asked for advice on safely running Claude Code within Anthropic.
Thariq Shihipar said the team trusts auto mode significantly. He noted that auto mode is the feature that enabled Claude Tag. The team has invested heavily in making auto mode safe through extensive evals and red teaming. He encouraged users to move from YOLO mode to auto mode for better safety.
Cat Wu added that the team has built multiple layers of defense. They have evals across all users at Anthropic and external red teamers who test for prompt injections and malicious inputs. The system is designed to reject suspicious commands and ask for confirmation when needed.