prompt
FreeAgent Trajectory Triage Specialist
About prompt
This prompt, found in the ai-boost/awesome-prompts repository, instructs an LLM to act as an Agent Trajectory Triage Specialist. It is designed to filter production agent execution traces for evaluation, debugging, fine-tuning, skill mining, and incident review when the volume of traces is too large to read manually. The prompt outlines a lightweight signal-based approach that uses interaction signals (e.g., user repeats, corrections, frustration), execution signals (e.g., tool errors, plan revisions, cost spikes), and environment signals (e.g., state changes, permission requests). It emphasizes no ground-truth labels and cost efficiency, requiring triage rules to justify any additional LLM calls against simple heuristics. The core responsibilities include defining a single triage purpose per pipeline (e.g., eval set construction, regression hunting, skill mining, safety review, cost outliers) and building a signal taxonomy across three dimensions. The prompt covers assumptions, targets, and example signals in a structured format.
Key Features
Pros & Cons
- No ground-truth labels needed for trace triage
- Lightweight heuristics minimize additional LLM calls for filtering
- Covers multiple triage targets with dedicated pipelines
- Accounts for cost and scalability in production environments
- Provides a structured taxonomy for identifying informative traces
- May miss interesting traces that fall outside the defined signal taxonomy
- Requires careful tuning of thresholds for signals like retry count or plan revisions
- Effectiveness depends on domain-specific adaptation of signal categories