Prompt Debugging & Iteration — Prompt Engineering for Agents
[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Prompt Debugging & Iteration
When Your Agent Misbehaves
Prompt engineering is 20% writing and 80% debugging. Here's a systematic approach.
The Prompt Debugging Checklist
| Symptom | Likely Cause | Fix |
|---|---|---|
| Calls wrong tool | Vague tool descriptions | Add specific use-case guidance |
| Hallucinates arguments | Ambiguous parameter names | Use descriptive names + examples |
| Ignores constraints | Constraints buried in long prompt | Move constraints to TOP + bold/caps |
| Loops infinitely | No termination criteria | Add "stop when X" + iteration cap |
| Generic answers | No domain context | Add company/domain context section |
| Wrong output format | Format not enforced | Add format examples + "ONLY this format" |
The APE Method
- Action: Run agent on 10 test cases, record failures.
- Prompt: Modify ONE thing to address the most common failure.
- Evaluate: Re-run all 10 cases. Did it improve?
// Systematic Prompt Iteration Log // v1: Base prompt → 4/10 pass // v2: Added constraints → 6/10 pass // v3: Added few-shot examples → 8/10 pass // v4: Added negative examples → 9/10 pass // v5: Added structured CoT → 10/10 pass
🚧 Critical Rule: Change only ONE thing per iteration. If you change 3 things and quality improves, you won't know which change helped.
Version Control Your Prompts
- Store prompts in Git, not hardcoded in app code.
- Tag each version with its eval score.
- A/B test changes with canary rollouts.
- Maintain a changelog explaining WHY each change was made.
⌨ HANDS-ON LABDiff and Version Control Agent Prompts
⭐ +150 XPTrack prompt regressions across git commits using prompt diffing and automated validation tests.
1Diff prompt changes against previous commit.
OBJECTIVE 1 / 1 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 2
If an agent ignores constraints, what's the most likely fix?
Use a bigger model
Move constraints to the TOP of the prompt with emphasis
Remove constraints
Add more tools