AgentGuard

Introduction

AgentGuard is a full-stack platform for evaluating AI agents before they touch production. Point it at an agent workflow and it scores the run across reliability, tool-call correctness, security, and execution behavior — then generates a report you can actually act on.

Why I built it

Demos lie. An agent that works in a screen recording can fail silently in production — wrong tool, wrong arguments, infinite loop, leaked secret. I wanted an honest answer to "does this agent actually work?" So I built the evals first and the platform around them.

Core features

• Configurable evaluation pipeline — compose checks per workflow, from deterministic assertions to trace analysis.
• Reliability scoring — task completion, stall/loop detection, and execution behavior tracking.
• Tool-call correctness — validates the right tools were called, in the right order, with valid arguments.
• Security layer — SSRF protection, API-key authentication, and rate limiting baked in.
• Automated reports — scored runs with traces you can inspect, not just a pass/fail.

Stack

React frontend, FastAPI backend, PostgreSQL for trace storage, fully containerized with Docker. Deterministic checks run server-side so results are reproducible.