June – August 2026
New York, NY
Zeta Global
Core AI Engineer Intern
01Found where it failed. Classified production LangSmith traces into three recurring failure types: tools that broke but still reported success, bad tool parameters, and tools called out of order. The labeled dataset set fix priority and became the baseline for measuring improvement.
02Fixed the agent’s tools and instructions. Rewrote tool descriptions and schemas and wrote agent instructions and skills, spelling out each tool’s purpose, inputs, outputs, and when not to use it.
03Made the fixes measurable. Built smoke, regression, and progression evaluation suites with custom Python evaluators and promptfoo that check backend state instead of trusting a successful reply. Progression tests that pass consistently are promoted into regression, so new abilities can’t silently break.
04Cleaned up the infrastructure. Split one large MCP server into two focused FastMCP servers on AWS Bedrock AgentCore, sharing tool code from one internal package, and fixed the platform SDK’s local evaluation pipeline so trajectory evaluators grade real agent runs instead of empty traces.
ResultAn improvement loop the team can run in production, which I presented along with a handoff plan.
Python · LangSmith · promptfoo · MCP · FastMCP · AWS Bedrock AgentCore
