GenAI, 2023 to present
Conversational analytics assistant
An agentic RAG assistant that answers questions about workforce data for Amazon employees, in plain language, with sources.
Lead engineer and architect for the AI system.
Helpfulness, as scored by the judge, rose 20 percentage points.
- Anthropic Claude
- Bedrock
- OpenSearch
- Python
- TypeScript
- AWS CDK
- Lambda
- ECS
What I built
- Retrieval and generation on Anthropic Claude with OpenSearch, serving tens of thousands of people a week, then a multi-agent refactor measured with controlled experiments on cost, quality, and latency.
- A custom embedding model and reranker for semantic retrieval, which won 65 percent of head-to-head comparisons against the baseline.
- An LLM-as-a-judge evaluation method, plus benchmarking infrastructure for on-demand evals, continuous monitoring, and AI checks in CI.