Overview
This project builds a domain-specific Q&A assistant backed by retrieval-augmented generation (RAG). The assistant answers questions about regulatory guidance for AI/ML in medical devices by retrieving relevant information from a curated knowledge base of FDA guidance documents and generating informed responses.
The implementation uses OpenAI's Agent Builder as the primary platform, providing a complete environment for agent workflow design, built-in vector storage for document ingestion and retrieval, and a front-end framework, letting the team focus on knowledge base curation, retrieval configuration, and prompt engineering rather than infrastructure.
The workflow was designed through a cognitive mapping exercise with a domain expert — a structured methodology for extracting tacit process knowledge and translating it into an implementable agent architecture. Each workflow component was mapped with explicit inputs, outputs, decision logic, validation criteria, and domain references before any code was written. This approach produced a multi-agent pipeline with structured state management, conversational data collection, confidence-checked classification, and automated report generation — demonstrating how complex domain expertise can be systematically captured and operationalized as an agentic workflow.
Key Concepts
- How RAG works: document ingestion, embedding, vector retrieval, and grounded generation
- Knowledge base curation: selecting, organizing, and preparing domain-specific source documents for ingestion
- Prompt engineering for Q&A accuracy: system instructions, retrieval tuning, and grounding techniques to minimize hallucination
- Cognitive mapping: a facilitated methodology for extracting expert workflow knowledge into structured, buildable specifications — documenting inputs, outputs, decision criteria, and validation rules for every process node
- Multi-agent orchestration: designing pipelines where specialized agents handle discrete tasks (data collection, classification, report generation) coordinated through state-driven routing and decision logic
- Agent deployment on a commercial platform: configuring and shipping an assistant using OpenAI Agent Builder
Deliverables
A deployed, functional Q&A assistant that answers questions about AI/ML regulatory frameworks for medical devices using a curated knowledge base. The assistant is built on a multi-agent workflow that collects product information through guided conversation, classifies products against FDA frameworks, and generates preliminary regulatory assessment reports. Demonstrates the ability to ship an AI-powered information tool end-to-end, domain expertise in navigating complex regulatory guidance, and a repeatable methodology for turning expert knowledge into production agentic workflows.
Applied Skills
- RAG architecture fundamentals (document ingestion, embedding, vector retrieval, grounded generation)
- Knowledge base curation and document preparation for domain-specific retrieval
- Prompt engineering for Q&A accuracy and hallucination reduction
- Cognitive mapping and expert knowledge extraction for agentic workflow design
- Multi-agent workflow architecture (state management, decision routing, validation gates, and scoped agent responsibilities)
- Agent deployment using a commercial platform (OpenAI Agent Builder)