Embeddable AI Chatbot on AWS Bedrock with Feedback-Driven Prompt Versioning
A production, embeddable RAG assistant with a feedback loop and prompt-versioning system built from scratch — streaming responses over WebSocket for low-latency, chat-speed answers.
Sage is a customer/product-facing AI assistant designed to be dropped into any of the client's own properties: upload PDFs and other source documents to an AWS Bedrock Knowledge Base, and Sage answers questions grounded in that content, wherever it's embedded. Because it's customer-facing, Sage needed a feedback loop so the client could see which answers were landing, and a way to iterate on prompts without a code deploy every time.
A customer-facing assistant needs more than a working chatbot.
The client needed answers grounded in its own documents, a chatbot embeddable into any property as a reusable component, feedback captured on every answer so quality could be measured instead of assumed, and prompt iteration without a full backend redeploy for every wording change — all while feeling fast enough that it never reads as broken.
- Answers grounded in the client's own documents, not generic model knowledge
- Embeddable into any of the client's products or pages as a reusable component
- Feedback capture (thumbs up/down + free-text comment) on every answer
- Prompt iteration without a full backend redeploy for every change
How we built it.
AWS Bedrock Knowledge Base + Lambda
Source documents are uploaded to a Bedrock Knowledge Base, which handles chunking, embedding, and retrieval; Python Lambda functions orchestrate each conversation turn.
API Gateway WebSocket API
Rather than request/response HTTP polling, Sage communicates over a WebSocket connection through API Gateway, streaming responses back to the React front-end as they're generated.
Feedback and comment capture as first-class data
Every Sage response is paired with a thumbs up/down control and an optional free-text comment field, written to a dedicated feedback store as part of the core conversation schema.
Prompt versioning as data, not code
System prompts are stored as versioned records rather than hardcoded into Lambda functions, so feedback can be attributed to a specific prompt version.
The solution.
Sage — an embeddable, document-grounded assistant
A React chat widget that can be dropped into any of the client's properties, backed by a Bedrock Knowledge Base populated with the client's own documents.
Real-time streaming over WebSocket
Sage communicates over an API Gateway WebSocket connection, streaming responses to the front-end as they're generated instead of waiting for a full response.
Feedback capture on every response
A thumbs up/down control plus an optional comment on each answer, written to a dedicated feedback store.
Prompt versioning tied to feedback
Each response is tagged to the prompt version that produced it, so the team can see which prompt version is generating better-rated answers and roll changes forward or back without a redeploy.
Outcomes that shipped.
Sage could be embedded into any client-owned property as a drop-in assistant, grounded in whatever documents the client uploaded. The feedback and prompt-versioning loop gave the client a concrete way to see which answers — and which prompt versions — were actually performing well, rather than guessing, and responses streamed back fast enough that the assistant read as responsive rather than "thinking," even on multi-paragraph answers. Validated a build-vs-buy playbook: build custom on Bedrock/Lambda when a use case needs fine control over retrieval, feedback, and prompt iteration.
What we took with us.
- 01Build an automated prompt-version scorecard earlier — early prompt-version comparisons were done by manually reviewing feedback scores.
- 02Tighten WebSocket reconnect handling based on early production usage — real client network conditions surfaced reconnect edge cases not fully visible in initial testing.
Want something like this?
The tiers below are how we'd quote a build in the same space.
Have a build that needs this level of thinking?
Free 30-min audit. We'll tell you what to build, what it'd cost, and whether we're the right shop.