Customer-support agent with knowledge base and handoff

Architecture and data flow

Organisation size

20–2000 employees

Data classes

personal, confidential

Constraints

documented answers already exist, or someone will write them; a helpdesk or ticketing system to hand over to; a support lead who owns what the assistant is allowed to answer; a 48 GB GPU for a 32B model, or a 24 GB card for a narrower scope

Industries

Retail, Ecommerce, Technology, Telecommunications, Professional services

Jurisdictions

any

• Cloud GPU instance — 1 × NVIDIA L4 24 GB

  GPU

  NVIDIA L4 24 GB (AWS G6, Azure NVadsA10/NCads equivalents, GCP G2)

  VRAM

  24 GB

  System RAM

  64 GB

  Storage

  1000 GB

  CPU

  8–16 vCPU

  Form factor

  Cloud instance

  Indicative costUS$1 – US$2

  indicative on-demand rental, USD per hour, Aug 2026 — verify against the provider price list for your region

  The private-cloud counterpart of `onprem-small-24gb`: same model sizes, no capital outlay, and a region you choose explicitly. Running it continuously for a year usually costs more than buying the equivalent box, so it suits pilots, bursts and firms without a server room. The cloud provider becomes a data processor — a DPA and a documented region are required.

• On-premise single 48 GB GPU server

  GPU

  NVIDIA L40S 48 GB (rack) or NVIDIA RTX 6000 Ada 48 GB (office workstation)

  VRAM

  48 GB

  System RAM

  128 GB

  Storage

  4000 GB

  CPU

  24–32-core x86 server CPU (AMD EPYC or Intel Xeon Scalable)

  Form factor

  Rack server

  Indicative costUS$12,000 – US$22,000

  indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier

  The step up when answer quality matters more than price: a 32B model at 4-bit with generous KV cache, or a 14B model at fp16 with a long context. ECC memory and a standard 2U chassis mean it belongs in a rack or comms room rather than under a desk.

API integrationapi-integration

development

Change managementchange-management

operations

Dockerdocker

infrastructure

Linuxlinux

operations

LLM evaluationllm-evaluation

ml

LLM inferencellm-inference

ml

Prompt engineeringprompt-engineering

ml

Retrieval-augmented generationrag

ml

Vector databasesvector-databases

data

Workflow automationworkflow-automation

operations

Commands are copied from each project’s own documentation, and the page they came from is linked under the step. 0 of 7 steps currently open an evidence record. The rest are linked to their source; §10 says which of those documents were fetched and which were fetched without their anchor being found — two different states, named differently there.

1. 01

   Write the list of questions it is allowed to answer

   Assessment

   Take the last three months of tickets and rank the topics. Pick the top ten that have a documented, stable answer. Everything else is out of scope on day one and goes straight to a person. A narrow assistant that is right is worth more than a broad one that is plausible.

2. 02

   Deploy Dify with Docker Composeversion-sensitive

   Assessment

   Clone the repository, copy the example environment file and start the stack. Change the default credentials before it is reachable by anyone else, and put it behind your reverse proxy rather than exposing the compose ports directly.

   cd dify/docker

   cp .env.example .env

   docker compose up -d

   docker compose ps

   check every service is healthy

   Source documentation

3. 03

   Point Dify at a local modelversion-sensitive

   Assessment

   Run vLLM as an OpenAI-compatible endpoint and register it in Dify as a model provider. Keep the API key out of the compose file and in your secret store; support traffic often means the endpoint is the busiest service you run.

   docker run --runtime nvidia --gpus all \
       -v ~/.cache/huggingface:/root/.cache/huggingface \
       --env "HF_TOKEN=$HF_TOKEN" \
       -p 8000:8000 \
       --ipc=host \
       vllm/vllm-openai:latest \
       --model Qwen/Qwen3-0.6B

   from the vLLM docs — substitute the model you sized for

   Source documentation

4. 04

   Load the knowledge base and make it citable

   Assessment

   Load only the documented answers you chose, tagged by product and language. Configure the assistant to show which article an answer came from — support agents will not trust it otherwise, and neither should customers.

   CREATE EXTENSION vector;

   if you are using pgvector rather than Qdrant

   Source documentation

5. 05

   Design the refusal and the handoff first, not last

   Assessment

   The assistant must say "I do not have a documented answer for that — connecting you to the team" whenever retrieval is weak, the topic is off-list, or the customer asks twice. The handoff carries the transcript so nobody repeats themselves. Test this path more than the happy path: it is the one that protects the brand.

   docker volume create n8n_data
   
   docker run -it --rm \
    --name n8n \
    -p 5678:5678 \
    -e GENERIC_TIMEZONE="<YOUR_TIMEZONE>" \
    -e TZ="<YOUR_TIMEZONE>" \
    -e N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true \
    -e N8N_RUNNERS_ENABLED=true \
    -v n8n_data:/home/node/.n8n \
    docker.n8n.io/n8nio/n8n

   from the n8n docs

   Source documentation

6. 06

   Run it internally before any customer sees it

   Assessment

   Put it in front of the support team as a suggestion tool for a month. They will find the wrong answers faster than any evaluation set, and the corrections improve the knowledge base rather than the prompt.

   Source documentation

7. 07

   Measure containment and escalation quality

   Assessment

   Track the share of conversations resolved without a person, and — more importantly — the share of escalations that arrived with useful context. A high containment rate with angry escalations is a worse outcome than a lower one with clean handoffs.

Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.

• A support-desk vendor with a built-in AI agent

  If your helpdesk already offers one, it arrives integrated with tickets, routing and reporting, which is most of the work in this recipe.

  • — Integrated on day one
  • — Customer conversations go to a processor; check DPA, retention and region
  • — Priced per resolution or per seat, so success costs more

  intercom-fin

• Onyx or a RAG stack without the agent layer

  For an internal helpdesk, plain retrieval over the policy documents is often enough and is far quicker to stand up.

  • — Much simpler
  • — No workflows, no handoff logic, no customer-facing channel

  onyxopen-webui

• Self-hostedPrivate company knowledge base (self-hosted RAG)Open WebUI as the employee interface, vLLM serving a Qwen2.5-14B-Instruct model on a single 24 GB GPU, PostgreSQL with pgvector for chats and embeddings, and OIDC single sign-on — all in Docker on one server in your office or colocation rack. Ollama replaces vLLM for teams under about 20 users; a 48 GB GPU lets you run a 32B model for better answers.

Not yet in the ledger

1. Dify — Deploy with Docker Compose

   Docs

   not yet fetched

2. vLLM — Using Docker

   Docs

   not yet fetched

3. vLLM — Quickstart (online serving)

   Docs

   not yet fetched

4. n8n — Install with Docker

   Docs

   not yet fetched

5. pgvector — README (installation and getting started)

   Readme

   not yet fetched

6. Qdrant — Quickstart (local deployment)

   Docs

   not yet fetched

7. Open WebUI — Quick Start (Docker)

   Docs

   not yet fetched

Deployed this stack, or hit something this page does not cover? Corrections, sources and implementation reports are what keep a recipe worth reading.

If you would rather not build it, we can introduce a forward-deployed engineer who has deployed this stack before. The enquiry form starts from this recipe.