Skip to content
Is there an AI for this?

Self-hosted

Document classification and OCR pipeline

Docling (or Unstructured) converts and OCRs, embeddings plus rules classify, a local model reads only the documents that need reading, and Paperless-ngx gives the business a place to search and file. Starts CPU-only; a GPU is added when the volume justifies it.

Save to your workspace
Source
Editorial recipe — no step evidence has been fetched yet
Verified
Evidence not verified
Confidence
Low

01Objective

Convert a backlog of scanned and mixed-format documents into searchable text, sort them into the right categories, and pull out the few fields each category needs.

02Recommended stack

6 components
RoleComponent
InferenceBGE-M3 embeddings
InferenceoptionalQwen2.5-14B-Instruct on vLLM, for the hard cases only
IngestionDocling
IngestionoptionalUnstructured (open source)
UiPaperless-ngx
Vector storePostgreSQL with pgvector

Architecture and data flow

Architecture for Document classification and OCR pipeline10 components in 6 layers. Trust boundaries: COMPANY NETWORK. External data transfer: NONE. No data leaves the boundary drawn here.Consume folder / mail intake → Docling OCRPaperless-ngx (search, tags, correction)Classification (rules → embeddings → model)PostgreSQL + pgvector (document embeddings)PostgreSQL (documents, categories, fields)Embeddings on CPU; vLLM on GPU for hard casesBGE-M3 · Qwen2.5-14B-InstructCPU server, GPU added when volume requires itRecords and admin staffPEOPLEConsume folder / mail intake → Docling OCRConsume folder / mail i…STORAGEPaperless-ngx (search, tags, correction)Paperless-ngx (search, …APPLICATIONClassification (rules → embeddings → model)Classification (rules →…RETRIEVALPostgreSQL + pgvector (document embeddings)PostgreSQL + pgvector (…VECTOR STOREPostgreSQL (documents, categories, fields)PostgreSQL (documents, …DATABASEOriginal scans and filesSTORAGEEmbeddings on CPU; vLLM on GPU for hard casesEmbeddings on CPU; vLLM…INFERENCE SERVERBGE-M3 · Qwen2.5-14B-InstructBGE-M3 · Qwen2.5-14B-In…MODELCPU server, GPU added when volume requires itCPU server, GPU added w…HARDWARECOMPANY NETWORKHTTPSCONFIDENTIALdocuments to indexCONFIDENTIALquestion + user groupsCONFIDENTIALembedding searchCONFIDENTIALdocuments + permissionsCONFIDENTIALchats, users, settingsPERSONALoriginal filesCONFIDENTIALprompt + retrieved passagesCONFIDENTIALloaded weightsGPU memoryGPU memoryEXTERNAL DATA TRANSFER · NONE

Components

  • Records and admin staff — people
  • Consume folder / mail intake → Docling OCR — storage
  • Paperless-ngx (search, tags, correction) — application
  • Classification (rules → embeddings → model) — retrieval
  • PostgreSQL + pgvector (document embeddings) — vector store
  • PostgreSQL (documents, categories, fields) — database
  • Original scans and files — storage
  • Embeddings on CPU; vLLM on GPU for hard cases — inference server
  • BGE-M3 · Qwen2.5-14B-Instruct — model
  • CPU server, GPU added when volume requires it — hardware

Connections

  • Records and admin staff to Paperless-ngx (search, tags, correction) — HTTPS (confidential data)
  • Consume folder / mail intake → Docling OCR to Classification (rules → embeddings → model) — documents to index (confidential data)
  • Paperless-ngx (search, tags, correction) to Classification (rules → embeddings → model) — question + user groups (confidential data)
  • Classification (rules → embeddings → model) to PostgreSQL + pgvector (document embeddings) — embedding search (confidential data)
  • Classification (rules → embeddings → model) to PostgreSQL (documents, categories, fields) — documents + permissions (confidential data)
  • Paperless-ngx (search, tags, correction) to PostgreSQL (documents, categories, fields) — chats, users, settings (personal data)
  • Classification (rules → embeddings → model) to Original scans and files — original files (confidential data)
  • Classification (rules → embeddings → model) to Embeddings on CPU; vLLM on GPU for hard cases — prompt + retrieved passages (confidential data)
  • Embeddings on CPU; vLLM on GPU for hard cases to BGE-M3 · Qwen2.5-14B-Instruct — loaded weights
  • BGE-M3 · Qwen2.5-14B-Instruct to CPU server, GPU added when volume requires it — GPU memory
  • Embeddings on CPU; vLLM on GPU for hard cases to CPU server, GPU added when volume requires it — GPU memory

External data transfer · NONE

  • No edge in this design crosses out of the company network.
  • Documents are converted, classified and stored entirely on your own machines.
  • The first phase runs without a GPU at all, which is what makes it a cheap way to prove the categories before spending on hardware.

03Suitable for

Organisation size
5–1000 employees
Data classes
confidential, personal
Constraints
a real backlog: filing cabinets, a shared drive of scans, or a mail intake; agreed categories — if the business cannot name them, neither can the pipeline; someone to check a sample of the classifications each week at first
Industries
Legal, Accounting, Insurance, Healthcare, Public sector, Logistics
Jurisdictions
any

04Hardware

  • On-premise single 24 GB GPU server

    GPU
    NVIDIA RTX 4090 24 GB (or NVIDIA L4 24 GB for a rack-mounted, 72 W alternative)
    VRAM
    24 GB
    System RAM
    64 GB
    Storage
    2000 GB
    CPU
    16-core x86 server CPU (AMD EPYC 7003/9004 or Intel Xeon Scalable)
    Form factor
    Tower server

    Indicative costUS$4,000 – US$9,000

    indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier

    The default box for a 20–60 person firm. Fits a 14B model at 4-bit with roughly 8 GB of KV cache left for concurrent chat, or an 8B model at fp16. NVMe storage sized for the model cache plus a document corpus and its embeddings. Add a UPS and an offsite backup target — this machine holds the whole knowledge base.

  • CPU-only server (small models and embeddings)

    GPU
    unknown
    VRAM
    unknown
    System RAM
    64 GB
    Storage
    1000 GB
    CPU
    16–32 core x86 server CPU with AVX-512
    Form factor
    Tower server

    Indicative costUS$1,500 – US$4,000

    indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier

    No GPU. Embedding models and document parsing run acceptably here, which is enough for a search-only pilot or a nightly batch pipeline. Chat generation with a 7–8B model at 4-bit works but reads at a few tokens per second — usable for one person testing, not for a team. The honest use of this profile is to prove the retrieval quality before buying a GPU.

Indicative costUSD · one-off plus monthly

Software licences
All components are open source and self-installed. Check each licence for use limits.
US$0
Model usage
Open weights served locally: no per-token charge. The GPU cost sits under infrastructure.
US$0
Server: CPU-only to start, GPU when volume requires
Spans cpu-only-small-models to onprem-small-24gb — indicative build cost, Aug 2026, verify locally. Storage sizing depends entirely on the size of the backlog.
US$1,500 – US$9,000
Implementation (6–15 FDE-days)
6–15 FDE-days at US$760–1940 per day, converted from the HK$6,000–15,000 band at the HKMA Linked Exchange Rate band of HK$7.75–7.85 to one US dollar. One-off; excludes internal staff time.
US$4,560 – US$29,100
  • One intake channel, 5–15 categories, a backlog in the tens of thousands of pages.
  • Excludes scanning the paper, which is usually outsourced and priced per page.
  • Every figure is an assessment from the inputs listed, not a quotation.

05Difficulty

3 / 5

A project week with Linux, Docker and GPU experience

06Skills

Data engineeringdata-engineering
data
Dockerdocker
infrastructure
Linuxlinux
operations
LLM evaluationllm-evaluation
ml
OCR and document parsingocr
ml
PostgreSQLpostgresql
data
Pythonpython
development
Vector databasesvector-databases
data

07Deployment steps

6 steps

Commands are copied from each project’s own documentation, and the page they came from is linked under the step. 0 of 6 steps currently open an evidence record. The rest are linked to their source; §10 says which of those documents were fetched and which were fetched without their anchor being found — two different states, named differently there.

  1. 01

    Agree the categories and count them

    Assessment

    Sample 200 documents and sort them by hand into the categories the business names. Two things fall out: whether the categories are real, and how imbalanced they are. A category with three examples will not classify reliably however good the model is.

  2. 02

    Convert and OCRversion-sensitive

    Assessment

    Install Docling and convert the sample, including the scans. Check reading order on multi-column pages and table structure on anything with figures. Where the inputs are email and office formats rather than scans, Unstructured is the better fit.

    pip install docling
    docling https://arxiv.org/pdf/2206.01062

    from the Docling README

    Source documentation

  3. 03

    Classify with rules and embeddings before reaching for a model

    Assessment

    Many documents are identified by a form number, a sender or a phrase in the header — those are rules, and rules are free and explainable. Embed the rest and classify by nearest neighbour against your labelled sample. Measure this baseline: it is often good enough, and it is always the thing the model has to beat.

    CREATE EXTENSION vector;

    Source documentation

  4. 04

    Add a model for the residue and for field extractionversion-sensitive

    Assessment

    Send only the documents the baseline could not place, and ask for a category plus a confidence against a fixed schema. Use the same constrained-output approach for the two or three fields each category needs — dates, references, amounts — rather than free text.

    docker run --runtime nvidia --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=$HF_TOKEN" \
    -p 8000:8000 \
    --ipc=host \
    vllm/vllm-openai:latest \
    --model Qwen/Qwen3-0.6B

    from the vLLM docs — substitute the model you sized for

    Source documentation

  5. 05

    File the results into Paperless-ngxversion-sensitive

    Assessment

    Paperless-ngx gives you a consume folder, tagging, correspondents and search over the archive. The docs are direct about the trade-off: "For most users, Docker is the best option." Write the category as a tag and the extracted fields as custom fields, so the pipeline's output is visible and correctable by the people who own the documents.

    Source documentation

  6. 06

    Sample-check weekly and feed corrections back

    Assessment

    Twenty documents a week, checked by the person who owns the filing. Corrections become new labelled examples, which improves the embedding baseline directly. Stop when the error rate is boring rather than when it is zero.

08Compliance considerations

Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.

Applies everywhere

  • Personal data · Sensitive datahigh

    Backlogs contain more personal data than anyone expects, including special-category data in HR and medical files. Making it searchable changes the risk profile: what was practically obscure becomes instantly findable.

  • Retentionhigh

    Digitising is the moment to apply retention rules, not to postpone them. A pipeline that indexes documents which should have been destroyed has created a new problem.

  • Security · Confidentialityhigh

    The archive becomes a single searchable copy of the company's paper records. Access control on categories, encryption at rest and a tested backup are proportionate to that.

  • Auditability · Human oversightmedium

    Store the classification, the confidence and the method (rule, embedding or model) against each document, so a misfiling can be explained and corrected systematically.

09Alternatives

  • A commercial OCR and capture service

    Higher accuracy on difficult scans and no pipeline to build. Sensible where the documents are not sensitive and volume is spiky.

    • — Accurate immediately
    • — Documents go to a processor — DPA, region and retention questions
    • — Per-page pricing on a backlog can be substantial

    nanonetsrossum

  • Search without classification

    If the goal is "find the document", full-text and semantic search over converted text may remove the need to classify at all.

    • — Much less work
    • — No structured categories, so no reporting, routing or retention automation

    open-webuipgvector

10Evidence

0 of 7 fetched

Not yet in the ledger

  1. Docling — README (quickstart and conversion)

    Readme

    not yet fetched

  2. Docling — Usage

    Docs

    not yet fetched

  3. Unstructured — Open source quickstart

    Docs

    not yet fetched

  4. Paperless-ngx — Setup (installation)

    Docs

    not yet fetched

  5. pgvector — README (installation and getting started)

    Readme

    not yet fetched

  6. vLLM — Structured Outputs

    Docs

    not yet fetched

  7. vLLM — Using Docker

    Docs

    not yet fetched

11Community

Deployed this stack, or hit something this page does not cover? Corrections, sources and implementation reports are what keep a recipe worth reading.

Improve this page

Sign in to contribute

From the field

0 deployments · 0 questions

Nobody has reported deploying this here yet, and no question has been opened against this page. Both appear once a reviewer accepts them.

12Hire an FDE

If you would rather not build it, we can introduce a forward-deployed engineer who has deployed this stack before. The enquiry form starts from this recipe.

Hire an FDE for this stack