Alternatives
Self-hosted alternatives to LocalAI
Projects in the registry that can run on your own hardware and serve at least one of the same use cases as LocalAI.
- Source
- Registry relations over fetched repository records — the evidence sits on each repository page
- Verified
- Evidence not verified
- Confidence
- High
01How this list was built
- Compared against
- LocalAI
- Delivery
- Model servers
- Shared use cases
- Local LLM, Private LLM, OCR, Meeting transcription
- Selection rule
- open source, or self-hostable, and shares a use case
- Candidates found
- 19
02Alternatives
AnythingLLM
Hybrid — hosted or self-hosted
Desktop and server application that turns a document set into a chat workspace, with per-workspace embeddings, multiple model back ends and a built-in vector store.
- Repository
- Mintplex-Labs/anything-llm
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Local llm
Continue
Hybrid — hosted or self-hosted
Open-source IDE extension for VS Code and JetBrains that connects completion and chat to any model back end, including a local server, under a checked-in configuration file.
- Repository
- continuedev/continue
- Stars
- 35,622
- Licence
- Apache-2.0
- In stacks
- 1
Shares: Local llm
Docling
Libraries
Document conversion toolkit that parses PDF, Office and image files into structured Markdown or JSON, preserving reading order, tables and figures for downstream retrieval.
- Repository
- docling-project/docling
- Stars
- 65,519
- Licence
- MIT
- In stacks
- 4
Shares: Ocr
faster-whisper
Libraries
Reimplementation of Whisper on CTranslate2 with substantially lower memory use and faster inference, including int8 execution on CPU. The usual engine behind self-hosted transcription.
- Repository
- SYSTRAN/faster-whisper
- Stars
- —
- Licence
- unknown
- In stacks
- 2
Shares: Meeting transcription
LibreChat
Self-hosted projects
Self-hosted multi-model chat application with authentication, per-conversation model switching, plugins, file upload and an admin configuration file. Familiar interface for staff moving off consumer tools.
- Repository
- danny-avila/LibreChat
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Private llm
LiteLLM
Cloud platforms
Gateway that presents one OpenAI-compatible API in front of many providers and local servers, with per-team keys, budgets, rate limits, fallbacks and request logging.
- Repository
- BerriAI/litellm
- Stars
- 57,216
- Licence
- Other
- In stacks
- 2
Shares: Private llm · Local llm
llama.cpp
Libraries
C++ inference engine for quantised models on CPU, Apple Silicon and GPUs, with a bundled HTTP server. The engine underneath many desktop runtimes and the GGUF quantisation format.
- Repository
- ggml-org/llama.cpp
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Local llm · Private llm
LM Studio
Hybrid — hosted or self-hosted
Desktop application for downloading and running open-weight models locally, with a chat interface and a local OpenAI-compatible server. Windows, macOS and Linux.
- Repository
- none linked
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Local llm · Private llm
Ollama
Model servers
Local model runtime with a one-command install, a model library and an OpenAI-compatible API. The usual starting point for running open-weight models on a workstation or small server.
- Repository
- ollama/ollama
- Stars
- 179,403
- Licence
- MIT
- In stacks
- 3
Shares: Local llm · Private llm
Open WebUI
Self-hosted projects
Self-hosted chat interface for local and hosted models, with user accounts, groups, document upload and built-in retrieval. Runs in Docker against Ollama, vLLM or any OpenAI-compatible endpoint.
- Repository
- open-webui/open-webui
- Stars
- 149,870
- Licence
- Other
- In stacks
- 4
Shares: Private llm · Local llm
PaddleOCR
Libraries
OCR toolkit with detection, recognition and layout models, including Chinese and other East Asian scripts, plus table and formula recognition. Runs offline on CPU or GPU.
- Repository
- PaddlePaddle/PaddleOCR
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Ocr
paperless-ngx
Self-hosted projects
Self-hosted document management system that OCRs incoming scans and email, applies tags and correspondents through trainable rules, and keeps a searchable archive.
- Repository
- paperless-ngx/paperless-ngx
- Stars
- 44,585
- Licence
- GPL-3.0
- In stacks
- 2
Shares: Ocr
Qdrant
Hybrid — hosted or self-hosted
Vector search engine written in Rust with payload filtering, hybrid search, quantisation and snapshots. Runs as a single container or a cluster, or as a managed cloud service.
- Repository
- qdrant/qdrant
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Private llm
SGLang
Model servers
Serving framework for large models with prefix caching and structured-output support, aimed at high-throughput deployments and multi-GPU serving.
- Repository
- sgl-project/sglang
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Local llm · Private llm
Tabby
Self-hosted projects
Self-hosted coding assistant with its own inference server, repository-aware context and IDE extensions. Runs entirely inside the network, including on a single GPU.
- Repository
- TabbyML/tabby
- Stars
- 33,834
- Licence
- Other
- In stacks
- 1
Shares: Local llm
Tesseract OCR
Libraries
Long-established open-source OCR engine with trained data for many languages and scripts, usable offline and embedded in most self-hosted document pipelines.
- Repository
- tesseract-ocr/tesseract
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Ocr
Unstructured
Hybrid — hosted or self-hosted
Library and hosted API that partition documents of many formats into typed elements for indexing, with connectors to common storage systems and vector databases.
- Repository
- Unstructured-IO/unstructured
- Stars
- —
- Licence
- unknown
- In stacks
- 1
Shares: Ocr
vLLM
Model servers
High-throughput GPU inference server using paged attention and continuous batching, with an OpenAI-compatible API. Built for many concurrent users rather than single-session use.
- Repository
- vllm-project/vllm
- Stars
- 89,956
- Licence
- Apache-2.0
- In stacks
- 7
Shares: Local llm · Private llm
WhisperX
Libraries
Whisper-based transcription pipeline adding word-level timestamps, forced alignment and speaker diarisation. Runs locally on GPU for batch transcription of recordings.
- Repository
- m-bain/whisperX
- Stars
- —
- Licence
- unknown
- In stacks
- 1
Shares: Meeting transcription
03Repository facts
| Repository | Stars | Health |
|---|---|---|
| Mintplex-Labs/anything-llm | — | Health not measuredunknown |
| continuedev/continueopen-source coding agent | 36k | Maintenance 100%, Releases 60%, Contributors 50%, Issues 74%, Community 91%80 |
| docling-project/doclingGet your documents ready for gen AI | 66k | Maintenance 100%, Releases 100%, Contributors 50%, Issues 85%, Community 96%90 |
| SYSTRAN/faster-whisper | — | Health not measuredunknown |
| danny-avila/LibreChat | — | Health not measuredunknown |
| BerriAI/litellmThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] | 57k | Maintenance 100%, Releases 100%, Contributors 50%, Issues 14%, Community 95%83 |
| ggml-org/llama.cpp | — | Health not measuredunknown |
| ollama/ollamaGet up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. | 179k | Maintenance 100%, Releases 100%, Contributors 50%, Issues 79%, Community 100%90 |
| open-webui/open-webuiUser-friendly AI Interface (Supports Ollama, OpenAI API, ...) | 150k | Maintenance 100%, Releases 80%, Contributors 50%, Issues 99%, Community 100%88 |
| PaddlePaddle/PaddleOCR | — | Health not measuredunknown |
| paperless-ngx/paperless-ngxA community-supported supercharged document management system: scan, index and archive all your documents | 45k | Maintenance 100%, Releases 80%, Contributors 50%, Issues 100%, Community 93%87 |
| qdrant/qdrant | — | Health not measuredunknown |
| sgl-project/sglang | — | Health not measuredunknown |
| TabbyML/tabbySelf-hosted AI coding assistant | 34k | Maintenance 60%, Releases 20%, Contributors 50%, Issues 90%, Community 91%61 |
| tesseract-ocr/tesseract | — | Health not measuredunknown |
| Unstructured-IO/unstructured | — | Health not measuredunknown |
| vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs | 90k | Maintenance 100%, Releases 80%, Contributors 50%, Issues 22%, Community 99%81 |
| m-bain/whisperX | — | Health not measuredunknown |
04Ask about your own case
Which alternative is right depends on how many people will use it, what the data is and what you can operate. Ask and the answer is researched for your organisation.