Alternatives
Self-hosted alternatives to Ollama
Projects in the registry that can run on your own hardware and serve at least one of the same use cases as Ollama.
- Source
- Registry relations over fetched repository records — the evidence sits on each repository page
- Verified
- Evidence not verified
- Confidence
- High
01How this list was built
- Compared against
- Ollama
- Delivery
- Model servers
- Shared use cases
- Local LLM, Private LLM, Private company ChatGPT, AI coding assistant
- Selection rule
- open source, or self-hostable, and shares a use case
- Candidates found
- 15
02Alternatives
AnythingLLM
Hybrid — hosted or self-hosted
Desktop and server application that turns a document set into a chat workspace, with per-workspace embeddings, multiple model back ends and a built-in vector store.
- Repository
- Mintplex-Labs/anything-llm
- Stars
- 65,673
- Licence
- MIT
- In stacks
- 0
Shares: Private company chatgpt · Local llm
authentik
Hybrid — hosted or self-hosted
Identity provider with OpenID Connect, SAML and proxy-based authentication, application-level policies and a forward-auth outpost for services that have no login of their own.
- Repository
- goauthentik/authentik
- Stars
- 25,644
- Licence
- Other
- In stacks
- 0
Shares: Private company chatgpt
Continue
Hybrid — hosted or self-hosted
Open-source IDE extension for VS Code and JetBrains that connects completion and chat to any model back end, including a local server, under a checked-in configuration file.
- Repository
- continuedev/continue
- Stars
- 35,622
- Licence
- Apache-2.0
- In stacks
- 1
Shares: Ai coding assistant · Local llm
Dify
Hybrid — hosted or self-hosted
Platform for building LLM applications: visual workflow editor, retrieval pipelines, agent tools, prompt management and an API layer. Available self-hosted or as a managed cloud service.
- Repository
- langgenius/dify
- Stars
- 153,448
- Licence
- Other
- In stacks
- 1
Shares: Private company chatgpt
Jan
Hybrid — hosted or self-hosted
Open-source desktop assistant that runs open-weight models locally and can also call remote endpoints. The open alternative to LM Studio for a single-machine deployment.
- Repository
- janhq/jan
- Stars
- 44,559
- Licence
- Other
- In stacks
- 0
Shares: Local llm · Private llm
Keycloak
Self-hosted projects
Identity and access management server providing OpenID Connect and SAML single sign-on, user federation, groups and roles for self-hosted applications.
- Repository
- keycloak/keycloak
- Stars
- 36,449
- Licence
- Apache-2.0
- In stacks
- 4
Shares: Private company chatgpt
Langfuse
Hybrid — hosted or self-hosted
Tracing, evaluation and prompt management for LLM applications, self-hostable so prompts and responses can stay inside the network being audited.
- Repository
- langfuse/langfuse
- Stars
- 34,243
- Licence
- Other
- In stacks
- 1
Shares: Private company chatgpt
LibreChat
Self-hosted projects
Self-hosted multi-model chat application with authentication, per-conversation model switching, plugins, file upload and an admin configuration file. Familiar interface for staff moving off consumer tools.
- Repository
- danny-avila/LibreChat
- Stars
- 42,850
- Licence
- MIT
- In stacks
- 0
Shares: Private company chatgpt · Private llm
LiteLLM
Cloud platforms
Gateway that presents one OpenAI-compatible API in front of many providers and local servers, with per-team keys, budgets, rate limits, fallbacks and request logging.
- Repository
- BerriAI/litellm
- Stars
- 57,216
- Licence
- Other
- In stacks
- 3
Shares: Private llm · Private company chatgpt · Local llm · Ai coding assistant
llama.cpp
Libraries
C++ inference engine for quantised models on CPU, Apple Silicon and GPUs, with a bundled HTTP server. The engine underneath many desktop runtimes and the GGUF quantisation format.
- Repository
- ggml-org/llama.cpp
- Stars
- 128,796
- Licence
- MIT
- In stacks
- 0
Shares: Local llm · Private llm
LMDeploy
Model servers
Serving toolkit from the InternLM team with its own inference engine and quantisation tooling, and strong coverage of Chinese open-weight model families.
- Repository
- InternLM/lmdeploy
- Stars
- 8,082
- Licence
- Apache-2.0
- In stacks
- 0
Shares: Local llm · Private llm
LM Studio
Hybrid — hosted or self-hosted
Desktop application for downloading and running open-weight models locally, with a chat interface and a local OpenAI-compatible server. Windows, macOS and Linux.
- Repository
- none linked
- Stars
- —
- Licence
- unknown
- In stacks
- 1
Shares: Local llm · Private llm
LocalAI
Model servers
Drop-in OpenAI-compatible API server that runs text, embedding, image and audio models locally across several back ends, including CPU-only deployments.
- Repository
- mudler/LocalAI
- Stars
- 49,166
- Licence
- MIT
- In stacks
- 0
Shares: Local llm · Private llm
NVIDIA NIM
Model servers
Prebuilt inference containers with an OpenAI-compatible API, deployable into your own cluster including air-gapped sites. Licensed through NVIDIA AI Enterprise rather than open source.
- Repository
- none linked
- Stars
- —
- Licence
- unknown
- In stacks
- 0
Shares: Local llm · Private llm · Private company chatgpt
NVIDIA TensorRT-LLM
Libraries
Compiles models into optimised engines for NVIDIA GPUs. The build step is the trade: peak throughput on a fixed configuration, in exchange for recompiling when it changes.
- Repository
- nvidia/tensorrt-llm
- Stars
- 14,663
- Licence
- Other
- In stacks
- 0
Shares: Local llm · Private llm
03Repository facts
| Repository | Stars | Health |
|---|---|---|
| Mintplex-Labs/anything-llmStop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience | 66k | Maintenance 60%, Releases 60%, Contributors 50%, Issues 95%, Community 96%71 |
| goauthentik/authentikThe authentication glue you need. | 26k | Maintenance 80%, Releases 80%, Contributors 50%, Issues 57%, Community 88%75 |
| continuedev/continueopen-source coding agent | 36k | Maintenance 60%, Releases 40%, Contributors 50%, Issues 74%, Community 91%64 |
| langgenius/difyBuild Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack. | 153k | Maintenance 60%, Releases 60%, Contributors 50%, Issues 94%, Community 100%72 |
| janhq/janJan is an open source alternative to ChatGPT that runs 100% offline on your computer. | 45k | Maintenance 80%, Releases 60%, Contributors 50%, Issues 88%, Community 93%76 |
| keycloak/keycloakOpen Source Identity and Access Management For Modern Applications and Services | 36k | Maintenance 60%, Releases 60%, Contributors 50%, Issues 14%, Community 91%62 |
| langfuse/langfuseOpen source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. YC W23 | 34k | Maintenance 60%, Releases 60%, Contributors 50%, Issues 73%, Community 91%68 |
| danny-avila/LibreChatEnhanced ChatGPT Clone: Features Agents, MCP, Skills, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active | 43k | Maintenance 60%, Releases 30%, Contributors 50%, Issues 83%, Community 93%63 |
| BerriAI/litellmThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] | 57k | Maintenance 60%, Releases 60%, Contributors 50%, Issues 14%, Community 95%63 |
| ggml-org/llama.cppLLM inference in C/C++ | 129k | Maintenance 80%, Releases 60%, Contributors 50%, Issues 81%, Community 100%77 |
| InternLM/lmdeployLMDeploy is a toolkit for compressing, deploying, and serving LLMs. | 8.1k | Maintenance 80%, Releases 60%, Contributors 50%, Issues 26%, Community 78%66 |
| mudler/LocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. | 49k | Maintenance 80%, Releases 80%, Contributors 50%, Issues 97%, Community 94%81 |
| nvidia/tensorrt-llmTensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way. | 15k | Maintenance 80%, Releases 40%, Contributors 50%, Issues 0%, Community 83%60 |
04Ask about your own case
Which alternative is right depends on how many people will use it, what the data is and what you can operate. Ask and the answer is researched for your organisation.