This review is built from the project’s documentation and release notes plus third-party reviews rather than a formal test of our own. Versions, prices and dates are attributed in the text.
AnythingLLM is an all-in-one app for chatting with your documents using local or cloud language models. You install it, drop in PDFs, Word files or web pages, pick a model, and start asking questions grounded in your own material. It also does agents, web search and much more, but document chat is why most people install it.
The honest verdict: for an individual or a small team who wants working document chat in an afternoon, it is the easiest serious option available, and the core is free and MIT licensed. The AIFOSS review put it well: if you want a working RAG setup in an afternoon rather than a week, this gets you there. The trade-offs are breadth that makes the app feel busy, team features that need the Docker version, and answer quality that depends on tuning you will eventually need to do yourself.
Table of Contents
- What AnythingLLM Does
- AnythingLLM in Late 2026
- Desktop vs Docker
- AnythingLLM Document Chat
- Getting Better Answers
- Models and Providers
- The New llama.cpp Engine
- AnythingLLM Agents and Tools
- AnythingLLM Pricing
- Privacy and Data Handling
- Use Cases That Work Well
- Maintaining an AnythingLLM Knowledge Base
- Security for Team Deployments
- AnythingLLM vs Open WebUI, LibreChat and RAGFlow
- Where It Falls Short
- Hardware You Need
- Who Should Use What
- FAQ
- Getting Started With AnythingLLM
- Verdict
- Sources and Further Reading

What AnythingLLM Does
Built by Mintplex Labs, the app takes documents, URLs or text and turns them into a chat-ready knowledge base, as Stork described it. Uploaded files, including PDFs, Word documents, web pages and audio transcripts, are split into chunks, converted into vector embeddings and stored, so the model can find the relevant passages when you ask a question.
Pasquale Pillitteri’s review summarised the scope: retrieval-augmented generation, AI agents and more than 30 model providers in one desktop or Docker app. You can run everything locally for privacy, or connect cloud models when you need more capability.
The organising idea is the workspace. Each workspace has its own documents, settings and chat history, which keeps projects separate: one for contracts, one for research papers, one for internal policies.
AnythingLLM in Late 2026
The project moves fast. The AIFOSS review covered v1.12.1 from April 22, 2026. By September, Pillitteri reported v1.16.1 from August 27 as the stable release, with GitHub stars past 65,000.
Two changes in the second half of the year stand out. An optional Pro subscription for the desktop app arrived, removing daily limits on its Magic features while keeping a free tier for every feature. And in October, the team announced that the built-in model engine would move to llama.cpp managed by Lemonade, as MyClaw reported, while connecting a separate Ollama installation remains supported.
Desktop vs Docker
There are two main ways to run it, and the choice matters more than it first appears.
The desktop app is the fastest start. Install it on Windows, macOS or Linux, choose a model, and you are chatting with documents within minutes. AI Coolies called the desktop onboarding its most impressive feature. It is designed for one person on one machine.
The Docker version runs as a server. Vellum’s comparison noted that multi-user access and the embeddable chat widget are Docker-only features that the desktop app does not include. If more than one person needs access, or you want to embed a chat box on an internal site, plan on Docker from the start.
| Desktop | Docker | |
|---|---|---|
| Setup | Minutes | An hour or two |
| Users | One | Many, with roles |
| Embeddable widget | No | Yes |
| Best for | Personal document chat | Teams and internal tools |
AnythingLLM Document Chat
Document chat is the core feature, and it works well out of the box. Upload files to a workspace, and the app chunks and embeds them with a default embedding model. Ask a question, and it retrieves relevant passages and passes them to the model with your question. Answers can cite which documents they drew on.
Stork noted the main benefit: grounding responses in your own data reduces hallucinations. For contracts, manuals, research papers and internal policies, that grounding is the difference between a useful assistant and a confident guesser.
It also supports pinning whole documents into context, which bypasses retrieval for short, important files such as a style guide or a project brief.
Getting Better Answers
Defaults get you started; tuning gets you good answers. Stork made the point that performance scales with your own hardware and operations. The settings that matter most are chunk size and overlap, the embedding model, the number of passages retrieved, and the similarity threshold.
Long, structured documents usually benefit from larger chunks with some overlap. Short, varied documents benefit from smaller ones. A stronger embedding model often improves retrieval more than a bigger chat model. Our guide to RAG chunking strategies explains the trade-offs in detail, and fine-tuning vs RAG covers when retrieval is the right approach at all.
Test with real questions. Write ten questions you know the answers to, check what the app retrieves for each, and adjust settings until the right passages come back.
Models and Providers
The app supports more than 30 providers, from local runners to every major cloud API. Locally, you can use its built-in engine, Ollama, LM Studio or any OpenAI-compatible server. In the cloud, you can bring keys for OpenAI, Anthropic, Google and others.
Each workspace can use a different model, which is handy: a fast local model for casual questions, a cloud model for a workspace that needs stronger reasoning. Our LM Studio vs Ollama comparison helps you pick a local runner.
The New llama.cpp Engine
The October 2026 move to llama.cpp managed by Lemonade is a meaningful change for desktop users. Lemonade is AMD’s local AI server, and llama.cpp is the most widely used inference engine for local models. Together, they should give broader hardware support, including AMD GPUs and NPUs, and faster updates as llama.cpp improves.
For most users the change is invisible: models still load and run from the app. If you prefer to manage models yourself, connecting a separate Ollama installation is still supported.
AnythingLLM Agents and Tools
Beyond chat, the app includes agents that can browse the web, search documents, write files, summarise and call tools. Agent skills can be added from a community hub or written yourself, and MCP servers can be connected for more capabilities.
Agents work best with capable models. Small local models often struggle to use tools reliably, so if agents are important to you, test with your intended model before relying on it. For many users, agents are a useful extra rather than the main reason to choose the app.
Stork also highlighted extras such as a Meeting Assistant for on-device transcription and summarisation, plus desktop, cloud and mobile apps.
AnythingLLM Pricing
The core software is free. MyClaw’s pricing table, verified on November 4, 2026, set out the options.
| Option | Price | Who runs it |
|---|---|---|
| Desktop core | Free | You |
| Desktop Pro | Paid subscription | You |
| Self-hosted Docker | Free software | You |
| Cloud Basic | $50 a month | AnythingLLM |
| Cloud Pro | $99 a month | AnythingLLM |
| Enterprise | Custom quote | Depends on contract |
MyClaw noted that Cloud plans require your own model API keys, so model costs come on top. Desktop Pro removes daily limits on Magic features and adds watermark-free generated documents. For most self-hosters, the free tiers are enough.
Privacy and Data Handling
Wyse’s review described it as one of the few AI products that really means it when it says private. With a local model, local embeddings and the built-in vector database, nothing leaves your machine.
The privacy picture changes the moment you connect a cloud model or embedding provider. Document passages are sent to that provider with every question. Check which providers each workspace uses, and keep sensitive workspaces on local models only.
Use Cases That Work Well
Some uses suit the app especially well. A solo consultant can keep a workspace per client, with contracts, briefs and meeting notes, and ask questions across them without uploading anything to a cloud service. A researcher can load a folder of papers and ask for summaries, comparisons and the source passage behind each claim. A small support team can load product manuals and internal FAQs, then embed a chat widget on an intranet page.
Students and writers use it to interrogate long reading lists. Developers use it to chat with documentation for a library or an internal API. In each case, the pattern is the same: a bounded set of documents, questions that the documents can answer, and a need to keep the material private.
Uses that work less well are open-ended research across the web, tasks that require reasoning over hundreds of documents at once, and anything where the answer depends on tables or charts buried in scanned PDFs.
Maintaining an AnythingLLM Knowledge Base
A knowledge base is only as good as its freshest document. Policies change, manuals get new versions and old drafts linger. If outdated files stay in a workspace, the model will happily quote them.
Build a simple routine. Keep source documents in one folder per workspace, remove superseded versions when new ones arrive, and re-embed after significant changes. Name files clearly so citations make sense to readers. For team deployments, assign an owner to each workspace who is responsible for keeping it current.
Check retrieval quality periodically with the same set of test questions you used at setup. If answers drift, the cause is usually stale or duplicated documents rather than the model.
Security for Team Deployments
A shared document assistant concentrates sensitive information in one place. Run the Docker version behind HTTPS, use individual accounts with appropriate roles, and restrict each workspace to the people who should see its documents.
Review agent skills and MCP connections before enabling them, since they can read files, browse the web or call external services. Keep the server updated, and back up its storage regularly. If you embed the chat widget on an internal site, make sure that site is not reachable from the public internet. Our LLMjacking guide explains why exposed AI endpoints attract abuse.
AnythingLLM vs Open WebUI, LibreChat and RAGFlow
| Tool | Strength | Best for |
|---|---|---|
| AnythingLLM | Simplest document chat, desktop app | Individuals and small teams |
| Open WebUI | Polished chat for local models, many features | Ollama users and teams |
| LibreChat | Many providers, MCP, MIT licence | Organisations using several APIs |
| RAGFlow | Deep document parsing | Complex documents and enterprise RAG |
If document chat is your main goal and simplicity matters, this app is the best start. If you need deeper parsing of complex PDFs with tables and layouts, RAGFlow is stronger. If you want a general chat interface where documents are one feature among many, Open WebUI or LibreChat may fit better.
Where It Falls Short
Wyse identified the main weakness: the breadth that makes it useful also makes it busy. Desktop, cloud, mobile, self-hosted, browser extension, agents, skills and community assets create options, and with them decision overhead.
Team features need Docker, which adds setup work that the friendly desktop experience hides. Retrieval quality on complex documents, such as scanned PDFs, tables and multi-column layouts, depends on parsing that dedicated tools handle better. And as with any fast-moving project, settings and features change between releases.
Moving From Desktop to a Server
Many people start on the desktop app and later want to share their workspaces. Plan for that early. Keep your source documents organised in folders outside the app, so you can re-import them into a Docker deployment rather than relying on exporting the app’s internal storage.
Note the settings that worked for each workspace, including chunk size, embedding model and retrieval count, and recreate them on the server. Test the same questions after the move to confirm nothing changed. The migration is straightforward when documents and settings are written down, and painful when they only exist inside one person’s laptop.
Finally, document who owns each workspace and what it contains. A short note at the top of each workspace description saves confusion months later, when nobody remembers why a folder of old policies is still there.
Hardware You Need
The app itself is light. The model is what needs hardware. For document chat with a local model, an 8B to 14B model works well on a GPU with 12 to 16GB of VRAM, or a Mac with 16GB or more of unified memory.
Embedding large document collections takes time on the first import, especially on a CPU. A modest GPU speeds it up considerably. Our best GPU for local AI guide and RX 9060 XT review cover affordable options.
Who Should Use What
| If you are | Start with | Why | Link |
|---|---|---|---|
| Wanting private document chat on your own computer | AnythingLLM Desktop | Working in minutes, free | Download |
| A small team sharing knowledge bases | AnythingLLM Docker | Multi-user, embeddable widget | Docs |
| Wanting a general chat interface for local models | Open WebUI | Broader chat features | Get it |
| Handling complex enterprise documents | RAGFlow | Deeper document parsing | Get it |
Frequently Asked Questions About AnythingLLM
Is AnythingLLM free?
The desktop core and self-hosted Docker version are free. Desktop Pro is an optional subscription, and Cloud plans cost $50 or $99 a month.
Does it work fully offline?
Yes, with a local model, local embeddings and the built-in vector database.
Can several people use it?
Yes, with the Docker version. The desktop app is single-user.
Which models does it support?
More than 30 providers, including its built-in engine, Ollama, LM Studio and major cloud APIs.
How do I improve answer quality?
Tune chunk size, the embedding model and retrieval settings, and test with questions you know the answers to.
Is it better than Open WebUI?
For simple document chat, it is easier. For a general chat interface, Open WebUI offers more.
Getting Started With AnythingLLM
Download the desktop app, choose the built-in engine or connect Ollama, and pick a model suited to your hardware. Create a workspace and upload a handful of documents.
Ask questions you already know the answers to, and check the citations. Adjust chunking and retrieval settings if the wrong passages come back. Once it works for you, decide whether a team needs it, and move to Docker if so.
Keep sensitive workspaces on local models only.
Verdict
Use AnythingLLM if you want private document chat working quickly, on your own computer or for a small team. It is free at its core, supports almost every model provider, and gets non-experts to useful results faster than any comparable tool.
The honest caveat is that simplicity has limits. Team features need Docker, complex documents need more careful parsing than it provides, and good answers still require tuning. The busy feature set can also overwhelm new users.
Next step: install the desktop app, load ten real documents and test ten real questions. If the answers are good, you are done. If not, tune retrieval, and consider RAGFlow for complex files.
Sources and Further Reading
- Pasquale Pillitteri: AnythingLLM review, for features, versions and Pro.
- MyClaw: features and pricing, for the pricing table and engine change.
- AIFOSS: AnythingLLM review, for a hands-on look at desktop and Docker.
- Vellum: AnythingLLM alternatives, for Docker-only features and comparisons.
- Wyse: AnythingLLM review, for privacy and usability.
On this site: compare interfaces in our Open WebUI and LibreChat reviews, improve retrieval with RAG chunking strategies, and choose storage in vector databases for RAG.




