chromadb#

What it is#

chromadb is the Python distribution of Chroma, an open-source vector database for AI applications. The package ships the embedded in-process engine, the persistent SQLite/DuckDB-backed store, an HTTP client/server, and a small library of pluggable embedding functions in a single import. The same chromadb.Client API works whether you are running in-memory for a notebook prototype or pointing at a remote Chroma server cluster.

Reach for chromadb when you want zero-infrastructure RAG storage in Python and are happy to scale up later. Reach for qdrant-client, weaviate-client, or hosted services like Pinecone when you need strict multi-tenant isolation, advanced filtering, or production-grade clustering from day one.

Install#

pip install chromadb

Output: (none — exits 0 on success)

uv add chromadb

Output: dependency resolved + added to pyproject.toml

poetry add chromadb

Output: updated lockfile + virtualenv install

pip install chromadb-client      # thin HTTP-only client (no server, no embedding deps)

Output: installs the slim client that talks to a remote chroma run server

Versioning & Python support#

• Chroma releases are pre-1.0 and move quickly — the 0.4.x → 0.5.x jump in 2024 changed the on-disk persistent-store layout and required a migration script. Always read the changelog before bumping.
• Recent versions support Python 3.8+ on Linux, macOS, and Windows. Wheels are published for common architectures; building from source needs a C++ toolchain because of the HNSW index code.
• Client/server version skew matters. If you run the Chroma server in Docker, the chromadb (or chromadb-client) version in your application should track the server’s minor version. Cross-minor combinations sometimes work and sometimes fail with opaque protocol errors — pinning both to the same minor is the safe path.
• Roadmap targets a 1.0 once the storage format and tenant model stabilise; until then treat every minor as a potential breaking release in CI.

Package metadata#

• Maintainer: Chroma (the company, formerly Chroma Inc.) and community contributors
• Project home: github.com/chroma-core/chroma
• Docs: docs.trychroma.com
• PyPI: pypi.org/project/chromadb
• License: Apache-2.0
• Governance: company-led with open contributions; commercial Chroma Cloud offering tracks the open core
• First released: 2022
• Downloads: multiple million per month on PyPI; the default vector store in many LangChain and LlamaIndex tutorials

Optional dependencies & extras#

chromadb ships as one PyPI package that bundles the server, the local persistent store, and the in-process engine. There are no published feature extras in the usual chromadb[xxx] form — instead, optional functionality lives in companion packages and in the small built-in chromadb.utils.embedding_functions module, which loads its own extras lazily.

Common companions to install alongside:

• chromadb-client — slim HTTP-only client (no server, no embedding deps). Use it in lightweight containers that only talk to a remote chroma run server.
• sentence-transformers — local CPU/GPU embeddings via the built-in SentenceTransformerEmbeddingFunction.
• openai — required if you use the built-in OpenAIEmbeddingFunction.
• cohere, google-generativeai, voyageai — each backs the corresponding built-in embedding function.
• onnxruntime — used by the bundled default MiniLM embedding model.
• tiktoken — token counting when you mix Chroma with OpenAI chat models.
• langchain-chroma or llama-index-vector-stores-chroma — framework adapters in the LangChain / LlamaIndex ecosystems.

Alternatives#

Common gotchas#

1. 0.4.x → 0.5.x persistent-store migration. The on-disk format changed; existing chroma.sqlite3 stores need the maintainer-provided migration tool. Snapshot the directory before upgrading.
2. Collection-API churn. Client.persist(), Client(persistence_dir=...), and PersistentClient(path=...) have been the recommended entrypoints at different times. Pin a version and copy-paste from that version’s docs, not Stack Overflow.
3. Default embedding function downloads a model on first use. The bundled MiniLM ONNX model is fetched from a CDN — in air-gapped environments, pass an explicit embedding_function= or pre-cache the model.
4. HNSW index parameters are set at collection creation. hnsw:space, hnsw:M, and hnsw:construction_ef cannot be retroactively changed without rebuilding the collection. Decide on cosine vs L2 distance up front.
5. Tenancy model is recent and still maturing. Tenants and databases inside a single Chroma server are usable but underdocumented; production multi-tenant designs should test isolation carefully.
6. Client/server version mismatch is silent. A chromadb-client from 2024 talking to a 2026 server may appear to work for add but fail on a newer query parameter. Match minor versions.
7. In-memory Client() does not persist. Calling Client() with no path gives you an ephemeral store that vanishes on process exit. Use PersistentClient(path=...) or run the HTTP server for durability.

Real-world recipes#

The recipes below are package-level vignettes — they focus on the install footprint and the client/server topology each pattern requires, rather than re-teaching collection methods (the companion sections/ai/chromadb covers the API surface).

Persistent local store for a single-process app — the smallest possible Chroma deployment. PersistentClient writes to a directory of SQLite + parquet shards; restarting the process re-opens the same data.

import chromadb
from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction

client = chromadb.PersistentClient(path=".chroma")
emb = SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
docs = client.get_or_create_collection("kb", embedding_function=emb)
docs.upsert(
    ids=["a", "b"],
    documents=["Chroma is an embedded vector DB.", "Pinecone is a hosted SaaS."],
    metadatas=[{"source": "intro"}, {"source": "intro"}],
)
print(docs.query(query_texts=["embedded database"], n_results=1))

Output: the closest match with its id, distance, and metadata; the .chroma/ directory holds chroma.sqlite3 plus per-collection parquet segments

Client-server split for multi-process serving — run chroma run --path .chroma --host 0.0.0.0 --port 8000 in one container, then point every app at it with the slim client. This is the only safe way to share a Chroma store across workers.

import chromadb

client = chromadb.HttpClient(host="chroma.internal", port=8000)
docs = client.get_collection("kb")
print(docs.count())

Output: count of documents in the remote collection; the worker image only needs chromadb-client (~5 MB) — no ONNX, no sentence-transformers, no server dependencies

Custom embedding function for an in-house model — Chroma’s embedding-function interface is a duck-typed callable. Implement __call__(self, input: list[str]) -> list[list[float]] and pass it as embedding_function=.

import chromadb
from chromadb import Documents, EmbeddingFunction, Embeddings

class MyEmbedder(EmbeddingFunction):
    def __init__(self, model):
        self.model = model
    def __call__(self, input: Documents) -> Embeddings:
        return self.model.encode(input).tolist()

client = chromadb.PersistentClient(path=".chroma")
col = client.get_or_create_collection(
    "docs",
    embedding_function=MyEmbedder(my_local_model),
    metadata={"hnsw:space": "cosine"},
)

Output: the collection materialises with a custom embedder; metadata pins the distance function at creation time (cannot be changed retroactively)

Hybrid filter + vector query — Chroma supports a structured where= filter over metadata and a where_document= substring/regex filter, applied as a prefilter to the ANN search.

results = col.query(
    query_texts=["how does HNSW work"],
    n_results=5,
    where={"section": {"$in": ["intro", "tuning"]}},
    where_document={"$contains": "HNSW"},
)

Output: the top-5 hits restricted to documents whose metadata section is intro or tuning AND whose body literally contains the substring HNSW

Multi-tenant via collection-per-tenant — Chroma’s tenants/databases primitive is still maturing; a robust pattern today is one collection per tenant with a name prefix and shared embedding function.

def get_tenant_collection(tenant_id: str):
    return client.get_or_create_collection(
        name=f"tenant_{tenant_id}",
        embedding_function=emb,
    )

Output: every tenant’s data lives in its own collection, with hard isolation at the storage layer — no risk of a faulty where clause leaking rows across tenants

Production deployment#

The two production topologies are embedded persistent (single process owns the directory) and client-server (the chroma run HTTP server fronts a shared volume). Pick early — the on-disk format is the same but the failure modes are not.

Topology checklist:

Pinning client and server. When you run the server in Docker (chromadb/chroma:0.5.x), pin the application’s chromadb-client to the same minor. Cross-minor combinations silently fail on newer query parameters (include=["embeddings"] was added late in 0.4.x; older clients ignore it).

Telemetry opt-out — Chroma sends anonymous PostHog pings on startup. Disable per-process with CHROMA_TELEMETRY=False, or in code via Settings(anonymized_telemetry=False). Required for many compliance reviews.

Backups. The persistent directory is roughly safe to tar while the process is idle. For zero-downtime backups, run chroma run with a filesystem that supports atomic snapshots (ZFS, btrfs, LVM, EBS snapshot) and snapshot the volume rather than copying the live directory.

Multi-tenancy strategy. Two paths exist; both work in production today:

1. Collection-per-tenant — strong isolation; tenant deletion is a single delete_collection. Limit: ~thousands of collections per server before metadata lookups slow down.
2. Filter-per-tenant — one shared collection with tenant_id in metadata, queried via where={"tenant_id": "..."}. Cheaper at scale, but a missing where clause leaks rows. Add an assertion in your query wrapper.

The newer tenants/databases primitive (client.create_tenant(...)) is still maturing — test isolation explicitly before relying on it for regulated workloads.

Index tuning & retrieval quality#

Chroma uses HNSW (Hierarchical Navigable Small World) under the hood. The three parameters worth knowing are hnsw:space (distance metric), hnsw:M (graph connectivity), and hnsw:construction_ef (build-time candidate pool). All three are set at collection creation time and cannot be changed without rebuilding.

col = client.create_collection(
    name="tuned_kb",
    embedding_function=emb,
    metadata={
        "hnsw:space": "cosine",         # cosine | l2 | ip
        "hnsw:M": 32,                   # default 16; higher = better recall, more RAM
        "hnsw:construction_ef": 200,    # default 100; higher = better index, slower build
        "hnsw:search_ef": 100,          # default 10; raise at query time for higher recall
    },
)

Output: a collection whose HNSW index is built with a larger candidate pool and higher graph degree — measurably better recall at p95 latency cost

Trade-off table:

Distance metric choice. For most modern sentence embeddings (MiniLM, BGE, OpenAI text-embedding-3-*), cosine is the right default — the embeddings are already unit-normalised in spirit. Use ip (inner product) only with explicitly unnormalised embeddings, and l2 only when the embedding model recommends it.

Hybrid filter + vector. Chroma applies metadata where= and document where_document= filters as prefilters — the ANN search runs over the filtered subset. This is fast when filters are selective (the search index is restricted to a small candidate set) and slow when filters match most of the corpus (the prefilter scan dominates).

Reranking pattern. Chroma does not ship a built-in reranker. The standard pattern is to over-retrieve (n_results=50), then rerank in your application with a cross-encoder (sentence-transformers/ms-marco-MiniLM-L-6-v2 or Cohere Rerank), and keep the top 5–10.

Version migration guide#

The 0.4.x → 0.5.x boundary is the largest on-disk change to date. Lesser bumps within 0.5.x also rename APIs.

0.4.x → 0.5.x checklist:

• On-disk persistent-store schema changed. Existing chroma.sqlite3 stores need the maintainer-provided migration script. Snapshot the directory first.
• Client(persist_directory=...) removed. Use PersistentClient(path=...) for embedded persistence, or HttpClient(host=..., port=...) for the server.
• Client.persist() removed. Persistent clients write through automatically; the no-op call was misleading and is gone.
• Settings(chroma_db_impl=...) removed. Backend is implicit from which client class you instantiate.
• Telemetry env var is CHROMA_TELEMETRY=False; older ANONYMIZED_TELEMETRY no longer applies.

0.5.x minor-to-minor:

• Tenants/databases primitive evolved across 0.5 minors — client.create_tenant(...) and client.create_database(...) signatures shifted. Pin if you depend on the new isolation model.
• get_or_create_collection metadata validation tightened — invalid hnsw:* keys now raise instead of being silently dropped.
• include= parameter on query() added options ("embeddings", "distances", "metadatas", "documents", "data"); older clients send unrecognised values and 0.5 servers may reject them.

Client/server pinning. Always run the same minor on both sides. The Docker image tag (chromadb/chroma:0.5.x) and the chromadb-client minor must match — cross-minor combinations sometimes work for add()/get() and silently fail on newer query parameters.

The roadmap targets a 1.0 once tenancy and the on-disk format stabilise; treat every pre-1.0 minor as a potential breaking release in CI.

Troubleshooting common errors#

The error catalogue below is what trips up new users most often. Most are environmental rather than code bugs.

• DuplicateIDError on add(...) — the ID already exists in the collection. Switch to upsert(...), which inserts or updates.
• InvalidCollectionException — collection name doesn’t exist or was deleted. Use get_or_create_collection for idempotent code.
• ValueError: Expected metadata to be a non-empty dict — Chroma rejects empty metadatas dicts. Either omit metadatas= entirely or pass at least one key per row.
• ConnectionError against chroma run — the server is not listening on the expected port, or the firewall is blocking it. curl http://chroma:8000/api/v1/heartbeat should return {"nanosecond heartbeat": ...}.
• Could not connect to tenant with HttpClient — the server is on an older minor that doesn’t know about tenants. Either upgrade the server or instantiate without tenant=.
• Default embedder downloads ONNX model on first run — air-gapped environments need the model pre-cached, or use a custom embedding_function=. Symptom: hang on first add() while a CDN times out.
• OperationalError: database is locked — two PersistentClient instances opened the same directory. Embedded mode is single-writer; move to chroma run or coordinate access.
• Cross-version protocol error — chromadb-client from 2024 hitting a 2026 server (or vice versa) fails with opaque 400 responses on newer query parameters. Match minors.

Performance tuning#

Chroma’s performance levers are about telling the engine what to skip more than telling it to go faster. The HNSW index handles vector search; everything else (filters, payload returns, embedding-function calls) is in your control.

Bulk ingestion pattern. Batch documents into upsert(...) calls of a few hundred at a time. Smaller batches are network-overhead-bound; larger batches occupy the server’s index build budget and stall reads.

def chunks(iterable, n):
    buf = []
    for item in iterable:
        buf.append(item)
        if len(buf) >= n:
            yield buf
            buf = []
    if buf:
        yield buf

for batch in chunks(iter_rows(), 500):
    col.upsert(
        ids=[r["id"] for r in batch],
        documents=[r["text"] for r in batch],
        metadatas=[r["meta"] for r in batch],
    )

Output: streams documents into the collection in 500-row batches; the server returns once each batch is persisted

Query-time search_ef. Set per collection at creation, or tune per query if your version supports it. Higher search_ef improves recall linearly with query latency — production deployments set this from a p95 latency budget rather than a fixed default.

Avoid the default embedder in production. The bundled MiniLM-via-ONNX function downloads a model on first call and re-loads it per process. Pass an explicit embedding_function= that uses a long-lived embedding service (OpenAI, Cohere, local sentence-transformers) — both faster and easier to upgrade.

Embeddings & chunking strategy#

Chroma stores vectors and metadata; it does not produce embeddings (the bundled MiniLM ONNX function is a default for convenience, not a recommendation). The embedding choice dominates retrieval quality far more than HNSW tuning, so it deserves explicit attention.

Embedding-model choice. A short-list current at writing:

Higher-dimensional embeddings improve recall but cost storage and RAM linearly. For corpora over ~1M chunks, prefer dim ≤ 1024 with quantization, or a Matryoshka model trimmed to a shorter dimension.

Chunking decisions live upstream. Chroma stores whatever chunks you give it; bad chunking can’t be fixed at retrieval time. The standard heuristics:

• 300–500 character chunks for narrow factual questions.
• 800–1500 character chunks for general RAG with modern long-context LMs.
• ~100 character overlap to avoid edge-case context truncation.
• Respect document structure (titles, sections) — unstructured’s chunk_by_title is a sensible default.

HyDE / query rewriting. When user queries are short and noisy, embedding them directly gives weak retrieval. The Hypothetical Document Embeddings pattern (HyDE) asks an LM to generate a plausible answer, embeds that, and queries Chroma with the synthetic embedding — often better than embedding the raw query.

Parent-document retrieval. Embed small chunks for precision; return the parent document (or a wider window) to the LM for context. Store parent_id in metadata and look it up after retrieval.

Security considerations#

Chroma’s default setup has minimal security — appropriate for embedded use, dangerous for a network-exposed server.

• No auth by default on chroma run. Place behind an authenticated reverse proxy (nginx with basic auth, an API gateway, or a VPC) before exposing to a network. Recent versions support a static API key via CHROMA_SERVER_AUTHN_PROVIDER/CHROMA_SERVER_AUTHN_CREDENTIALS; verify the version’s docs.
• TLS not built in — the HTTP server speaks plaintext. Terminate TLS at a proxy.
• Telemetry phones home unless disabled with CHROMA_TELEMETRY=False. Required for many compliance reviews.
• Multi-tenant leakage risk under filter-per-tenant. A missing where={"tenant_id": ...} clause returns rows across tenants. Either wrap every query in code that enforces the filter, or use collection-per-tenant for hard isolation.
• Prompt injection via retrieved content. Documents in Chroma are returned verbatim to the LM. A malicious uploaded document can contain instructions the LM follows (“ignore previous instructions and…”). Validate upload provenance; consider a sanitisation pass on retrieved content before prompt assembly.
• PII in metadata. Metadata is returned in every query response; don’t store anything you wouldn’t want in logs.
• Backup security. The persistent directory is plaintext SQLite + parquet. Encrypt at rest at the volume layer (LUKS, EBS encryption, etc.).

When NOT to use this#

Chroma is the easiest vector DB to start with; it stops being the right tool at a clear scale boundary.

• Corpora over ~10 million vectors with strict p95 latency. Qdrant or Milvus have more mature distributed stories. Chroma works but tuning gets painful.
• Hybrid (BM25 + vector) is a core requirement. Weaviate ships hybrid out of the box; Chroma needs application-side BM25 (e.g. rank-bm25 + post-merge), which is more code.
• Strict multi-tenant isolation with thousands of tenants. Collection-per-tenant slows down past low thousands; filter-per-tenant is leaky if a query forgets the where clause. Postgres-with-pgvector or a hosted service may fit better.
• You want a fully-hosted, managed service. Pinecone, Weaviate Cloud, and Qdrant Cloud all front their own engines. Chroma Cloud exists but is younger.
• Sparse-vector workloads (e.g. SPLADE). Qdrant and Weaviate have first-class sparse support; Chroma is dense-only at this writing.

See also#

• AI: chromadb — collections, queries, embedding functions, framework integration
• Concept: RAG — retrieval-augmented generation patterns
• Concept: API — REST design fundamentals