Chroma

Chroma's design philosophy

Chroma is built for developers who want to go from "zero to working RAG" in minutes, not hours. It prioritises simplicity: one pip install, no external service, runs in-process. The tradeoff is that it's not designed for billion-scale production deployments — for that, reach for Qdrant, Pinecone, or Milvus.

When to use Chroma: prototyping, personal projects, small-to-medium production RAG (<1M docs), local development without a running service.

In-memory quick start

pip install chromadb

import chromadb

# Ephemeral in-memory client (data lost on restart)
client = chromadb.Client()

# Create a collection (like a table in SQL)
collection = client.create_collection(name="my_docs")

# Add documents — Chroma handles embedding with its default model
collection.add(
    documents=[
        "Python was created by Guido van Rossum in 1991.",
        "JavaScript is the language of the web browser.",
        "Rust provides memory safety without garbage collection.",
    ],
    ids=["doc-1", "doc-2", "doc-3"]    # unique IDs, you choose
)

# Query
results = collection.query(
    query_texts=["Who invented Python?"],
    n_results=2
)
print(results["documents"])   # [["Python was created by...", "JavaScript is..."]]
print(results["distances"])   # similarity scores

Persistent storage

import chromadb

# Persistent client — saves to disk automatically
client = chromadb.PersistentClient(path="./chroma_db")

# Collections persist across restarts
collection = client.get_or_create_collection("my_docs")
collection.add(
    documents=["Document content here."],
    ids=["doc-001"]
)

# Later, in a new process:
client2 = chromadb.PersistentClient(path="./chroma_db")
collection2 = client2.get_collection("my_docs")
print(collection2.count())   # still 1

For a standalone server (useful for multi-process or client-server setups):

chroma run --path ./chroma_db --port 8000

client = chromadb.HttpClient(host="localhost", port=8000)

Embedding functions

By default, Chroma uses all-MiniLM-L6-v2 via sentence-transformers. Override with any embedding function:

import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

# Use OpenAI embeddings
openai_ef = OpenAIEmbeddingFunction(
    api_key="your-api-key",
    model_name="text-embedding-3-small"
)
collection = client.create_collection(
    name="openai_collection",
    embedding_function=openai_ef
)

# Or bring your own via sentence-transformers
from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction
st_ef = SentenceTransformerEmbeddingFunction(model_name="BAAI/bge-large-en-v1.5")
collection = client.create_collection(name="bge_collection", embedding_function=st_ef)

Querying with filters

import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection("docs")

# Add docs with metadata
collection.add(
    documents=["FAQ: returns are free within 30 days.", "Blog: our story began in 2020.", "FAQ: shipping takes 3-5 days."],
    metadatas=[{"type":"faq","year":2023},{"type":"blog","year":2020},{"type":"faq","year":2023}],
    ids=["faq-1","blog-1","faq-2"]
)

# Filter: only FAQ documents
results = collection.query(
    query_texts=["How do I return a product?"],
    n_results=2,
    where={"type": {"$eq": "faq"}}       # metadata filter
)

# Full-text filter (where_document)
results = collection.query(
    query_texts=["return policy"],
    n_results=2,
    where_document={"$contains": "30 days"}
)

LangChain integration

from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain.schema import Document

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

# Create a vector store from documents
docs = [
    Document(page_content="Refunds within 30 days.", metadata={"source": "faq"}),
    Document(page_content="Free shipping over $50.", metadata={"source": "faq"}),
]
vectorstore = Chroma.from_documents(
    documents=docs,
    embedding=embeddings,
    persist_directory="./chroma_db"
)

# Use as a retriever in a RAG chain
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
docs = retriever.invoke("What is the return policy?")
print(docs[0].page_content)

Gotchas

Embedding function must match at read time. If you created a collection with OpenAI embeddings and query it with the default Chroma embeddings, you'll get garbage results. The collection doesn't store which embedding function was used — you must remember and pass the same one every time.

Not designed for concurrent writes. SQLite-backed persistent Chroma has write locking issues under high concurrency. For multi-process production workloads, use the HTTP server mode or a different database.

IDs must be unique strings. Upserting with an existing ID replaces the old document. Deleting requires knowing the IDs — there's no "delete by metadata" operation at the Python client level.

No built-in replication. The embedded/persistent mode has a single node. For HA, deploy the Chroma server behind your own load balancer, or use a database with native replication (Qdrant, Weaviate).