Pinecone

The managed database tradeoff

Self-hosting a vector database (Qdrant, Weaviate, Milvus) gives you full control and zero per-query cost, but you own the infrastructure: provisioning, scaling, backups, upgrades, and on-call. Pinecone is the opposite bet — you pay per vector and per query, but Anthropic operations are someone else's problem.

Pinecone makes sense when: your team is small, time-to-production matters, and your query volume is predictable. At very high query volumes, the per-query cost can exceed the cost of self-hosting — do the math for your workload.

Pinecone architecture

A Pinecone index stores vectors with associated metadata. Each vector has a unique ID (a string you choose), a values array (the embedding), and an optional metadata dict.

Pinecone uses a hierarchical navigable small world (HNSW) or IVF index under the hood for approximate nearest-neighbour search. You configure the metric (cosine, dot product, euclidean) at index creation time — choose cosine for normalised embeddings, dot product if you're using magnitude as a relevance signal.

Setup and first index

pip install pinecone-client

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="your-api-key")

# Create a serverless index (free tier: 1 index, 10M vectors)
pc.create_index(
    name="my-rag-index",
    dimension=1536,           # must match your embedding model's output dims
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

# Connect to the index
index = pc.Index("my-rag-index")
print(index.describe_index_stats())

Upsert and query

from openai import OpenAI
from pinecone import Pinecone

client = OpenAI()
pc = Pinecone(api_key="your-api-key")
index = pc.Index("my-rag-index")

def embed(text: str) -> list[float]:
    return client.embeddings.create(
        input=[text], model="text-embedding-3-small"
    ).data[0].embedding

# Ingest documents
docs = [
    {"id": "doc-1", "text": "Refunds are processed within 5 business days.", "source": "faq"},
    {"id": "doc-2", "text": "Shipping is free on orders over $50.", "source": "faq"},
    {"id": "doc-3", "text": "Our CEO is Jane Smith.", "source": "about"},
]

vectors = [
    {
        "id": doc["id"],
        "values": embed(doc["text"]),
        "metadata": {"text": doc["text"], "source": doc["source"]}
    }
    for doc in docs
]
index.upsert(vectors=vectors)

# Query
query_emb = embed("How long do refunds take?")
results = index.query(
    vector=query_emb,
    top_k=3,
    include_metadata=True
)
for match in results.matches:
    print(f"Score {match.score:.3f}: {match.metadata['text']}")

Metadata filtering

Pinecone supports filtering on metadata fields to narrow the search space before ANN — crucial for multi-tenant apps and category-scoped search:

# Only search within the "faq" source
results = index.query(
    vector=query_emb,
    top_k=5,
    include_metadata=True,
    filter={"source": {"$eq": "faq"}}
)

# More complex filter: source is faq AND doc_type is not "archived"
results = index.query(
    vector=query_emb,
    top_k=5,
    filter={
        "$and": [
            {"source": {"$eq": "faq"}},
            {"doc_type": {"$ne": "archived"}}
        ]
    },
    include_metadata=True
)

Supported operators: $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $and, $or.

Namespaces for multi-tenancy

Namespaces partition an index so different tenants can't see each other's data, without the cost of separate indexes:

# Ingest into a tenant-specific namespace
index.upsert(
    vectors=[{"id": "doc-1", "values": embed("..."), "metadata": {...}}],
    namespace="tenant-abc"
)

# Query only within that tenant's namespace
results = index.query(
    vector=query_emb,
    top_k=5,
    namespace="tenant-abc",
    include_metadata=True
)

# Delete all data for a tenant (GDPR compliance)
index.delete(delete_all=True, namespace="tenant-abc")

Gotchas

Dimension is permanent. You choose the vector dimension at index creation and cannot change it. If you switch embedding models, you must recreate the index and re-embed everything.

Metadata is not free. Large metadata dicts increase storage costs and slow down filtering. Store only the fields you'll actually filter on; keep the full document text in a separate store (Postgres, S3).

Serverless cold starts. Serverless indexes that haven't been queried recently may have a brief cold-start latency spike on the first query after inactivity. For SLA-sensitive endpoints, use pod-based indexes or keep a warm-up pinger.

Eventual consistency. Upserted vectors are not immediately queryable — there's a short propagation delay (usually <1 second). Don't assume a vector is searchable immediately after upsert in time-sensitive tests.