Gemini 1.5 Pro

Gemini 1.5 Pro overview

Gemini 1.5 Pro (released February 2024) is Google DeepMind's most capable publicly available model as of mid-2024. Its defining feature is a 1 million token context window — enough to fit the entire Harry Potter series (1.1M words), a full codebase, or 10 hours of audio in a single prompt.

Architecture: a sparse Mixture-of-Experts transformer, similar in concept to Mixtral but at frontier scale. MoE allows high quality with lower inference cost per token than a comparably capable dense model. The architecture details are not fully disclosed.

Capabilities: text, images, audio, video, and documents all as native input modalities in a single unified model — no separate vision or audio encoders called separately.

The 1M context window

The 1M context window enables qualitatively new use cases that smaller windows don't support:

• Whole-codebase analysis: Send an entire 100K-line codebase and ask for architectural issues, unused code, or refactoring suggestions.
• Long document Q&A without retrieval: Send the full 500-page PDF directly instead of building a RAG pipeline. Faster to prototype, no chunking errors.
• Video understanding: Process 1 hour of video (as frames + transcript) and ask questions about specific moments.
• Multi-turn conversations with full history: Never truncate conversation history for complex long-running tasks.

In practice, "needle-in-a-haystack" benchmarks show Gemini 1.5 Pro maintains ~98% recall across the full 1M context, though reasoning quality on very long contexts can still degrade for complex multi-hop questions.

Using the Gemini API

import google.generativeai as genai

genai.configure(api_key="your-api-key")
model = genai.GenerativeModel("gemini-1.5-pro")

# Text generation
response = model.generate_content(
    "Explain the difference between RAG and fine-tuning in 3 bullet points."
)
print(response.text)

# With system instruction
model = genai.GenerativeModel(
    "gemini-1.5-pro",
    system_instruction="You are a concise technical writer. Always use code examples.",
)

# Multi-turn chat
chat = model.start_chat()
r1 = chat.send_message("What is attention in transformers?")
r2 = chat.send_message("Now show me a NumPy implementation.")
print(r2.text)

# Generation config
response = model.generate_content(
    "Write a haiku about gradient descent.",
    generation_config=genai.types.GenerationConfig(
        temperature=0.9, max_output_tokens=100, top_p=0.95),
)

Multimodal capabilities

import google.generativeai as genai
from PIL import Image
import requests
from io import BytesIO

genai.configure(api_key="your-api-key")
model = genai.GenerativeModel("gemini-1.5-pro")

# Image understanding
img = Image.open("architecture_diagram.png")
response = model.generate_content([
    img,
    "Describe this architecture diagram and identify any potential bottlenecks.",
])
print(response.text)

# PDF analysis (send raw bytes)
with open("research_paper.pdf", "rb") as f:
    pdf_data = f.read()

response = model.generate_content([
    {"mime_type": "application/pdf", "data": pdf_data},
    "What are the 3 main contributions of this paper?",
])

# Audio transcription + analysis
with open("meeting.mp3", "rb") as f:
    audio_data = f.read()

response = model.generate_content([
    {"mime_type": "audio/mp3", "data": audio_data},
    "Summarise this meeting and list all action items.",
])

Gemini 1.5 Flash

Gemini 1.5 Flash is a smaller, faster, cheaper variant optimised for high-throughput tasks where cost matters more than maximum quality. It maintains the 1M context window and multimodal capabilities but runs faster and costs ~10× less than Pro.

model = genai.GenerativeModel("gemini-1.5-flash")

# Flash is good for:
# - Classification tasks (sentiment, routing)
# - Document summarisation
# - Quick Q&A over documents
# - High-volume pipelines

# Pricing comparison (approximate):
# gemini-1.5-pro:   $3.50/1M input tokens, $10.50/1M output
# gemini-1.5-flash: $0.35/1M input tokens, $1.05/1M output
# gemini-1.5-flash-8b: $0.037/1M input (cheapest in family)

response = model.generate_content("Classify this review as positive/negative: 'Great product!'")
print(response.text)  # Positive

Context caching for cost efficiency

With a 1M context window, sending a large document with every request becomes expensive. Gemini's context caching lets you upload a document once, get a cache handle, and reuse it across many queries at a fraction of the cost.

import google.generativeai as genai
from google.generativeai import caching
import datetime

genai.configure(api_key="your-api-key")

# Upload and cache a large document
with open("large_codebase.txt") as f:
    codebase = f.read()

cache = caching.CachedContent.create(
    model="models/gemini-1.5-pro",
    contents=[{"role": "user", "parts": [{"text": codebase}]}],
    ttl=datetime.timedelta(hours=1),
    display_name="my-codebase-cache",
)

# Use the cache for multiple queries at reduced cost
model = genai.GenerativeModel.from_cached_content(cached_content=cache)
r1 = model.generate_content("Find all TODO comments in this codebase.")
r2 = model.generate_content("What are the main entry points?")
r3 = model.generate_content("List all database queries.")
# The codebase tokens are only billed once

Gotchas

Rate limits on long contexts: The 1M context is available but rate-limited heavily. Free tier allows only a few requests per minute with long contexts. Production use requires paid tier.

Context window ≠ context quality: Gemini 1.5 Pro can recall facts from 1M tokens, but complex reasoning over very long contexts still degrades. For multi-hop questions requiring information from many scattered locations, retrieval augmentation often beats raw long context.

Pricing with long contexts: At $3.50/1M input tokens, sending a 100K-token document per query costs $0.35 per call. Budget this before building long-context pipelines. Context caching (see above) is essential for cost control.

API vs Vertex AI: The Google AI Studio API (api.generativeai.google.com) is simpler for prototyping. Vertex AI is recommended for production: better SLAs, IAM-based auth, data residency controls, and enterprise support.