Streamlit

Why Streamlit for LLM apps

Streamlit is a Python library that renders interactive web apps directly from your script — no HTML, CSS, JavaScript, or web framework knowledge required. Every time the user interacts with a widget, Streamlit re-runs the script from top to bottom and updates the displayed output.

For LLM applications, Streamlit provides exactly the primitives you need: st.chat_message (speech bubble UI), st.chat_input (message input box), st.session_state (persist conversation history across script re-runs), and native support for streaming responses token-by-token. A complete LLM chatbot with streaming, conversation history, and a model selector is ~60 lines of Python.

pip install streamlit openai
streamlit run app.py

Building a chatbot in 30 lines

import streamlit as st
from openai import OpenAI

st.title("My LLM Chatbot")
client = OpenAI()

# Initialize conversation history in session state
if "messages" not in st.session_state:
    st.session_state.messages = [
        {"role": "system", "content": "You are a helpful assistant."}
    ]

# Display conversation history
for msg in st.session_state.messages[1:]:  # skip system message
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

# Handle new user input
if prompt := st.chat_input("Ask me anything..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=st.session_state.messages,
        )
        reply = response.choices[0].message.content
        st.markdown(reply)
    st.session_state.messages.append({"role": "assistant", "content": reply})

Streaming responses

import streamlit as st
from openai import OpenAI

client = OpenAI()

if "messages" not in st.session_state:
    st.session_state.messages = []

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

if prompt := st.chat_input("Message..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        # Stream tokens as they arrive
        stream = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "system", "content": "You are helpful."}]
                     + st.session_state.messages,
            stream=True,
        )
        # st.write_stream handles the OpenAI stream natively
        response = st.write_stream(stream)

    st.session_state.messages.append({"role": "assistant", "content": response})

st.write_stream (added in Streamlit 1.31) consumes an OpenAI stream and renders tokens progressively — no manual chunk handling needed.

Session state and conversation history

import streamlit as st

# st.session_state persists data across re-runs for each user session
# Perfect for: conversation history, user preferences, loaded models

# Pattern: initialize once, use everywhere
if "chat_history" not in st.session_state:
    st.session_state.chat_history = []
if "model" not in st.session_state:
    st.session_state.model = "gpt-4o-mini"

# Can store any Python object
if "rag_pipeline" not in st.session_state:
    st.session_state.rag_pipeline = build_rag_pipeline()  # expensive — only once

# Clear history button
if st.button("Clear conversation"):
    st.session_state.chat_history = []
    st.rerun()  # re-run the script to update the display

# Download chat history
import json
if st.button("Export chat"):
    st.download_button(
        "Download JSON",
        json.dumps(st.session_state.chat_history, indent=2),
        "chat_history.json",
        "application/json",
    )

Sidebar controls

import streamlit as st
from openai import OpenAI

# Sidebar for configuration
with st.sidebar:
    st.header("Settings")
    model = st.selectbox("Model", ["gpt-4o", "gpt-4o-mini", "gpt-3.5-turbo"])
    temperature = st.slider("Temperature", 0.0, 2.0, 0.7, 0.1)
    max_tokens = st.number_input("Max tokens", 100, 4000, 500)
    system_prompt = st.text_area("System prompt", "You are a helpful assistant.")
    if st.button("Reset conversation"):
        st.session_state.messages = []
        st.rerun()

st.title(f"Chat with {model}")
client = OpenAI()

# Use sidebar values in the API call
if prompt := st.chat_input("..."):
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "system", "content": system_prompt}]
                 + st.session_state.messages
                 + [{"role": "user", "content": prompt}],
        temperature=temperature,
        max_tokens=max_tokens,
    )

Deploying to Streamlit Cloud

# requirements.txt
streamlit>=1.31.0
openai>=1.0.0
langchain>=0.1.0
chromadb>=0.4.0

Streamlit Cloud (streamlit.io/cloud) deploys public GitHub repos for free (with resource limits). Steps:

1. Push your app to GitHub (make sure API keys are not in the code).
2. Go to share.streamlit.io → New app → Select your repo and file.
3. Add secrets in Settings → Secrets: OPENAI_API_KEY = "sk-..."
4. Access secrets in code: openai.api_key = st.secrets["OPENAI_API_KEY"]

For private apps or higher resources, use Streamlit in Docker on any cloud VM. The official Docker image: FROM python:3.11; RUN pip install streamlit; EXPOSE 8501; CMD streamlit run app.py.

Gotchas

Re-runs and state: Streamlit re-runs the entire script on every interaction. Expensive operations (loading models, building indexes) must be wrapped in @st.cache_resource to avoid re-execution. Without caching, your LLM would reload on every message.

@st.cache_resource  # cached across all users and sessions
def load_rag_pipeline():
    return build_expensive_rag_pipeline()

pipeline = load_rag_pipeline()  # only runs once per Streamlit server start

Multiuser state: st.session_state is per-session, not global. Each browser tab gets its own session state. For shared state across users (admin controls, shared document store), use an external database or Redis, not session state.

Production limitations: Streamlit is excellent for demos and internal tools but not ideal for high-traffic production apps (no built-in auth beyond Streamlit Community Cloud's Google OAuth, limited horizontal scaling). For public-facing production apps with 1000+ daily users, consider FastAPI + React instead.