Gemini 3.1 Flash Live API Guide: Real-Time Voice, Camera, and Screen-Share Agents

Why Gemini 3.1 Flash Live matters in August 2026

Google launched Gemini 3.1 Flash Live via the Gemini Live API in March 2026, and by August it has become the recommended model for production voice-and-vision agents. Unlike batch generateContent calls, Live maintains a persistent WebSocket session that streams audio, video frames, and text bidirectionally — the model hears, sees, and speaks in one native pipeline without separate speech-to-text or text-to-speech services.

August 2026 brings three practical shifts for builders:

• 2.5 Flash Live models are deprecated — Google directs all new work to gemini-3.1-flash-live-preview.
• Ecosystem integrations matured — Stitch (design critique), Ato (elder-care companion), and Weekend RPG demos show camera-aware agents shipping in production.
• Live API + Gemini 3.7 Flash coexist — use Live for sub-second dialogue; use 3.7 Flash REST for deep coding and agent reasoning (see our 3.7 Flash announcement).

This guide covers architecture, audio/video parameters, deployment patterns, and migration from 2.5 Flash Live.

Architecture: WebSocket sessions, not REST

The Live API is stateful. A client opens a WebSocket to generativelanguage.googleapis.com, sends a BidiGenerateContentSetup message first, then streams BidiGenerateContentRealtimeInput frames until the session closes.

Layer	Responsibility
Transport	Stateful WSS (WebSocket Secure)
Setup	Model ID, modalities, system instruction, tools, VAD config
Realtime input	PCM audio chunks, JPEG/PNG video frames (≤1 FPS), inline text
Server output	PCM audio (24 kHz), transcriptions, tool calls, thinking metadata

Two deployment topologies:

1. Server-to-server — your backend proxies streams from clients. Best when you need centralized logging, rate limiting, or tool execution on trusted infrastructure.
2. Client-to-server — browser or mobile connects directly to Live API. Lower latency for mic/camera; use ephemeral tokens instead of exposing API keys.

import asyncio
from google import genai
from google.genai import types

client = genai.Client()
MODEL = "gemini-3.1-flash-live-preview"

async def main():
    config = {
        "response_modalities": ["AUDIO"],
        "speech_config": {
            "voice_config": {
                "prebuilt_voice_config": {"voice_name": "Aoede"}
            }
        },
        "system_instruction": {
            "parts": [{"text": "You are a helpful assistant. Be concise."}]
        },
    }
    async with client.aio.live.connect(model=MODEL, config=config) as session:
        # Stream mic PCM16 @ 16 kHz
        await session.send_realtime_input(
            audio=types.Blob(data=pcm_chunk, mime_type="audio/pcm;rate=16000")
        )
        async for message in session.receive():
            if message.server_content and message.server_content.model_turn:
                for part in message.server_content.model_turn.parts:
                    if part.inline_data:
                        play_audio(part.inline_data.data)  # 24 kHz PCM out

asyncio.run(main())

Audio: native end-to-end speech

Live API audio is raw PCM — no MP3 or WAV wrappers on the wire.

Direction	Format	Sample rate
Input	16-bit PCM, little-endian, mono	16 kHz native (API resamples if needed)
Output	16-bit PCM, little-endian	24 kHz fixed

Browser capture via Web Audio API returns 32-bit float — convert to PCM16 before sending. Output playback requires buffering chunks and scheduling at 24 kHz.

Voice Activity Detection (VAD) handles turn-taking by default. Configure sensitivity with realtime_input_config.automatic_activity_detection:

• start_of_speech_sensitivity / end_of_speech_sensitivity
• prefix_padding_ms — audio captured before speech start
• silence_duration_ms — silence required to end a turn

For push-to-talk UIs, disable automatic VAD and send activityStart / activityEnd manually.

Video: camera and screen sharing at 1 FPS

Gemini 3.1 Flash Live accepts video as a stream of JPEG or PNG frames, capped at 1 frame per second. This supports:

• Front/rear camera — object identification, environment Q&A, guided tasks
• Screen share — code review, design critique (Stitch demo), accessibility assistance

import { GoogleGenAI, Modality } from '@google/genai';

const ai = new GoogleGenAI({});
const session = await ai.live.connect({
  model: 'gemini-3.1-flash-live-preview',
  config: {
    responseModalities: [Modality.AUDIO],
    realtimeInputConfig: {
      turnCoverage: 'TURN_INCLUDES_AUDIO_ACTIVITY_AND_ALL_VIDEO',
    },
  },
});

// Capture screen share frame every 1000 ms
setInterval(async () => {
  const jpegBase64 = await captureScreenFrame();
  session.sendRealtimeInput({
    video: { data: jpegBase64, mimeType: 'image/jpeg' },
  });
}, 1000);

Cost tip: 3.1 Flash Live defaults to TURN_INCLUDES_AUDIO_ACTIVITY_AND_ALL_VIDEO, meaning every frame in a turn is billed. If you stream constant screen video, consider sending frames only during detected speech activity.

Limitation: 1 FPS is unsuitable for fast-motion analysis (sports play-by-play). Use batch video understanding models for offline review.

Session limits and extensions

Session type	Default limit	Extension
Audio-only	15 minutes	Session resumption + context compression
Audio + video	2 minutes	Same session management APIs

Use session resumption tokens to reconnect after network drops without losing context. GoAway signals warn before server-side session termination — handle gracefully in production clients.

Tool calling and Search grounding

Live supports synchronous function calling during conversation. When the model emits a toolCall, execute the function server-side and return BidiGenerateContentToolResponse before the model continues.

Supported tool types in preview:

• Custom function declarations (JSON schema)
• Google Search grounding — real-time web results mid-conversation
• Code execution (enterprise tier)

Note: 3.1 Flash Live does not support behavior: NON_BLOCKING async tools that 2.5 Flash Live allowed. Tool calls are sequential — the model waits for your response.

Thinking levels and multilingual support

3.1 Flash Live uses thinkingLevel (not thinkingBudget):

Level	Latency	Use case
minimal	Lowest (default)	Live dialogue, customer support
low	Low	Simple tool routing
medium	Moderate	Multi-step in-session reasoning
high	Higher	Complex visual analysis during call

The model supports 90+ languages for real-time multimodal conversation — pitch, pace, and tone recognition improved over 2.5 Flash Native Audio per Google’s March 2026 benchmarks.

Security: ephemeral tokens for client apps

Never ship API keys in mobile or browser bundles. The Live API supports ephemeral tokens — short-lived credentials minted by your backend:

1. Client requests a session token from your auth endpoint.
2. Backend calls Google’s token API with your master key.
3. Client connects to Live API with the ephemeral token (minutes TTL).
4. Token expires when session ends — no long-lived secret on device.

For WebRTC-scale deployments (global edge, telephony), Google recommends partner integrations: Fishjam, Stream Vision Agents, Voximplant, and others listed in the Live API overview.

Migration from Gemini 2.5 Flash Live

Google deprecated these models — migrate before shutdown:

Deprecated model	Shutdown	Replace with
gemini-2.5-flash-native-audio-preview-12-2025	Rolling	gemini-3.1-flash-live-preview
gemini-live-2.5-flash-preview	Dec 9, 2025	gemini-3.1-flash-live-preview
gemini-2.0-flash-live-001	Dec 9, 2025	gemini-3.1-flash-live-preview

Migration checklist:

1. Update model string to gemini-3.1-flash-live-preview.
2. Replace thinkingBudget with thinkingLevel.
3. Use send_realtime_input for mid-conversation text — send_client_content is only for seeding initial history.
4. Process multiple parts per server event — 3.1 can return audio + transcript in one message.
5. Review turnCoverage default change — affects video billing.
6. Remove proactive_audio and enable_affective_dialog — not supported on 3.1.

3.1 Flash Live vs batch Gemini 3.7 Flash

Factor	3.1 Flash Live	3.7 Flash (REST)
Protocol	WebSocket streaming	HTTP REST
Latency	Sub-second first audio	85 ms+ first token (fast mode)
Modalities in	Audio + video stream + text	Text, image, audio, video files
Modalities out	Native audio (+ transcripts)	Text (+ optional audio)
Context	128K session window	2.5M tokens
Best for	Voice agents, live tutoring, screen assist	Coding agents, document RAG, batch analysis

Many production stacks use both: Live handles the user-facing voice layer; 3.7 Flash handles backend reasoning, code generation, and long-context retrieval triggered by Live tool calls.

Getting started

1. Open Google AI Studio → select Stream to test Live interactively.
2. Install SDK: pip install google-genai or npm install @google/genai.
3. Read the Live API capabilities guide for VAD tuning and session management.
4. Explore the Gemini Live API Skill for coding-agent workflows.

Related

• Gemini 3.7 Flash announcement
• Gemini Omni August 2026 update
• Gemini Omni API developer guide
• Gemini Omni in Google Flow