OpenAIEngineering

Scaling PostgreSQL to power 800 million ChatGPT users

January 22, 2026 at 12:00 PM

Summary

TL;DR

OpenAI kept PostgreSQL unsharded (single writer) while scaling reads to millions of QPS using replicas, aggressive query/app hygiene, and load-shedding controls.

What actually happened

PostgreSQL load grew by more than 10x over the past year.
OpenAI served global read-heavy traffic with one primary and nearly 50 read replicas.
Repeated incidents followed a pattern: upstream faults triggered cache misses, expensive joins, or write storms.
They moved shardable write-heavy workloads off Postgres and tightened operational guardrails.

Key numbers

800 million users.

More than 10x PostgreSQL load growth over the past year.

Nearly 50 read replicas across multiple regions.

Azure PostgreSQL connection limit: 5,000 per instance.

PgBouncer reduced average connection time from 50 ms to 5 ms.

p99 client-side latency: low double-digit milliseconds.

Availability: five-nines.

Only one SEV-0 PostgreSQL incident in the past 12 months.

ImageGen launch: write traffic surged by more than 10x.

Over 100 million new users signed up within a week.

One expensive query joined 12 tables.

5-second timeout enforced on schema changes.

Why this was hard

Single-primary architecture makes write spikes a direct path to system-wide degradation.
MVCC causes write amplification, dead tuples, bloat, and tricky autovacuum tuning under heavy writes.
Cache-miss and retry storms can turn a small upstream issue into a runaway load cycle.
Scaling replicas increases WAL shipping load, driving replica lag and primary CPU/network pressure.

How they solved it

Offloaded reads to replicas; kept primary capacity reserved for write spikes.
Migrated shardable write-heavy workloads to sharded systems (Azure Cosmos DB); optimized apps to reduce writes.
Added lazy writes and strict rate limits for backfills to smooth write pressure.
Continuous query optimization; avoided OLTP anti-patterns like large multi-table joins.
Reviewed ORM-generated SQL; set timeouts (e.g., idle_in_transaction_session_timeout) to prevent autovacuum blocking.
Ran primary in HA with a hot standby; promoted on failure/maintenance.
Isolated workloads by priority tiers onto dedicated instances to avoid noisy neighbors.
Deployed PgBouncer (statement/transaction pooling) via Kubernetes per replica; co-located proxy/clients/replicas regionally.
Implemented cache locking/leasing so only one miss per key hits Postgres; others wait for cache fill.
Added multi-layer rate limiting (app, pooler, proxy, query) and could block specific query digests.
Restricted schema changes to lightweight operations; allowed concurrent index ops; disallowed new tables in this Postgres.
Collaborated on cascading replication (intermediate replicas relay WAL) to scale beyond current replica counts.

What changed

Replication lag kept near zero while adding nearly 50 read replicas.
Read-heavy workloads sustained at millions of QPS.
p99 latency held at low double-digit milliseconds.
Achieved five-nines availability; only one SEV-0 Postgres incident in 12 months.

Why this matters beyond this company

Single-writer Postgres can scale far for read-heavy systems if writes are constrained and isolated.
Preventing load amplification (cache-miss storms, retries, idle transactions) is as important as raw capacity.
ORM usage requires continuous inspection and guardrails, not blind trust in generated SQL.

Stealable ideas

Use cache locking/leasing to collapse thundering herds on hot keys.
Add rate limiting across layers and block offending query digests during incidents.
Enforce strict schema-change timeouts and only allow non-rewrite operations.
Tier workloads and route low-priority traffic to separate instances to contain blast radius.

Read original article

Scaling PostgreSQL to power 800 million ChatGPT users

Summary

What actually happened

Key numbers

Why this was hard

How they solved it

What changed

Why this matters beyond this company

Stealable ideas

More from OpenAI

Scaling PostgreSQL to power 800 million ChatGPT users

Summary

What actually happened

Key numbers

Why this was hard

How they solved it

What changed

Why this matters beyond this company

Stealable ideas

More from OpenAI