OpenAI 用 Rust 重写 Habitat 的技术洞察报告OpenAI Rewrote Its Storage Tier in Rust. The Timing Is the Real Story.

结论先行：Rust 重写解决的不是单点性能问题，而是三层叠加的成本结构问题——① Python GIL 导致的事件循环调度延迟 → 尾延迟；② 绕开 GIL 采用的“海量进程横向扩展”策略引发的进程爆炸连锁成本（连接惊群、NAT 耗尽、部署抖动）；③ 作为 OpenAI 按 CPU core 计第二大服务的资源摊销问题。但更深一层的洞察是：真正的杠杆不是 Rust 本身，而是 OpenAI 把“战略性技术债”的时间点押在了 AI 编程能力成熟曲线上——2 名工程师 + Codex + GPT-5.5 完成了传统上需要 12 人×6 个月级别的迁移，这重新定义了“何时偿还技术债”的决策函数。

---

01 / Habitat 是什么：被忽视的“隐形引擎”

谈到 AI Infra，公众注意力都在 GPU；但服务 10 亿周活用户的瓶颈往往在数据层。Habitat 是 OpenAI 自建的在线存储统一接入平台，所有产品（ChatGPT / API / Codex / 内部服务）的数据访问都经过它：schema 查找、路由、授权、加密、序列化、请求整形、连接池全部下沉到这里。

| 指标 | 数值 | |---|---| | 每秒请求数 | 7000 万+ | | 每周服务用户 | 10 亿+ | | 数据总量 | 500 PB+ | | 地理区域 | 近 40 个 | | 年增长率 | 连续三年 10x+ |

演进路径：2023 DevDay 上线时只是对接 Azure Cosmos DB 的 Python 客户端库 → 2025 年中升级为独立 Python 服务 → 2026 Q2 由 2 名工程师借助 Codex + GPT-5.5 重写为 Rust，当前承接 95% 生产流量，数周内完全下线 Python。Python 版退役前峰值已扛到 2000 万+ QPS——这本身就是一个值得记录的工程极限。

---

02 / Rust 到底解决了什么：三层问题分解

2.1 第一层：GIL 与事件循环调度延迟（尾延迟的真正来源）

这是最容易被误读的部分。Habitat 是 I/O 密集型代理，asyncio 本该够用——问题不在“慢”，在“不确定”。

• Habitat 不只是转发请求，还承担大量 CPU 密集任务：路由计算、压缩、加密、校验和、下游健康检查、影子流量、请求对冲（hedging）

• Python asyncio 只提供 I/O 并发，GIL 阻断 CPU 并行。于是出现了一个反直觉现象：trace 显示 Azure Cosmos DB 早已返回结果，但协程因 CPU 被占、排队等事件循环重新调度才能解析响应——p99 尾延迟停滞在“等待调度”而非“等待数据”

• OpenAI 自建了事件循环调度延迟度量：周期性投放后台探针任务，比较“预期执行时间 vs 实际执行时间”。高负载下抖动达数百毫秒，极端时数秒——而此时常规监控（CPU/内存/网络/磁盘）全部显示“健康”

Rust 的解法：无 GIL，CPU 密集任务真正并行化，tokio 调度器纳秒级开销且可预期。尾延迟从概率问题变成确定性问题——这是 6x CPU 效率之外更本质的收益。

2.2 第二层：进程爆炸的连锁成本（Python 架构的“利息”）

Python 时代的应对策略是“每进程低并发 + 海量横向扩 worker”。这解决了调度延迟，但制造了新的系统性问题：

| 问题 | 机制 | 后果 | |---|---|---| | 惊群效应 | 海量进程同时向下游建连 | 一次日常部署的连接风暴即造成严重 CPU 抖动；一次连接泄漏可打满 NAT 网关，瘫痪整个网络 | | 配置风暴 | Statsig 每分钟无 jitter 轮询全量配置，每 pod 8 个 worker 同时解析巨型 JSON | 周期性尾延迟尖峰：所有 worker 同时停止处理在途请求去解析配置 | | 亚稳态故障（metastable failure） | aiohttp TCPConnector 默认 LIFO 连接复用：过载服务器连接更晚归还连接池 → 反而更优先被复用 → 收到更多请求 → 更慢 | 正反馈循环：最不该有流量的服务器获得最多流量，直至重启才能恢复 |

Rust 重写后，单进程高并发（多核并行 + 无 GIL），进程数量下降一个数量级，上述三类问题的触发门槛被整体抬高。值得注意的是：这三类问题中，jitter、FIFO 连接池、Envoy HTTP/2 扇入 + 集中式限流熔断，都是在 Python 阶段已经修复过的架构级问题——Rust 解决的是“架构修复后仍然存在的物理上限”。

2.3 第三层：资源摊销（为什么是“现在”必须重写）

Habitat 已是 OpenAI 按 CPU core 计第二大服务（Envoy 排第四）。在 10x 年增长曲线下，“继续加机器”是不可持续的：CPU 效率 6x、内存效率 15x 意味着同等硬件下容量天花板抬高 6–15 倍，这是唯一能跟上增长曲线的路径。

---

03 / 容易被忽略的前提：API 设计才是扩展性的根基

Rust 之前的 Python 服务能撑到 20M QPS，一半功劳属于“故意做得少”的 API 哲学：

• 受约束的 NoSQL API，拒绝任意 SQL。Postgres 时代的教训：团队规模一大，“写 SQL 便宜、跑 SQL 昂贵”的成本失衡导致一条热路径新查询就能打挂数据库

• 数据模型受 Meta TAO 论文启发：object + edge 图式结构，但不支持任意图遍历——对象及其边共置在同一存储分区，一跳之外的遍历可能跨两个地区的 Cosmos DB 账户。用图遍历效率换取水平分区的天然可扩展性

• 复杂查询的逃生通道：CDC 近实时流式同步到隔离的 Rockset 实例，各团队自己扩容自己的分析视图——把读密集的分析负载与在线存储彻底隔离，并让“昂贵操作”变成需要主动付费选择的显式决策

洞察：语言重写（Rust）优化的是“每次请求的成本”，API 约束优化的是“请求总量分布”——后者是常数级收益，前者是系数级收益。没有后者，前者只是让错误查询跑得更快。

---

04 / 深层洞察：AI 如何改写“技术债偿还时机”的决策函数

这是本案例对行业最有价值的部分。

传统决策模型：技术债偿还成本 = 工程师人数 × 时间 × 机会成本。在业务 10x 增长期，任何语言迁移都会挤占核心功能迭代，所以“永远不要在高增长期重写”是教条。

OpenAI 的做法是把这个决策拆成了三步时序：

1. 2025 年中：明知 Python 撑不住 100x 增长，仍主动接受这笔债——因为当务之急是解除产品阻塞、稳定平台、锁定 API 形态（架构未稳定前重写，等于把错误设计翻译成 Rust）

1. 债务持有期：不是躺平，而是把 Python 榨到极限——自建事件循环延迟监控、CPU profiling 定位配置风暴、LIFO→FIFO 打破亚稳态、Envoy 集中扇入。用基础设施工程对冲语言缺陷，为迁移争取一年时间

1. 偿还时机：押注自家模型（Codex + GPT-5.5）的成长曲线。一年后，迁移成本从“12 人团队半年”压缩到“2 人 + AI 完成全量重写”，且生产验证（95% 流量承接、无重大事故披露）

这改变了什么：技术债的贴现率被 AI 下调了。当“偿还成本”随时间递减（而非传统假设的递增），最优策略从“尽早偿还”变成“延迟偿还、持有期间榨干现有资产价值”。OpenAI 自己的总结很精确：“在现有技术栈中榨取最大价值，同时抵御容量危机，为基础性投资争取时间。”

需要泼的冷水（勿过度外推）：

• 6x/15x 是单一工作负载下的自报数据，未经独立验证；不能推出“所有 Python 服务改 Rust 都有 15x”——收益高度依赖“CPU 密集任务 + 超高并发 + 海量进程”这一特定画像。普通 CRUD 服务收益可能只有 1.5–3x

• “2 名工程师”的前提是他们深度理解系统全部故障模式（正是那一年 Python 排查积累的知识）。AI 压缩的是编码日历时间，不是问题理解时间

• 重写的安全性依赖 Habitat API 足够窄（约束式 NoSQL），接口面小才可能被 AI 高保真翻译

---

05 / 对我们的启示

1. 基础设施层是 AI coding 的最佳首战场：接口窄、测试可验证、行为有生产流量对拍（shadow traffic）——Codex 类工具在“约束式翻译”任务上的可靠性远高于开放式产品开发

1. 创业团队的技术栈选型：早期用最熟的技术换速度是对的，但要刻意保持 API 面窄、可测量、可替换——Habitat 的 Python→Rust 之所以可行，是因为 2023 年起就没把业务逻辑和存储抽象耦合

1. 尾延迟监控盲区：事件循环调度延迟、后台任务 jitter、连接池复用策略——这三类问题在标准监控里全部不可见，值得加入任何高并发 Python/Node 服务的观测清单

1. 对 Axiom 自己的注脚：本报告的资料收集与交叉验证正是由 AI 完成的——OpenAI 的赌注在我们这一侧同样在兑现

---

参考：OpenAI 官方博客（Scaling storage to serve over one billion users, Part One）· 钛媒体深度整理 · 稀土掘金 · ExplainX 分析 · Oddship Reading List 工程笔记 · Meta TAO 论文（USENIX ATC 2013）。注：所有性能数据均为 OpenAI 自报，第二篇文章将披露 Rust 迁移的更多经验教训与多租户可靠性细节，值得持续跟踪。

The Rust rewrite did not fix one performance problem. It removed three stacked costs: event-loop scheduling delay caused by the Python GIL and the tail latency it produced, the knock-on costs of scaling out with enormous process counts, and the resource bill of running OpenAI's second-largest service by CPU core. The more interesting move sits underneath all of that. OpenAI timed a strategic rewrite against the maturity curve of its own coding models, and two engineers plus Codex and GPT-5.5 finished a migration that classically would have taken twelve engineers six months. That changes when you should pay technical debt down.

---

01 / Habitat: the invisible engine nobody talks about

Talk about AI infrastructure and the attention goes to GPUs. Serving 1B+ weekly users usually breaks somewhere else: the data layer. Habitat is the unified access platform OpenAI built for online storage. Every product reads and writes through it — ChatGPT, the API, Codex, internal services. Schema lookup, routing, authorization, encryption, serialization, request shaping, connection pooling: all of it lives in this one layer.

| Metric | Value | | --- | --- | | Requests per second | 70M+ | | Users served weekly | 1B+ | | Total data | 500 PB+ | | Geographic regions | Nearly 40 | | Annual growth | 10x+, three years running |

The evolution path matters as much as the scale. Habitat shipped at the 2023 DevDay as little more than a Python client library for Azure Cosmos DB. By mid-2025 it had become a standalone Python service. In Q2 2026, two engineers rewrote it in Rust with Codex and GPT-5.5. It now carries 95% of production traffic, and Python was retired completely within weeks. At retirement, the Python version was peaking at 20M+ QPS — an engineering ceiling worth recording on its own.

---

02 / What Rust actually fixed: three layers

2.1 First layer: GIL, event-loop scheduling delay, and the real source of tail latency

This is the part most readers get wrong. Habitat is an I/O-heavy proxy, so asyncio should have been enough. The problem was never slowness. It was nondeterminism.

• Habitat does much more than forward requests. It carries real CPU work: routing computation, compression, encryption, checksums, downstream health checks, shadow traffic, request hedging.

• Python's asyncio buys you I/O concurrency; the GIL blocks CPU parallelism. That produced something counterintuitive: traces showed Azure Cosmos DB had already returned its response, while the coroutine sat in a queue waiting for the event loop to schedule it again before it could parse that response. p99 stalled waiting for scheduling, not waiting for data.

• OpenAI built its own measure of event-loop scheduling delay: periodic background probe tasks comparing expected execution time against actual execution time. Under high load, jitter reached hundreds of milliseconds and occasionally seconds — while CPU, memory, network, and disk metrics all reported healthy.

Rust's answer: no GIL, so CPU-heavy work runs truly in parallel, and the tokio scheduler has predictable nanosecond-level overhead. Tail latency stops being probabilistic and becomes deterministic. That is a deeper win than the 6x CPU efficiency number.

2.2 Second layer: the knock-on costs of process sprawl

Python's answer to scheduling delay was low concurrency per process and massive horizontal scale-out across workers. It solved one problem and created systemic ones.

| Problem | Mechanism | Consequence | | --- | --- | --- | | Thundering herd | Huge numbers of processes opening connections downstream at the same time | The connection storm from a routine deploy causes severe CPU jitter; a single connection leak can saturate the NAT gateway and take down the entire network | | Config storm | Statsig polls the full config set every minute with no jitter, and eight workers per pod parse one giant JSON blob simultaneously | Periodic tail latency spikes: every worker stops handling in-flight requests at once to parse config | | Metastable failure | aiohttp's TCPConnector defaults to LIFO connection reuse: connections to overloaded servers return to the pool later, so they get reused first, receive more requests, and get slower | A positive feedback loop — the server that should get the least traffic gets the most, until a restart breaks the cycle |

After the Rust rewrite, one process handles high concurrency through multi-core parallelism with no GIL, and process counts dropped by an order of magnitude. That raises the trigger threshold for all three failure modes. Read the sequence carefully, though: jitter fixes, FIFO connection pools, Envoy HTTP/2 fan-in, and centralized rate limiting and circuit breaking were all already fixed during the Python era. Rust addressed the physical ceiling that remained after the architectural problems were gone.

2.3 Third layer: resource amortization, and why now

Habitat is already OpenAI's second-largest service by CPU core, with Envoy fourth. On a 10x annual growth curve, adding machines does not hold. 6x better CPU efficiency and 15x better memory efficiency raise the per-hardware-unit capacity ceiling by 6–15x, which is the only path that tracks that growth curve.

---

03 / The overlooked precondition: API design is where scalability comes from

The Python service reached 20M QPS before Rust existed in the picture, and half the credit goes to an API philosophy of doing deliberately less.

• A constrained NoSQL API, no arbitrary SQL. The lesson came from the Postgres era: writing SQL is cheap and running SQL is expensive. Once the team grew, one new query on a hot path could take the database down.

• A data model borrowed from Meta's TAO paper. Objects and edges in a graph shape, without arbitrary graph traversal. An object and its edges sit in the same storage partition, so walking more than one hop can cross two regional Cosmos DB accounts. Traversal efficiency was traded for natural horizontal partitionability.

• An escape hatch for complex queries. CDC streams data near-real-time into isolated Rockset instances, and each team scales its own analytical views. Read-heavy analytical load is fully separated from online storage, and an expensive operation becomes an explicit decision someone chooses to pay for.

So: rewriting the language optimizes the cost per request. API constraints optimize the distribution of total requests. The second is a constant-order gain, the first a coefficient-level gain. Without the second, the first just runs the wrong queries faster.

---

04 / The deeper insight: how AI rewrites the decision function for technical debt

The traditional model prices debt repayment as engineers × time × opportunity cost. Under 10x business growth, any language migration competes with core feature work, so "never rewrite during hypergrowth" became dogma.

OpenAI split the decision into three timed steps.

1. Mid-2025 — take the debt on purpose. They knew Python would not survive 100x growth. The priority was unblocking products, stabilizing the platform, and locking the API shape, because rewriting before the architecture settles only translates the wrong design into Rust.

1. The holding period — nothing passive about it. They pushed Python to its limit: custom event-loop latency monitoring, CPU profiling to find the config storm, LIFO to FIFO to break metastability, Envoy for centralized fan-in. Infrastructure engineering hedged the language's defects and bought a year.

1. The repayment moment — a bet on their own models. One year later, with Codex and GPT-5.5, migration cost compressed from "12 engineers, six months" to "2 engineers plus AI rewrote the whole thing," with production validation behind it: 95% of traffic served, no major incidents disclosed.

What changed is the discount rate. When the cost of repayment falls over time instead of rising, the optimal strategy shifts from paying early to paying late — extracting everything the current asset can give you while you wait. OpenAI's own summary is precise: "maximizing value out of the existing stack while fending off capacity crises, buying time for foundational investments."

Now the cold water, before anyone extrapolates too far:

• The 6x and 15x figures are self-reported numbers from a single workload, not independently verified. They do not imply that every Python service gets 15x in Rust. The gain depends on a very specific profile: CPU-heavy work, extreme concurrency, enormous process counts. A typical CRUD service might see 1.5–3x.

• Those "2 engineers" started with deep knowledge of every failure mode in the system, acquired during that year of Python debugging. AI compressed calendar coding time, not comprehension time.

• The rewrite was safe because Habitat's API is narrow — constrained NoSQL with a small surface area — which is what allows high-fidelity translation by AI at all.

---

05 / What it means for your team

1. Infrastructure is the best first battlefield for AI coding. Narrow interfaces, tests that verify behavior, production traffic to diff against — shadow traffic turns a rewrite into a checkable translation task, which is where Codex-class tools are far more reliable than they are on open-ended product work.

1. Pick your early stack for speed, but keep it exchangeable. Starting with what your team knows best is right. The habit that paid off here was keeping the API surface narrow, measurable, and replaceable: since 2023, business logic was never coupled to the storage abstraction.

1. You probably have the same tail-latency blind spot. Event-loop scheduling delay, background task jitter, connection pool reuse policy — none of these show up in standard dashboards. Add them to the observability checklist of any high-concurrency Python or Node service.

1. A note on this report: the source collection and cross-checking behind it were done by AI. The bet OpenAI placed is paying off on our side of the table too.

---

Sources: OpenAI's official blog (Scaling storage to serve over one billion users, Part One), TMTPost in-depth coverage, Juejin, ExplainX analysis, Oddship Reading List engineering notes, and Meta's TAO paper (USENIX ATC 2013). All performance figures are self-reported by OpenAI and not independently verified. A second post will cover more lessons from the Rust migration along with multi-tenant reliability details, so this thread is worth following.