OpenAI’s Jalapeño Chip Beats Nvidia Blackwell on Speed and Efficiency in First Tests

OpenAI has published the first measured performance results for the OpenAI Jalapeño chip, its custom AI inference processor built with Broadcom, and the numbers show a clear lead over today’s leading Nvidia systems. Tested on the public SemiAnalysis InferenceX benchmark, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency, with even larger gains on interactive workloads. The chip is still an engineering sample, but it signals how OpenAI plans to make powerful AI faster, cheaper to serve, and more widely available.

What Is OpenAI’s Jalapeño Chip?

Jalapeño is OpenAI’s first custom inference processor, unveiled in partnership with Broadcom in late 2025 and presented in detail at Hot Chips on August 25, 2026. Unlike a general-purpose GPU, it is an intelligence processor designed from the ground up around large language model inference, especially for interactive agents that need to complete many steps in sequence.

OpenAI describes Jalapeño as the first generation of a multigenerational platform. The company designed models, serving software, memory, networking, and the rack-scale system together, using insights from its own real workloads to tune every layer. That full-stack control is a key part of the strategy.

• Purpose-built for inference: Jalapeño is optimized for serving language models, not training them. It handles the two main inference phases, prefill (processing the prompt) and decode (generating tokens), within one balanced architecture.
• Built with Broadcom: Broadcom co-developed the silicon, with engineering samples now running machine learning workloads in the lab at production target frequency.
• First-party silicon: OpenAI calls it working first-party silicon with measured results, not a concept or simulation. Production qualification and software maturation are underway.
• Flexible across models: The architecture is designed to work with all large language models, not only OpenAI’s own. Tests included models from outside OpenAI to prove that point.

That flexibility matters because the AI hardware market is crowded. For context on how other custom accelerators are evolving, see our coverage of Alibaba’s RISC-V chip that runs a 27B model without a GPU.

First Benchmark Results on InferenceX

To measure Jalapeño, OpenAI turned to InferenceX, a public benchmark from SemiAnalysis that evaluates the full process of serving an AI request, including throughput, latency, interactivity, and energy use. OpenAI compared Jalapeño against leading commercially available systems across the tested operating range, from high-throughput batched serving to highly interactive low-latency use.

Crucially, OpenAI measured performance at a matched user experience, asking how much useful work each system can complete per unit of power while meeting the latency that customers and agents require. The company also normalized results by each accelerator’s published package power to compare performance per watt, which it argues is more useful than per-chip metrics for power-constrained data centers.

• Models tested: GPT-OSS 120B (OpenAI), DeepSeek R1 670B, and Kimi K2.5 1T (Moonshot AI)—spanning 120 billion to 1 trillion parameters.
• Workload shape: Nominal 8,000-token input and 1,000-token output, single-turn, single-token prediction, without speculative decoding or prefill-decode disaggregation.
• Power rating: Jalapeño is rated at 700 watts package TDP, compared with 1,200 watts for Nvidia GB200 and 1,400 watts for GB300 in these comparisons. Measured sustained power stayed at or below 550 watts on the workloads tested.
• Result: Jalapeño sat on the Pareto frontier across all three models, meaning it offered a better combination of throughput per kilowatt and latency than any comparison point.

Performance Highlights Across Three Models

Across the three public models, Jalapeño delivered 1.5 to 1.9 times higher peak throughput per kilowatt and 1.7 to 3.6 times lower end-to-end latency. For highly interactive workloads, the advantage widened to 2.1 to 4.1 times higher performance. On the largest model tested, Kimi K2.5 1T, Jalapeño delivered about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the GB300 comparison system. OpenAI said the advantage widened further on its internal frontier models, suggesting the architecture scales better as workloads grow.

Jalapeño vs. Nvidia GB200 and GB300: Head-to-Head Numbers

The headline comparison pits Jalapeño against Nvidia’s Blackwell-generation GB200 and GB300 systems, which held the previous best results on InferenceX. While Nvidia’s newer Vera Rubin platform is already shipping to customers, OpenAI’s published comparison used Blackwell, which SemiAnalysis itself flagged as an imperfect but available baseline.

The appendix in OpenAI’s report breaks out peak and matched throughput using package TDP. At matched decoding speed, where interactivity is held equal, Jalapeño dramatically outperforms Blackwell, and at its own peak it still leads on efficiency.

An even starker figure appears when throughput is matched at the previous best interactivity level. At 535 tokens per second per user on GPT-OSS 120B, Jalapeño delivered 22,935 mixed tokens per second per kilowatt versus 427 for GB200—a 53.7 times advantage. On DeepSeek R1 at 169 tokens per second per user, the gap was 104.3 times. Those extremes reflect how much more headroom Jalapeño retains when latency is constrained, which is exactly where agent workloads live.

Why Jalapeño Avoids the Throughput-Latency Tradeoff

Existing hardware often forces a choice: tune for high throughput with large batches, or tune for low latency for interactive use, but not both. Jalapeño was designed to excel at both by minimizing data movement and communication delays across the full system.

Language model inference has distinct bottlenecks. Prefill is compute-intensive as the system ingests the prompt, while decode is memory-bandwidth bound as it generates one token at a time. Communication overhead can leave cores idle while data moves between chips. A system optimized for one phase can lose its edge waiting for the other.

• Locality by design: Model state, including the key-value cache used during generation, can be explicitly placed and kept local. The system then activates the right mix of compute, memory, and networking for each inference phase without shuffling state between resources.
• Network as architecture: The interconnect is integral, not an add-on. Its large domain lets the entire workload stay within one connected system across racks, reducing data movement from start to finish.
• Balanced and fungible: Rather than splitting prefill and decode into separate pools, Jalapeño uses a homogeneous pool that adapts as the balance shifts. OpenAI argues this is more resilient as workloads move from knowledge retrieval to reasoning to agentic tasks.
• Power efficiency as economics: In power-limited data centers, throughput per watt translates directly to revenue and capacity. Producing more useful work per watt lets OpenAI serve more demand without proportionally higher energy cost.

Under the hood, details shared via SemiAnalysis fill out the picture: a reticle-sized die on TSMC N3P delivering 13.4 PFLOPs of MXFP4 compute, HBM4 memory delivering 15.4 terabytes per second of bandwidth at 10 Gbps per pin, and a 2,048-XPU scale-up domain linking racks with 600 GB/s local and 200 GB/s global fabric. At 700 watts, Jalapeño achieves the highest HBM bandwidth per watt and FLOPs per watt of any tested accelerator to date, without using multi-token prediction. Competitor systems in the comparison used their best multi-token configurations where available, making Jalapeño’s single-token lead more notable.

AI Designed the Chip and Now Programs It

One of the most unusual parts of the Jalapeño story is how much AI itself contributed to building it.

OpenAI said AI helped move the project from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize arithmetic circuits, allowing the team to pack more compute performance into the die on schedule.

Equally important is how Jalapeño was designed to be programmed. The chip exposes a clear, predictable target based on local tensors, explicit communication, and predictable synchronization. That structure gives both human engineers and AI models a tractable way to handle parallel programming, which is traditionally difficult to optimize.

• Rapid bring-up: Using Codex with GPT-Astra, the team brought three open-weight models that were not part of the original production plan to high performance within two months, showing the architecture’s flexibility.
• AI-written kernels: For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than human-expert-written versions. Those gains apply to specific blocks, not the full model, but point to a new development loop where models optimize their own runtime.
• Tapeout speed: From design start in mid-2024 to CoWoS tapeout in November 2025 was about 16 months, a pace SemiAnalysis called extremely fast for an ASIC of this scale. After the chips returned from fabrication, it took just three months to get them running stably and produce the benchmark results.

The same dynamic that is reshaping chip design broadly is visible here—see how Nvidia says AI turned a 10-month GPU design task into an overnight run—but OpenAI’s claim goes further by having AI actively program the deployed accelerator.

When Will Jalapeño Reach Data Centers?

Jalapeño is not yet a fleet. OpenAI stressed that the chip is working first-party silicon with measured results, but it is still in production qualification with software being matured for scale.

• Late 2026: OpenAI plans to begin deploying Jalapeño within its own compute infrastructure by the end of the year, in very small volumes, according to hardware chief Richard Ho.
• 2027 ramp: More significant deployment comes in 2027, with most volume scheduled for the end of that year.
• Roadmap: Generation 2 is already deep in development, and Generation 3 is taking shape, each building on lessons from real serving workloads.
• Coexistence with Nvidia: OpenAI said it will continue to widely deploy accelerators from Nvidia and other partners for both training and inference, because meeting growing demand will require compute from every available source.

What changes for users when Jalapeño scales is both cost and experience: ultra-fast-mode inference at efficiencies previously only possible in fast mode, fast-mode inference at batched-mode efficiencies, and higher efficiency for batched mode overall. In practical terms, that means faster responses, more responsive agents, and more reliable access under load. Samsung may play a growing role in that ramp, as detailed in our coverage of the Samsung Mistral AI partnership and its OpenAI and Anthropic chip deals.

What the Benchmarks Do Not Yet Prove

The first results are strong, but they come with caveats that OpenAI and SemiAnalysis both acknowledge. Understanding the limits is essential before extrapolating to real-world fleet performance.

• OpenAI ran the tests: SemiAnalysis verified the InferenceX runs in person in the lab but did not execute the full benchmark suite itself. All headline figures originate from OpenAI.
• Narrow workload shape: Results cover 8k input / 1k output single-turn requests with single-token prediction. Longer contexts, multi-turn conversations, and AgentX workloads that stress prefix caching and routing were not included.
• Blackwell is not Rubin: The comparison is against GB200 and GB300. Nvidia’s Vera Rubin generation, which is now shipping, uses HBM4 and delivers higher per-die performance. Any lead over Blackwell will be tested again against Rubin at scale, and Rubin commonly benefits from speculative decoding that reduces cost per token.
• Lab versus fleet: Data center performance depends on reliability, yield, networking at thousands of accelerators, and software maturity over months, none of which a lab benchmark captures. GSM8K correctness was reported as on par with Nvidia, but broader evals are still pending.
• Single metric framing: Tokens per watt and latency are critical, but total cost of ownership, availability, and the speed at which new model architectures can be supported will decide adoption. OpenAI’s advantage partly reflects Broadcom’s cost structure versus Nvidia’s margins, though execution history matters—prior in-house chip efforts at other hyperscalers have stalled despite similar cost logic.

Taken together, Jalapeño has cleared the first hurdle convincingly: an untuned, single-token implementation beating tuned Blackwell competition on both efficiency and latency. The next hurdles—AgentX results, independent third-party benchmarking, and deployment at scale—will determine whether the lab lead becomes a fleet advantage.

Frequently Asked Questions

What is OpenAI Jalapeño?

Jalapeño is OpenAI’s first custom AI inference chip, co-developed with Broadcom. It is an intelligence processor optimized for serving large language models, designed to deliver higher throughput per watt and lower latency than general-purpose GPUs by co-designing the chip, memory, network, software, and rack system around real inference workloads.

What chips is OpenAI using now?

OpenAI continues to run large-scale training and inference on Nvidia GPUs, including Blackwell-generation systems, alongside Jalapeño engineering samples. Jalapeño is inference-only and will initially complement, not replace, Nvidia hardware, with early deployment inside OpenAI infrastructure starting late 2026.

How does Jalapeño compare to Nvidia Blackwell?

On SemiAnalysis InferenceX across GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño delivered 1.5 to 1.9 times higher peak throughput per kilowatt and 1.7 to 3.6 times lower end-to-end latency than GB200 and GB300 systems. On interactive workloads the gain was 2.1 to 4.1 times, and interactivity at concurrency 1 reached about 700 to 1,400 tokens per second per user depending on the model.

How much power does Jalapeño use?

Jalapeño is rated at 700 watts per package—roughly half the 1,200 to 1,400 watts of the Nvidia Blackwell systems it was tested against. In testing it actually drew even less, staying at or below 550 watts. OpenAI highlights performance per watt because, in power-constrained data centers, that is what determines how much useful AI work you can deliver within a fixed power budget.

How long will AI chips like Jalapeño last?

Accelerator generations typically turn over every one to two years, but Jalapeño is positioned as a multigenerational platform. OpenAI says Generation 2 is deep in development and Generation 3 is taking shape, with each generation building on lessons from the previous. Longevity will depend on how quickly model architectures and serving patterns evolve, which is why OpenAI favored a fungible, homogeneous design over fixed prefill-decode splits.

Will Jalapeño be available outside OpenAI?

OpenAI has not announced external sales of Jalapeño. Initial deployment is within its own infrastructure, and broader ecosystem availability would depend on future product decisions. In the near term, the benefit to external users comes indirectly through faster, more efficient OpenAI services.

Why Jalapeño Matters Beyond the Benchmark

Jalapeño is more than a spec sheet. It is a test of whether designing the full stack together—models, serving software, memory, networking, and silicon—can break the historic tradeoff between doing more work and responding quickly. If the lab lead holds at scale and across agentic workloads, the savings compound: more work per watt, lower latency per interaction, and lower cost per successful task. Those are the levers that turn a faster chip into more capable products for more people, which OpenAI frames as central to making advanced AI broadly affordable.

At the same time, the broader infrastructure race is accelerating. Competitors are advancing their own accelerators, interconnects, and data center power strategies, and Nvidia’s Rubin ramp changes the baseline before Jalapeño reaches volume. The most telling comparison may not be Jalapeño versus Blackwell today, but Jalapeño at scale versus whatever ships next when Generation 2 arrives.