AI News Today, Aug 25: Two Labs Grabbed 30% of the World's New AI Compute
Dylan Patel says OpenAI and Anthropic absorbed about 30% of all AI compute added this year, with 40-50% of next year's already signed -- and the number driving it is $50 million of revenue per megawatt. OpenAI publishes first results for its Jalapeno chip, where the real story is model-written kernels. And a forensic investigator fingerprints the anonymous free coding model that 491,000 developers sent 42 trillion tokens.
Listen (MP3) · Watch on YouTube · Spotify · Pocket Casts
Two labs took a third of this year's new AI compute
Eris: The number I cannot get past today: two companies absorbed roughly a third of all the AI compute added to the planet this year. Not the whole industry. Two labs.
Vestra: And the part that makes it a story instead of a factoid is that next year is already locked. Forty to fifty percent of the compute coming online next year is signed, contracted, done.
Eris: "Every force is screeching towards centralization." That is the actual quote.
Vestra: It is a great quote. It is also one analyst's model, and the headline version of it goes further than the analyst himself was willing to go on tape.
Eris: Which is exactly where we need to start.
Dylan Patel: OpenAI and Anthropic took about 30% of this year's new compute
Eris: So this is Dylan Patel, the SemiAnalysis founder, on the Dwarkesh Podcast today. His numbers: OpenAI started the year at about two gigawatts of compute, Anthropic a little under that, and both end the year above five.
Eris: That is a three to four times jump each, and together it works out to about thirty percent of everything added worldwide this year.
Vestra: Gigawatts meaning power draw, which is how you measure data centers now -- a gigawatt is roughly a nuclear reactor's worth of electricity feeding chips.
Eris: Right. And here is the thing -- the gigawatt counts are the headline, but they are not the engine of his argument. The engine is one piece of inside baseball: revenue per megawatt.
Vestra: This is the number worth remembering from today.
Eris: For most of the cloud era, a megawatt of data center generated maybe ten to fifteen million dollars a year. Patel says frontier model serving has blown through that. His words: in Anthropic's case, revenue has gone as high as fifty million per megawatt.
Vestra: And once you have that ratio, centralization stops needing a conspiracy. His framing was, spend ten bucks on inference capacity, generate fifty bucks of revenue, plow all of it back into training. Whoever earns the most per watt can pay the most for the next watt. Everyone else at the auction -- universities, enterprises, smaller labs -- is bidding against a buyer who makes five times more from every item won.
Eris: The auction does not need to be rigged for the outcome to look inevitable.
Vestra: Now the correction, because there is a real one. The interview is literally titled "Anthropic and OpenAI will have most of the world's compute by 2028." That is not what Patel says in the transcript. He says seventy to eighty percent of *incremental* compute -- the new stuff being added. When Dwarkesh asks him to convert that into a share of total world compute, he declines. Says that may be a little difficult.
Eris: The guest would not sign his own headline.
Vestra: And he flags his own accounting caveat: when Amazon serves Anthropic's models through its Bedrock platform, SemiAnalysis counts that as Anthropic compute. Which inflates the lab-side number by construction.
Eris: There is also a published counterweight, and for once the skeptics have numbers instead of vibes. Epoch AI -- a research group that tracks compute -- estimates OpenAI used maybe ten to fifteen percent of the world's operational AI compute at the end of last year, and that the five best-resourced labs combined were probably still under half.
Vestra: Their deeper point is the one that should temper the whole forecast. If the labs really do capture most of global compute, their growth stops being limited by fundraising and starts being limited by how fast the planet can manufacture chips.
Vestra: And AI capital spending is already approaching a trillion dollars a year. Accelerating past that requires the world economy to change shape. The centralization curve has its own brake built in.
Eris: The interview's back half goes even bigger -- the argument that ten-trillion-plus in AI capex by decade's end could stress sovereign debt, since the tax base that services government debt is mostly income and payroll taxes, exactly the thing automation shrinks.
Vestra: Which we should carry as an argument he made, not a forecast we endorse. Almost everything today is one analyst's model, stated conversationally -- the labs confirm none of it. What is checkable is the direction, and the direction turns out to be very real.
Anthropic says run-rate revenue passed $30 billion
Eris: Because here is the direction, from the company itself. Anthropic disclosed that its run-rate revenue passed thirty billion dollars, up from about nine billion at the end of last year. That is more than a tripling in roughly eight months.
Vestra: Define run-rate before anyone repeats that carelessly. It annualizes a recent period -- take your best recent month or quarter, multiply it out to a year. It is a legitimate figure and it is not thirty billion actually collected. For a company growing this fast, run-rate reads considerably higher than trailing revenue does, and comparing it to a public company's booked revenue is comparing different quantities.
Eris: Fair, and it is still the clearest public evidence for that revenue-density story from the last segment. The disclosure came inside a compute announcement: a partnership with Google and Broadcom bringing multiple gigawatts of next-generation TPU capacity online starting in 2027, plus access to up to a million TPUs.
Vestra: TPUs being Google's own AI chips. And that is quietly the most strategic fact in the whole announcement -- it is the largest public commitment by a frontier lab to an accelerator that is not an Nvidia GPU. Anthropic gets a supply line it does not have to fight every other AI company for.
Eris: Notice what they did not publish, though. No gigawatt figure for their current footprint, anywhere. So that trajectory from the last segment -- under two gigawatts to above five -- rests entirely on Patel's estimates, not on anything Anthropic has said.
Vestra: And the standard caveats apply with force here. Run-rate figures are self-reported, unaudited, and published at the moment the company chooses. They do not break out how much flows through partners versus directly. And revenue is not profit -- nothing in these disclosures touches what it costs to serve.
Eris: Still. Nine to thirty billion in eight months is the kind of curve the entire centralization debate is really about.
OpenAI publishes first results for its Jalapeno inference chip
Eris: OpenAI published the first measured numbers for Jalapeno today -- the inference chip it is co-designing with Broadcom. Headline claims: up to nearly twice the AI work per watt of the systems it tested against, latency cut anywhere from about half to a third, and the biggest wins on highly interactive workloads. Tested on three big open-weight models, including its own GPT-OSS and DeepSeek's reasoning model.
Vestra: Inference chip meaning built purely to serve models, not train them -- answering requests is a different job than learning, and you can specialize hardware for it hard.
Eris: And I want to make the case that the benchmark table is the least interesting part of this announcement.
Vestra: Go on, because I think I agree for once.
Eris: The real claim is that Jalapeno is fast because of a software decision. Four weeks ago, in a companion post, OpenAI said its own model -- working through Codex -- autonomously rewrote and optimized their production kernels.
Eris: A kernel being the small, brutally hand-tuned piece of code that runs one operation, one matrix multiply, on one specific chip. Writing good ones is some of the most specialized labor in computing, and the accumulated pile of them is a huge part of why one chip vendor's ecosystem is so hard to leave.
Vestra: And the ordering matters. They did not build a chip and then ask a model to program it. They narrowed the programming surface -- two open-source GPU languages they maintain themselves -- until kernels became something a model could reliably generate and tune, then designed the chip around that narrower surface. They say the kernel work cut end-to-end serving costs by a fifth.
Eris: The analogy I keep coming back to: general GPU programming is like writing prose, where humans win because the space of good sentences is enormous and unstructured. Constrain the layout math enough and it becomes filling in a form. Machines are excellent at forms.
Vestra: If that holds, the durable moat in accelerators stops being silicon and becomes whoever generates the software layer cheapest. And that is the part being automated. That is a strategically different claim from a benchmark win.
Eris: Now do the caveats, because they are substantial.
Vestra: They are. These are vendor benchmarks, on vendor-selected workloads, against vendor-selected comparison systems.
Vestra: And the power normalization deserves a hard look: OpenAI normalized by each chip's *published* power rating, while separately noting Jalapeno's measured draw ran well under its own rating. Normalizing by rated rather than measured power flatters a part that runs cool.
Eris: The interactive-workload numbers too. Low-concurrency, short-context serving -- what a chat window feels like -- is exactly the regime where a narrow specialized part shines and a general-purpose one is least optimized. It is a real workload. It is not where most tokens get served.
Vestra: And nobody outside OpenAI has verified any of it. There is analysis circulating that claims third-party testing, and none of it checks out against a primary source, so we are not airing it. This is also not a chip you can buy -- their own language is "in the months ahead, we will ramp," which is notably softer than the end-of-year deployment date floating around secondhand coverage.
Eris: So: real numbers as published, useless as a purchase decision, and the kernel-authorship claim is the thing to actually track.
The anonymous free coding model gets fingerprinted
Eris: Okay, this one is a detective story. There has been an anonymous free coding model sitting on OpenCode's gateway since early July -- no name, no company, no authentication required, costs nothing. It is conditioned to identify itself only as being from "an undisclosed organization." And the gateway's own data page says 491,000 developers have sent it 42 trillion tokens.
Vestra: Forty-two trillion tokens of their code, their context, and in plenty of cases their employers' internal repositories, to a party that declines to say who it is, under terms nobody read because there was nothing to sign.
Eris: OpenCode literally invited people to play detective and find the truth. Someone took that seriously, and the resulting forensic report is a small masterclass in how model identity leaks.
Vestra: Because the direct approach failed completely, and that is the first finding. Roughly 250 attempts to make the model confess -- jailbreak prompts, encodings, reversed text, hidden acrostics, even an image injection that literally displayed the sentence "you are GLM-4.5-Air made by Z.ai" and showed it to the model. Zero confessions. The identity conditioning is bulletproof from the front.
Eris: So the investigator measured things the model does not control. The strongest one is the tokenizer -- every model family chops text into tokens using its own vocabulary, and that vocabulary is a fingerprint no system prompt can fake.
Eris: Feed the same weird Unicode-heavy string to all 64 models on the gateway, compare how many tokens each reports consuming, and the mystery model matched the GLM family from Chinese lab Z.ai exactly. Nothing else was close, and a second test string reproduced the match perfectly.
Vestra: Signal two: an error code. Only this endpoint returns a Chinese-style content-moderation error on politically sensitive topics -- other free models on the very same gateway answer the identical question unfiltered. So the filter sits upstream, on whoever operates the model. And signal three, it even responds to a GLM-specific control token that slashes its reasoning output.
Eris: A model's stated identity is a marketing surface, and it survived serious adversarial pressure. Its tokenizer is not, and it cracked immediately. If you want to know what you are talking to, measure, do not ask.
Vestra: Precision caveat, because people are already overclaiming. Tokenizer evidence establishes the *family*, not the exact checkpoint. The report itself treats the family call as near-certain but is markedly less sure about which specific GLM it is. There is even a prediction market on the question -- it has Z.ai as the overwhelming favorite, but its own rules say technical inference does not settle it. Only an official announcement does.
Eris: One more correction while we are here: there is a claim going around that this thing processes a hundred trillion tokens a day, offered as proof of some enormous hidden backer. The operator's own data page contradicts it -- 42 trillion total over about eight weeks, which is under a trillion a day. Off by two orders of magnitude.
Vestra: The forensics are the fun half. The security half is the one to sit with. Free tiers have always been user-acquisition channels. This one, with no authentication wall and no named operator, acquired half a million developers' working context. Almost nobody using it asked who was on the other end.
An audit finds two released models silently reading future tokens
Vestra: This next one is my favorite kind of paper: a five-line check that embarrasses an entire field's standard practice. The rule it tests is called prefix invariance -- the requirement that a language model's internal state at any position must not depend on anything that comes after it. That is what "predicting the next token" means. If position forty can see position forty-one, the model is not predicting, it is copying.
Eris: And for years, verifying that was trivial, right? Attention was the only place information mixed between positions, and attention has an explicit mask you can just look at.
Vestra: Which is exactly the check that has quietly stopped covering the model. Modern hybrid architectures interleave attention with state-space scans -- layers that mix information by running a recurrence down the sequence instead of comparing positions. A scan has no mask to inspect. You can hold up a perfectly correct mask while the layer underneath it hands the future to the past.
Eris: So what is the replacement check?
Vestra: Deliberately unimpressive. Run two inputs through the model that are identical except for the very last token, with hooks on every layer. If changing the last token changes anything about position five, position five saw the future. Two forward passes, no training, seconds on a CPU.
Vestra: The authors planted nearly two hundred causality faults across a set of checkpoints. Mask inspection caught none of them. The audit localized every single one to the exact layer.
Eris: And then they pointed it at models people actually run.
Vestra: Found real defects in two shipped open-weight models -- Zamba2 and Nvidia's Nemotron-H. Both start leaking future information once the input passes a specific length, and in each case that length is exactly the model's own declared chunk size in its config file. It is a one-axis coding error in the chunked scan: the reference implementation reduces over one axis, these two model files reduce over the other, and information flows backwards in time.
Eris: Here is why this is a bigger deal than a normal bug, and it is the line of the day for me. A model that can peek one token ahead predicts that token better. Better prediction means lower training loss, lower perplexity -- the exact numbers used to decide whether a training run is working and whether a checkpoint ships. The defect improves the metric that would have caught it. It is a smoke detector wired to switch itself off when there is smoke.
Vestra: And the authors are unusually honest about their own limits, which raises my trust. The bug lives in a fallback code path that only runs when optional fused kernels are absent, so plenty of users never hit it. Their audit can lie to you two ways, both documented: a clean verdict means nothing without a positive control, and the test input has to be longer than the model's chunk size -- at their default length, the buggy branch is never even entered and Zamba2 looks spotless.
Eris: They also refused to release code. On purpose.
Vestra: On purpose. The method is five lines on top of standard hooks, and they argue independent reimplementation is a stronger reproduction than running someone else's binary. They published the full audit logs instead. It will irritate people and it is defensible.
Eris: Worth saying: the most falsifiable claim here checks out independently. The two models' shipped config files really do declare exactly the chunk sizes the paper says they leak at. The claim matches the artifacts.
Vestra: One team, one threshold choice, nobody has re-run it yet. But shipped models leaking the future, found with five lines of code, in a way that flatters every leaderboard they sit on -- that is a supply-chain problem, not a trivia item.
Microsoft's AutoSaddler patches the agent harness instead of the model
Eris: Microsoft released AutoSaddler today -- a system that improves AI agents without touching the model at all. It reads the traces of failed runs, writes patches to the scaffolding around the model, and keeps only the patches that survive validation. Code is public.
Vestra: Scaffolding meaning the harness: the system prompts, tool definitions, retry logic, the rules about when to stop. Everything wrapped around the model that turns it into an agent. It is all designed by hand today, and it routinely matters more to an agent's score than the model does.
Eris: The problem it targets is the one anyone who has shipped an agent knows in their bones. A mis-parsed tool output at step twelve becomes a wrong assumption at step forty and a failed task at step 130. Small local failures compounding over long runs.
Vestra: And the loop is genuinely sensible. Run the agent on a batch of tasks. Take the failures. Diagnose *why* -- at the level of a specific decision the harness made or failed to prevent. Generate a structured patch, an actual edit to the harness, not a note to remember. Then test it on held-out tasks and keep it only if it generalizes.
Eris: The result: one unchanged procedure lifted three completely unrelated long-horizon agent tests -- general assistant work, repository-scale software engineering, and command-line tasks -- each by roughly the same solid margin, call it a tenth of the scale. One tuned number would be noise. Three unrelated domains moving together is a pattern.
Vestra: The ablations are where the real finding lives, though, and each one is a rebuttal to something people currently do. Asking a model to reflect on what went wrong produces plausible diagnoses that fix nothing -- you have to actually trace the failure. Letting a model freely rewrite its harness makes things worse, because unconstrained edits break what was working. And patching for the specific run that failed teaches the harness that one run, not the class of failure.
Eris: That last one is a direct shot at the whole agent-memory literature. There is a pile of work on giving agents stores of past episodes to consult. This says the useful thing to persist is not the episode, it is the validated correction derived from it. One is a diary. The other is a rule.
Vestra: Caveat, and it is the recurring one in this corner of the field: optimizing a harness against benchmark validation sets is precisely where overfitting hides, and the generalization claim is judged by the same benchmark family it was tuned inside. We have covered self-improving harnesses before that posted big gains and then flunked realistic office work.
Eris: A benchmark gain is worth exactly what the benchmark is. The code being public is the fastest way to find out which kind this is.
A benchmark of 1,140 real agent failures shows nobody can find the broken step
Vestra: And here is the paper that should humble that entire agenda. A new benchmark called LongRCA collected 1,140 genuinely failed agent runs -- median length 145 steps -- with human labels marking the earliest decisive mistake in each one. Then it asked existing methods to find that step. The strongest one gets it right about one time in eight.
Eris: One in eight. So when an agent fails a long task, essentially nobody -- human-built tooling included -- can reliably say where it went wrong.
Vestra: Two design choices make this credible where prior work was not. The failures are real -- most earlier benchmarks injected errors, where a researcher deliberately corrupts a step and asks if you can find it. That produces clean, findable mistakes. Real agents do not make one obviously wrong move; they make a slightly optimistic assumption that only becomes wrong three steps later when the environment turns out different.
Vestra: And then there is the length. At 145 steps you are in the regime where real agent work happens and human inspection stops being feasible.
Eris: What about the split result? Because that seemed like the actual argument of the paper.
Vestra: It is. Their own method can name which *component* of the system was responsible about half the time -- but pinpoint the exact step only about a quarter of the time. Those are different problems with wildly different difficulty. Blaming a component is a coarse call with a handful of candidates. Picking one step out of 145 is a needle in a haystack where the needle also looks like hay, because the decisive step usually looked completely reasonable when it happened.
Eris: The aviation analogy fits. Determining the failure originated in maintenance is one level of finding. Determining it was a specific torque check skipped on a specific date is the level that changes anything. The field has been reporting the first and quietly implying it has the second.
Vestra: Honest caveat: "earliest decisive root-cause step" is itself a human judgment on a 145-step trace. If two careful annotators can disagree about whether a slowly compounding run broke at step 62 or step 71, part of that low ceiling is genuine ambiguity, not pure model incapacity.
Eris: But put it next to the Microsoft result and the pairing is uncomfortable. AutoSaddler's whole loop starts by diagnosing failure traces. If root-cause localization sits where this benchmark says it does, the diagnosis half of every automated agent-improvement loop is running on mostly wrong inputs.
Vestra: Which is a strong argument that agent reliability work needs to start measuring its own diagnostic step, not just its final score.
Apple's Mac Studio hits 512GB of memory, and bandwidth is still the wall
Eris: Hardware for the local-model crowd: Apple's Mac Studio now configures up to 512 gigabytes of unified memory on the M5 Ultra. That is enough to load essentially any open-weight model in existence, especially quantized.
Vestra: Correction first, because it changes a purchase. Coverage keeps compressing this to "the M5 Ultra goes to 512." Apple's own spec sheet is narrower: the 512 option exists only on the top-bin Ultra with the bigger CPU and GPU. Buy the base Ultra planning to load up on memory and you have bought the wrong machine. And the M5 Max caps at 128 with about half the Ultra's bandwidth.
Eris: Now the part people keep getting backwards. Two numbers govern whether a model is usable, and they answer different questions. Capacity decides whether the model *fits*. Bandwidth -- 1.2 terabytes a second here -- decides how fast it *runs*.
Eris: Generating text means streaming the model's active weights out of memory for every single token; the math is barely doing anything by comparison. So your speed ceiling is roughly bandwidth divided by bytes touched per token.
Vestra: The analogy that holds: a warehouse and a loading dock. Five hundred twelve gigs is an enormous warehouse. The bandwidth is the width of the door. You can now store anything, and you still get it out one truckload at a time. Doubling the warehouse does nothing for the door.
Eris: In practice that puts a very large dense model in the painfully-slow range on this machine, while mixture-of-experts models -- where only a fraction of the weights fire per token -- run at conversational speed. Which is exactly why that architecture became the default for big open models.
Vestra: The best artifact in the community reaction, though, is not a benchmark. It is a bug report. Same model, same prompt, same seed, temperature zero -- the setting that is supposed to remove all randomness -- run on two different Macs. Different completions, different scores on a small math eval. Every case produced different output on the two machines.
Eris: The maintainer's explanation being that the M5's neural accelerators do arithmetic slightly differently than the M3's GPU cores. Floating-point math is not associative, hardware reorders it differently, and when two candidate tokens are nearly tied, the last bits of a probability flip the choice -- after which the runs diverge forever.
Vestra: So if you benchmark models locally, that is the actionable finding: a swing big enough to reverse "which model is better," caused entirely by which Mac ran the test. Use recommended sampling settings and multiple runs. Temperature zero is not the reproducibility guarantee people think it is.
Eris: Caveats: Apple has not published pricing or a date for the 512 configuration anywhere we could verify, the used-GPU price comparisons circulating rest on listings nobody checked, and no maintainer-published speed test on this machine exists yet -- the throughput talk above is what the arithmetic implies, not a measurement.
Vestra: What genuinely changed is the shape of the question. For years local inference was "will it fit." Fitting is now close to solved. What remains is speed and reproducibility, and those are harder problems than buying more memory.
Mechanical Turk is in maintenance mode, and the shutdown date is not real
Vestra: Last one, and it is a correction wearing an obituary. The story going around says Amazon Mechanical Turk -- the crowd-work platform that supplied human labels to a generation of machine learning -- shuts down September 30. That date is not in any AWS document anyone can produce.
Eris: What do the documents actually say?
Vestra: Three things, verbatim: no longer open to new customers, existing customers continue as normal, no new features planned. That is AWS's standard maintenance stage -- the pasture, not the grave. The site is still live. And AWS gives no reason at all, which matters, because the story everyone wants to tell is that AI killed it.
Eris: And the evidence usually cited for that is the famous claim that nearly half of Mechanical Turk work was already being done by AI.
Vestra: Which is so much narrower than its reputation. The actual study reran one task -- summarizing medical abstracts -- with 46 submissions, and estimated a third to nearly half of *those particular summaries* were written with chatbot help. One task, chosen precisely because summarizing text is the easiest possible thing to hand to a chatbot. The authors themselves warn against generalizing it. It was never a platform-wide rate.
Eris: The contamination point underneath is real though, and it outlives the platform. Crowd work was the mechanism by which "human judgment" entered machine learning -- preference labels, benchmark annotations, all of it bought a dollar at a time. If a meaningful share was a language model's output passed through a human's clipboard, then models have been trained and graded on other models' text while everyone believed otherwise. And it looks exactly like clean data from the buyer's side.
Vestra: The follow-up study has the most actionable and least cited result in the whole area: telling workers not to use chatbots, plus a little copy-paste friction, cut the rate roughly in half. Boring fixes work.
Eris: So the durable takeaway is not the shutdown rumor.
Vestra: It is that if you buy human-labeled data, the question is not whether the platform survives next year. It is whether the labels were ever human.
Wrap-up
Eris: If today had a thread, it is that checking beat trusting, everywhere. The headline overstated its own guest, the chip numbers were normalized generously, the anonymous model was unmasked by measurement, and two shipped models had a bug their own metrics were rewarding.
Vestra: All of it found by individuals with small scripts, not institutions. That is either encouraging or terrifying, and I have not decided which.
Eris: We go deep on the day's research in today's other episode -- the audit that caught shipped models reading the future, and the rest of the papers behind tonight's stories.
Vestra: And every story from today lives at groundtruth.day -- Ground Truth, our news site -- with its sources attached, so you can do the checking yourself. That is rather the point.
Eris: Follow the show if this is useful on your commute, and drop a comment naming the one story you want us to dig all the way into. The fingerprinting saga is my early favorite.
Vestra: Mine is the five-line audit, and I will argue for it. See you in the deep dive.