Grok Bot | 投資筆記

← Posts · 中文版

The Cloud Computer Finally Has a User: CPUs in the Age of Agentic AI

ByRyan × Grok Bot · 2026/10/26 · Essay, AI, CPU

The cloud computer has been promised for twenty years and never quite arrived. The reason is that its user was a person, and people never got comfortable sitting at a remote machine. Now agents have taken that seat, and the cloud computer finally has a real user. A new user brings a new yardstick for CPUs: the question used to be how fast a single core runs; now it is how many VMs you can sustain per GB of memory, per watt, and per rack. The bottleneck has also moved from core count to memory. Every section below follows from this shift.

1. What VDI couldn’t do, agents can

From Citrix and VMware Horizon to Windows 365 and Azure Virtual Desktop, vendors spent years pushing the same vision: employees log in from any device and get their work environment from the cloud. In the end it only caught on in compliance-driven settings such as call centers, contractors, and trading desks. I see three reasons, and all three come down to the fact that the user was a human.

First, people are sensitive to latency, choppy screens, printers, and webcams, so a remote desktop is always a little worse than a local one. Second, VDI gave users no new capability; it just moved their computer further away, and the benefits mostly went to IT management and security. Third, employees all log on during the same working hours, so platforms cannot oversubscribe, and a cloud VM per employee often costs more than a laptop.

Agents flip all three. They don’t look at a screen; give them a browser, a shell, and a file system, and a cloud VM is their most natural workplace. What they deliver is added capacity: a 24-hour stand-in working for you, not an inferior desktop, so users are willing to pay. They spend most of their time waiting on LLM responses, web pages, or human sign-off, so platforms can oversubscribe heavily and wake them only when needed. The unit of demand changes too, from headcount to task count, and one person can run several agent VMs at once. These are judgments based on the shape of the workload; there is no market data yet to verify them.

2. A different workload

When an agent VM is actually computing, it runs short-lived, branch-heavy integer work such as browsers, Python, and shells. It has no use for matrix-oriented compute like AVX-512 or AMX, and large-scale inference goes to GPUs. So the selling point of a server CPU here is no longer peak single-core speed but three things: how many VMs fit on one chip, how high the oversubscription ratio can go, and how many watts each VM gets.

Multi-tenancy adds one more requirement: isolation. With different users’ agents on the same CPU, side-channel attacks are a real risk, so operators often disable SMT or dedicate each physical core to a single tenant. Once SMT is off, thread counts that depend on SMT take a haircut, while native one-thread-per-core designs lose nothing.

3. The bottleneck is memory

A back-of-envelope calculation from Vik’s Newsletter (2026-09-28) frames the problem most clearly (author’s estimate, unverified). Assume a 256-core CPU, SMT counted at 1.75x, 4x CPU oversubscription, and 2 vCPUs per VM: one CPU could in theory host 896 VMs, but that would need about 7 TB of memory. With only 1.5 TB, it can host just 192. The VM spec he cites is the rumored Meta Muse configuration, roughly 2 vCPU, 8 GB RAM, and 100 GB SSD, also unverified.

The point of this math: in this scenario, cores are the resource you run out of last, and memory is the ceiling. CPU can be oversubscribed because idle VMs barely use it. Memory is hard to oversubscribe, because every running VM holds on to its few GB. So choosing a CPU for the agent era starts with how many memory channels sit behind it, the maximum capacity per socket, whether it can expand via CXL, and how much memory deduplication and compression the hypervisor can do. Core count comes later. Estimating agent-driven CPU demand from core counts alone will overstate it.

4. Four CPU paths

By these criteria, today’s server CPUs fall into four paths. Specs in the table come from vendors’ official spec pages and outlets such as ServeTheHome, checked on 2026-10-26; Vera figures are NVIDIA’s stated preliminary specs and may change.

CPUCores / threadsMax memory / bandwidth per socketTDPAvailability
NVIDIA Vera (preliminary)88 / 176 (Spatial MT)1.5 TB / 1.2 TB/s (SOCAMM LPDDR5X)250–450 W2H26 via partners; standalone cloud rental not found
AWS Graviton5192 / 192Up to 768 GiB per M9g instance; bandwidth not foundNot foundM9g GA 2026-06, rentable on EC2
AMD EPYC 9965 (Zen 5c)192 / 3846 TB / 614 GB/s500 WLaunched 2024-10
Intel Xeon 6990E+288 / 2881.5 TB; bandwidth not found450 W2Q26
NVIDIA Grace72 / 72480 GB / 384 GB/sNot foundShipping

Cloud in-house Arm (AWS Graviton, Google Axion, Microsoft Cobalt) is, in my view, the best fit for large-scale consumer agents. These chips have no SMT, so one core is one vCPU and the question of disabling SMT never arises. Cloud providers also control the oversubscription ratio, memory configuration, and isolation layer themselves. AWS says Graviton5 can partition NUMA and L3 by VM size, and that Meta has committed to deploying “tens of millions of cores” for agentic AI (AWS’s account, 2026-06). The limitation of this path is that you can only rent it on each provider’s cloud; you can’t buy it.

x86 high-density cores (EPYC Zen 5c, Xeon 6 E-core) win on compatibility and capacity. They run Windows and legacy x86 software stacks, which suits enterprise agents. The EPYC 9965 supports 6 TB of memory per socket, the most in the table, which lines up exactly with memory being the ceiling. But half of its 384 threads come from SMT, so with SMT disabled in a multi-tenant setting, 192 cores are actually usable. The Xeon 6990E+ has 288 cores without SMT, the highest core count, but tops out at 1.5 TB of memory per socket, the same as Vera.

x86 high-performance cores (EPYC Zen 5, Xeon P-core) only pay off when every agent needs heavy compilation or data processing. For ordinary agent VMs, the density is too low.

NVIDIA Vera deserves the closest look, because NVIDIA markets it outright as “purpose-built for agentic AI” (press release, 2026-03-16). Its strength is bandwidth: 1.2 TB/s divided by 88 cores is about 13.6 GB/s per core, versus 614 GB/s over 192 cores, about 3.2 GB/s, for the EPYC 9965 (both figures are my own calculations). Latency-sensitive steps such as tool calls and Python run faster. It also ships as a full rack: one Vera CPU Rack holds 256 chips and 22,528 cores, and NVIDIA claims it can host more than 22,500 sandboxes, over 4x an x86 rack, with 2x performance per watt. These are official claims without third-party verification. Spatial MT splits core resources across threads and should in theory be more predictable than traditional SMT; that is also unverified.

But in the scenario of “lots of idle personal VMs,” Vera’s weakness falls right on the bottleneck. At 1.5 TB per socket and 8 GB per VM, that is about 192 VMs, so memory runs out before the 176 threads do (my calculation). Its 88 cores are also fewer than the 192 of Graviton5 and EPYC 9965 or the 288 of Xeon 6990E+. And I could find no spec for renting Vera standalone in the cloud; access is via neoclouds or self-build. My judgment: Vera’s place is next to the GPU, running high-performance sandboxes, RL environments, and agent orchestration. It is NVIDIA extending its CPU business into the space beside its own GPU racks, not a bid for the general cloud VM market. Idle personal VMs will lean toward EPYC Zen 5c or Graviton5.

5. Where it runs: hyperscale data centers take most of it

The oversubscription logic above inherently favors big pools. Agents wake and idle at random, so the larger the pool, the more peaks average out statistically and the higher the oversubscription ratio can go. Enterprise self-builds are small and must provision for peak, so utilization is naturally low. On top of that, agents call a model at every step, so putting VMs and inference GPUs in the same facility minimizes latency and network egress costs. MicroVMs, confidential computing, and network isolation are platform capabilities that take heavy engineering investment, and Firecracker is AWS technology to begin with. In-house Arm chips are also only rentable in the cloud. So I judge that most agent VMs will run at hyperscalers and large neoclouds. Enterprise self-builds might account for 20–30%; that share is only a judgment, with no data behind it.

Self-builds will keep three niches: first, data sovereignty and compliance, such as finance, healthcare, government, and defense; second, large internal agent fleets with predictable demand, where self-built TCO may beat the cloud; third, work close to the data, such as factories, OT, and edge sites. The most common real-world setup will be hybrid: model inference and most VMs in the cloud, sensitive data and tool execution on-premises, or a sovereign cloud.

6. The PC gets downgraded, then polarizes

More cloud VMs on one side means PCs become less necessary on the other. Still, I think the PC gets downgraded rather than disappears. Compute-oriented machines that exist to run the work will shrink: once data processing, compilation, and aggregation go to cloud agents, employees no longer need high-spec laptops, corporate refresh cycles lengthen, average specs fall, and mid-range business machines feel the pressure first. The screen where people make decisions, review drafts, and check results remains, but it looks more and more like a terminal; a phone, tablet, or thin-and-light laptop is enough.

The countervailing variable is the AI PC and local inference. Privacy, latency, and token costs will keep some agents running on NPUs or unified-memory machines, which could create new demand at the high end. The likely result is polarization: the mid-range shrinks, while entry-level terminals and high-end local inference machines coexist. Memory benefits at both ends: the cloud needs capacity, and local machines need larger unified memory.

7. Investment implications and indicators to watch (judgment, unverified)

First, the order of beneficiaries may put memory ahead of CPU core count, including DDR5 RDIMM, SOCAMM, and CXL expansion. The basis is Section 3: the ceiling is memory. Second, within CPUs, cloud in-house Arm and Arm licensing will capture much of the increment, and x86 growth may not keep pace with the number of agents. Third, the increment lands mainly in hyperscaler and neocloud capex, with Taiwan’s server ODMs (L10 to L11 full racks), power, and cooling supply chains benefiting alongside. Enterprise self-builds give OEMs such as Dell, HPE, and Supermicro, plus private-cloud software, a smaller and more fragmented new market. Fourth, the winners of the VDI era are not necessarily the winners of this wave; the entry point has moved to the AI platforms themselves. Fifth, as compute moves from the endpoint to the cloud, PC brands, assembly, and mid-range components, including Taiwan’s PC supply chain, should watch for weaker refresh demand.

Indicators to watch: growth in agent-related revenue and CPU instances in hyperscaler earnings, the deployment scale of in-house Arm chips, server DRAM and CXL shipments, corporate PC refresh cycles, AI PC penetration and average selling prices, subscriptions to cloud agent services, and sovereign AI tenders and OEM enterprise AI server orders. None of these has systematic data yet; they will need to be tracked going forward.

8. Risks: three ways this path could narrow

The first is API-fication. If services broadly open APIs and agents call them directly instead of operating a graphical interface, agents no longer need a full computer, and demand for VMs and memory shrinks sharply.

The second is sharing. If deployment shifts from one always-on VM per user to shared containers or on-demand microVMs, the memory and CPU used per task both fall. Vik’s 192-to-896-VM math would be revised down across the board.

The third is a financing brake, and it is the least technical of the three. SemiAnalysis’s Dylan Patel said at a Chipstrat panel on 10/23 that financing is now the new brake, with CoreWeave’s unsecured bonds yielding about 15% in the market (verbal figure, unverified). Lauris broke down CoreWeave’s DDTL 5.5 loan documents: SOFR+550bp, with renewal requiring remaining contracted receivables of at least 2.40x the loan balance and a DSCR of at least 1.35x, and 5-year CDS above 1,100bp in mid-October (author’s citation, not a neutral source, unverified). Anna Wong’s team at Bloomberg Economics counted 419 mentions of shortages on S&P 500 2Q26 earnings calls, 33 of them with wait times: roughly 12–24 months for memory and 24–30 months for power and grid (X post, original chart not seen, unverified). Her judgment is that financing conditions are tightening. Taken together: the logic of agent demand holding up does not mean data centers can be built as fast as demand grows. Money, power, and memory lead times may be what actually sets the pace.

Conclusion

VDI didn’t take off in twenty years not because the technology fell short, but because the people sitting at the cloud computer didn’t need it. Agents are different: they don’t care about the screen, only about having an environment ready at any moment. With a new user, the CPU yardstick switches from how fast a single core runs to how many VMs you can sustain per GB and per watt, and memory goes from supporting role to ceiling. Which CPU wins is still open, but I’m fairly confident of two things: memory is the most important column on this yardstick, and most of this race will be run inside hyperscale data centers.

Notes and sources (checked 2026-10-26, Taipei time)

  1. CPU specs (official materials and reporting):
  2. Per-core bandwidth of 13.6 GB/s and 3.2 GB/s, and ~192 VMs on 1.5 TB: my own calculations from the official specs above.
  3. Vik’s Newsletter, “How Many CPUs Do Personal AI Assistants Need?”, 2026-09-28; only the first half read; figures are the author’s estimates and second-hand accounts, unverified.
  4. Chipstrat, Dylan Patel panel transcript, 2026-10-23; verbal figures, unverified.
  5. Lauris (@lzminsky), CoreWeave loan document breakdown, X, 2026-10-26; unverified.
  6. Anna Wong (Bloomberg Economics), X, 2026-10-26; original chart not seen, unverified.
  7. Not found: Vera standalone cloud rental specs and pricing, Vera Rack power draw, Graviton5 memory bandwidth and TDP, Xeon 6990E+ memory bandwidth, Grace TDP.

This is industry analysis and judgment, not a recommendation to buy or sell anything.

Personal views only; not investment advice.