Weekly Digest 43

Topics we are tracking

Google and Amazon take aim at Nvidia’s software moat

Sources: AWS · Google Cloud · SemiAnalysis / InferenceX

For years, Nvidia’s dominant position has been partly supported by its CUDA ecosystem, commonly known as the “CUDA moat”. But we think this moat has gradually weakened. Nvidia itself knows this and has invested billions of dollars building a supply chain that we would now consider the moat. On the software front, Nvidia has adapted to a changing landscape by expanding its Python tools and continuing to improve integration with frameworks such as PyTorch. CUDA was originally centered on C/C++, but developers have been able to access it through Python for years. Nvidia has consistently made it easier to run a wide range of models efficiently on its hardware through optimized kernels and communication tools.

This week, both Google and Amazon unveiled new software packages to make it easier for developers to take their existing PyTorch models and run them on Google’s and Amazon’s custom AI chips. Amazon's new TorchNeuron package, now in public beta, brings native PyTorch support to its Trainium2 and Trainium3 chips. At the same time, Google announced the open source release of TorchTPU and new integrations with popular inference libraries like SGLang.

TPU v7 leads GB300 FP8 at 150–400 tok/s/user; GB300 FP4 leads both at those targets. Million total tokens / $1 TCO. TPU v7 · FP8: 100: 119.0, 150: 110.9, 200: 102.4, 300: 75.2, 400: 60.1; GB300 · FP8: 100: 121.2, 150: 88.6, 200: 55.5, 300: 37.2, 400: 26.8; GB300 · FP4: 150: 217.2, 200: 188.1, 300: 137.0, 400: 84.3. Interpolated estimates; TPU preview. Total tokens include cached inputs. Modeled ownership TCO for external buyers, not API prices. First-token delay is not matched; answer quality is not assessed. Serving systems differ. No FP4 estimate at 100 tok/s.

Google TPU v7 · FP8 · vLLMNvidia GB300 · FP8 · Dynamo SGLangNvidia GB300 · FP4 · Dynamo TensorRT-LLM

P90 streaming speed (output tokens/s/user)

Interpolated estimates; TPU preview. Total tokens include cached inputs. Modeled ownership TCO for external buyers, not API prices. First-token delay is not matched; answer quality is not assessed. Serving systems differ. No FP4 estimate at 100 tok/s.

Source: SemiAnalysis / InferenceX · 2026-10-19

The new software means developers can use more of their original model code, debuggers, and training processes. Previously, their chips had supported PyTorch through integrations that often necessitated different implementations or altered developer workflows. Hugging Face says TorchNeuron now supports 89 of the 100 most-used text-generation model architectures out of the box, compared with 46 through Optimum Neuron v0.4.5. Google also announced its new PyTorch-based SGLang backend is already in production at Meta.

Until now, Google and Amazon arguably had less incentive to make these software stacks accessible to a wider developer audience. TPUs have been used internally by Google and by large labs like Anthropic, which have the expertise and resources to invest in their own software. As both companies seek broader adoption of their chips, the software ecosystem becomes more important and helps support their investment in custom silicon. Nvidia employs a large number of software engineers to develop and maintain optimized libraries, kernels and communication tools that make it easier for customers to run models efficiently on its hardware.

Nvidia introduces a modular data center design

Source: Nvidia · Press release · Image

This week at GTC Berlin, Nvidia introduced the DSX POD, a reference design for a modular AI data center centered around its Vera Rubin platform. Partners including Dell, Schneider Electric, HPE and Vertiv will deliver systems assembled and tested in containers before shipment, with availability planned for early 2027.

Jensen has long been talking about the AI factory, for which he now seems to have laid out the blueprint. The idea is to standardise computing, networking, cabling, rack-level power and cooling into a factory-built design, which Nvidia says could cut deployment time at a prepared site from six months to as little as two weeks. As long as token demand outstrips supply, data centers can essentially be money-printing machines. There is therefore a strong financial incentive to get them online several months sooner. A more modular design would also make it easier to expand the project and potentially reduce the need for scarce skilled labour at the construction site.

For Nvidia, this is another example of making sure the company sits at the center of the ecosystem. Nvidia has invested in almost every aspect of the data center supply chain and, as a consequence, has become a central player in securing allocation and priority access at almost every level. Through its partner network, it can help data center developers gain access to cooling, networking and power equipment, all of which are scarce and have long lead times.

Kimi launches K3.1 Pro

Sources: Moonshot · Artificial Analysis

Earlier this year, Z.ai’s Jie Tang suggested on Twitter that we would get a Fable-class model before year-end. With two months to go, we now have a strong candidate in Kimi K3.1 Pro: a 2.8T-parameter model that ranks ahead of both Claude Fable 5.1 and GPT-6 Astra on the Artificial Analysis Intelligence Index. Moonshot plans to release the model weights by 4 November. The technical details are fairly sparse, which is common for a version update like this. The launch blog essentially points to substantially more RL post-training to deliver continued gains.

The launch also includes a Highspeed version that Moonshot says delivers 2–3× the output speed. We think that labs will increasingly push the speed frontier alongside efficiency and capabilities. Moonshot also offers Kimi Hosted Agents, a hosted harness for long-running background agents. The “Pro” designation could point to a future cheaper and smaller “Flash” version, which could be economically more interesting for many users. Flash variants currently occupy four of the top five entries in OpenRouter’s weekly token-usage ranking.

Moonshot says it removed cyber-related data from mid-training and reinforcement learning as a safety precaution. Nevertheless, the model performed strongly on cyber benchmarks, leading publicly accessible models on Artificial Analysis’s Cyber Index. Many of the reported coding benchmarks use the Claude Code harness, so those results reflect the model and harness together. It remains unclear how well the scores would translate to a different setup. We have also seen differences between inference providers on individual benchmarks for Kimi K3, which makes the hosting configuration another consideration when assessing these results.

Goldman Sachs lifts copper forecasts

Sources: Goldman Sachs Research — first page · Copper prices: Westmetall · Cochilco

Goldman Sachs recently raised its year-end copper price target to $14,700 per tonne and increased its average price outlook for 2027 to $15,400. The investment bank is now predicting a tighter market outside of the US, which Goldman attributes in part to two policy-related headwinds: the possibility of tariffs imposed on American imports and curbs on trade in Chinese scrap. The potential for tariffs is driving copper shippers to frontload the US market.

206 daily LME copper cash settlements from 2 January to 23 October 2026, in USD per metric tonne. The first quote is 12,571 and the last is 14,415.

Daily LME copper cash settlement, as published by Westmetall. Trading days only. Five dates cross-checked against Cochilco’s original daily reports. Prices are in USD per metric tonne.

Source: Westmetall, Cochilco · 2026-10-23

As of 22 October, Comex warehouses held roughly 724,000 tonnes of the metal, which has tightened the market for other buyers. While tariffs on refined copper are not yet a certainty, that possibility is enough for shippers to justify holding stocks in the US. This suggests the US market can be relatively well-supplied while Europe and China become increasingly tight. Reuters reports that Codelco has quoted European clients record $350-per-tonne premiums above LME prices for next year’s shipments.

Meanwhile, the tightening of tax compliance and document checks in China is making it harder for producers to source scrap with the documentation they need. This is driving some buyers to switch to refined copper. As a result, if a producer were to make the same amount of cable it always has, but had to change its supplier from scrap to refined, demand for refined copper would rise even as total copper consumption remained steady. The headwinds from mine outages are exacerbating these market imbalances. Yet the ICSG continues to predict that the global market will end the year with a small surplus in refined copper. Increasingly, copper prices are dictated by where the metal is located and which forms are available to manufacturers. So the prospect of a tariff decision in the US or a change in availability of scrap in China could bring a degree of relief to the market without the need for a significant uptick in mine supply.

Seen on X

Other interesting stories

• New York Times: It’s Not Just France. Europe’s Political Center Is Crumbling.
• Faint Signals: What Goes Quant Next?
• Vik’s Newsletter: The Four Horsemen of the AI Infrastructure Buildout
• Foreign Affairs: AI Is a Choice
• ECB: Resilience of the euro area economy – weathering one shock, preparing for the next