Last week two nodes sat directly cabled to each other, 391 Gb/s across two rails, and two models ran faster cross-node than on a single machine. Bandwidth was solved. What I hadn’t touched: both models were running with 65,536 tokens of context while they were capable of 262,144 and 524,288 respectively.
A quarter and an eighth of what was available. That’s fixed now, and getting there produced two things I didn’t know before.
A week ago I talked a UniFi switch into doing lossless RoCE, which its datasheet flatly rules out. Two 100G rails, PFC, the whole thing. That switch is still in the rack doing its job. It’s just not in the path I care about anymore.
Because two of the nodes now sit directly cabled to each other. No switch between them, 2×200G ConnectX, card to card. And the number that came out is unspectacularly precise: exactly a factor of 2.00.
If you have read the earlier posts, you already know the shape of me: I run a GPU cluster down in the basement, and I would rather spend a weekend elbow-deep in it than reach for a credit card. You can hear the fans spin up through the floor when something starts working hard. This is the story of one of those weekends, the one where I taught a switch to do something its datasheet swears it can’t.
One of our GPUs runs two things at once. A text-to-speech service and a worker, sharing the same physical card through MPS. That is on purpose: together they fill one card cleanly, and split apart each would waste one.
Sounds solved. It wasn’t. The scheduler counted that one card twice, put the node at used: 2, capacity: 1, and with that shut it down for every further pod, including ones that wanted no GPU at all. A node with one free card, claiming it has none.
In a campaign hall in Florida a young man steps on stage and says one sentence that earns him the loudest applause of the evening: he will not build any new data centers. Markus Lanz observed that, and for him the applause is a warning sign. The big AI euphoria, he says in the new episode of “Lanz & Precht”, is “completely over”.
I listened to the episode and had to put some things in perspective. Not because the two men are stupid, on the contrary, I enjoy the podcast, especially because they do not always agree with each other. But because nearly every argument follows the same reasoning error, and because that error has a long, embarrassing tradition. As someone who works with this technology all day, I laughed out loud at some points. Not from mockery, from recognition. These are the exact same sentences people said thirty years ago about the internet.
Everyone is counting GPUs. The press releases count them, the funding rounds count them, the datacenter announcements count them. Compute is the number that makes headlines.
I run a small GPU cluster. Nothing at hyperscaler scale, but enough to feel where things actually break. And the thing that breaks is almost never the compute. It is memory. How much I have, how fast I can move it, and lately, what it costs.
For three weeks an agent has been running my GPU cluster. It scales Blackwell cards up and down, picks nodes, and frees silicon when nobody is rendering.
Not autonomous magic. It runs the loop I used to babysit by hand. Here is the honest version of what that taught me, with the parts LinkedIn was too short to hold.
