

even if the chips do make it safely into orbit, physically repossessing them in the event that SpaceX defaults on its $43 billion loan would be a logistical nightmare.
Ohhhhhh.
I see. That’s why they want datacenters in space.


even if the chips do make it safely into orbit, physically repossessing them in the event that SpaceX defaults on its $43 billion loan would be a logistical nightmare.
Ohhhhhh.
I see. That’s why they want datacenters in space.


Generative ML inference isn’t so dramatically bandwidth limited anymore, either, so the “demand” for HBM isn’t as clear.


GDDR versioning doesn’t necessarily line up like that. For instance, GDDR7 is technologically akin to LPDDR5X, just with a different set of configuration tradeoffs.


The full report doesn’t mention CoPilot anywhere.
That’s what you meant, I suppose, but there are a lot of services they don’t mention.


Nah, Groq is toast. Those numbers are for toy models distributed across tons of cards, and splitting MoE LLMs across so many cards in SRAM an inefficient approach IMO.
The best model they serve is the kinda obsolete GPT-OSS 120B, at 350 token/s: https://openrouter.ai/openai/gpt-oss-120b
Which you can get with Deepseek, Muse or other open weights models these days, if you really need it.
The only ASIC maker I kind like is Cerebras, especially their scheme for hybrid inference in conjunction with AMD. Architecturally, that makes a lot of sense.
EDIT:
Upon further investigation, they host more models than I thought:
https://console.groq.com/docs/models
They have MiniMax 2.7 at 200 tokens/s, which isn’t too bad… but still largely obsolete now.


Yep. Samsung is already researching this. They have been for some time.
Also, I feel Bitnet would be useful here, as it would only require small adders, no huge multipliers.
They could even take a hybrid approach, processing higher-precision attention and small parts of the LLM on the GPU ‘traditionally’ while specialized adders in memory handle the huge ternary MLPs (or whatever the sparse equivalent is in other ML architectures). This would also give them architectural flexibility (as attention schemes are evolving wildly, the MLPs kinda aren’t).


Honestly CAMM makes way more sense. It’s just a better interface, electrically. Why do desktops need sticks if they’re going to be higher voltage and slower…
And we will be able to afford this. Some day.


I don’t understand. If it’s just a flag for apps to check, can’t WINE just spoof it if the user wants?
Also… I have secure boot enabled, and I think that whole list enabled, on my dual boot Linux rig. Works fine.
And if you don’t enable secure boot, Windows stops getting feature updates, which eventually stops security updates, so you’re already kinda screwed without it anyway.
In other words, this headline feels like a nothing burger.
Windows 11 basically requires secure boot already, and if you already have it off, you don’t need it… unless you are like me and discover Windows literally will not update without it.


Instead of crunching then firing, it would make more sense to interleave releases (like another Red Dead), so departments can work on the other game instead of getting laid off, or having nothing to do.
Which is of course what some game and movie studios do.
I get they were trying to make the biggest game of all time, but that doesn’t necessarily make sense these days…


Nah, there’s a person behind it at least.
…I have almost blocked eicker for posting Futurism clickbait, though. I didn’t come to Lemmy for that junk in my feed.


Sounds about right.


I think I’m blocking JS for a few months…


fluency gets mistaken for understanding
That’s it. I am going to borrow that phrase.
And a huge component is that Silicon Valley mindset that… values fluency over understanding, I guess?
The words to describe it are hard. The “Elizabeth Holmes Complex” in You Can See Everything pretty much sums it up.


GPT-3 was supposedly too catastrophic to release. Now that class of model can literally run on a fridge, yet here we are.


I am not with the Lemmy hive mind consensus on LLMs, not even close. But we are discussing something more specific.


Home inference.
And that’s a very broad statement. But nothing we can do locally will get an LLM to intuit or reveal its own dataset.
We can get it to output Claude traces it was trained on. I am not trying to be pedantic, but the semantics/phrasing there is very important.


I am not trying to overwhelm, but you challenged us to reproduce your assertion, and I can’t really do that without the prompt.


Hot beef is right. That’s a good way to phrase it.
And before you bring it up, I’ve been experimenting with local models since GPT-J and old GAN finetunes. I have MiMo 2.6 Flash loaded right now… TBH, I’m not sure how that’s relevant.


Ah, perhaps. I just skimmed it.
Nah it’s not a bot net. It’s just people on Lemmy. Folks on the internet (in aggregate) don’t mind clickbait as long as it’s a sentiment they already agree with, and will reflexively defend it. Sometimes, they even consciously prioritize that over information hygiene.
And I’ve found the Fediverse is no exception, sadly.
Yeah, that is a “fine wine” card like the early GCN AMD GPUs.
I wish they had made bigger die GPUs, though. And they’d have made a fortune from them now.