today at 9:44 AM
That's pretty innovative. But a brand new tokenizer and training method need a lot of effort to do the retraining. Whether it will be widely implemented depends on real frontier performance it can reach.
today at 12:38 PM
Exactly. I used to think new tokenizers for domain-specific models was the way (makes intuitive sense, better representations etc), but the added overhead of essentially training from scratch made it much more expensive than a FT.
today at 12:02 PM
If we cant' build the hardware fast enough. Can someone explain to me why there's apparently 50% of the gpu and memory that hyperscalers buy sitting in warehouses due to lack of data centers? The hyperscalers are hoarding tech and can't even use it. It's ridiculous.
today at 2:14 PM
As far as I can tell, because that is not happening.
The suggestion seems to have come from the claim that to maintain a priority queue, you have to be buying consistently. That might mean buying more than you need to ensure supply when you need it. The assumption that the excess goes into a warehouse seems to have come from someone's fevered imagination.
If there are GPUs built and not yet installed, and people who want to be using them now, money can be made getting the supply to find the demand. Most large scale companies know how to do that sort of thing. If someone has a data center and no GPUs arriving for six months, paying a few million to trade the GPUs that will exist in six months for the ones that exist now gets them an additional six months of operation, while ensuring the original buyer will have GPUs when they need them, plus a few million.
today at 3:11 PM
Yeah. Don't know anything about data center GPUs, but it's common knowledge you can get like 80% of the performance of a gaming GPU at half the power or even less. That's basically how Nvidia's mobile GPUs are. So if you had a warehouse full of GPUs, you could just go ahead and install them, run them at low power, and still get like 2/3rds of the performance.
today at 12:50 PM
There are reports and rumours that NVIDIA are forcing the data centre companies to buy the GPUs upfront. Though there isn't a lot of clear evidence that this is the case.
[1] https://tech.yahoo.com/gaming/articles/did-nvidia-manufactur...
[2] https://www.reddit.com/r/Games/comments/1xdjbd1/multiple_rep...
[3] https://www.techradar.com/computing/gpu/is-the-gpu-stock-cri...
[4] https://www.businessinsider.com/amazon-nvidia-ai-gpu-data-ce...
today at 12:49 PM
I dont understand how this comment connects to the linked research paper. Anyone?
today at 12:15 PM
That depends on how long it’s sitting there, given they are two different things to manage it makes sense that you’d either have gpus sitting unused while a dc gets built or a dc sitting empty while you wait for gpus.
today at 12:37 PM
They could probably rush them to production, but it's added logistics and would lead to the parts degrading faster.
today at 2:48 PM
Lack of customers*. SpaceXaitwitter has some big beautiful idle datacenters with all GPUs ready to use.
today at 12:23 PM
Because slowing down compute purchase is collapsing the hype train which is the same as death. Vibes are a wave that must be maintained or crash down onto reality.
today at 1:10 PM
Also, tinfoil alert, if they buy it means you can't (or are eventually priced out of the hw market). Their only "moat" (if you can call it that..) is the price of local inference capacity, which is still kinda prohibitive for the 99%.
today at 8:08 AM
Who's going to train a nice big coding model based on this? I don't have the GPUs to do it...
today at 9:35 AM
I hope Apple Silicon will one day be competitive with GPUs. It dominates much of the CPU market, and it has a GPU, but it's still not even remotely comparable to Nvidia for inference (factor of >5 in some cases, especially in prefill) or even for gaming ("edge computing" in Nvidia terms).
I feel like their GPU architecture is going to need a significant redesign. There are limits to this tile-based mobile architecture.
today at 2:23 PM
Depends what you are comparing. Apple Macbook Pro M5Max out performs NVIDIA RTX N1X laptops for about the same cost.
today at 3:09 PM
Fastest consumer Apple silicon versus fastest consumer Nvidia. So M5 Ultra vs RTX 5090.
Not comparing price or efficiency or value or anything like that. Just the platforms as a whole.
today at 3:24 PM
> Fastest consumer Apple silicon versus fastest consumer Nvidia. So M5 Ultra vs RTX 5090. Not comparing price or efficiency or value or anything like that. Just the platforms as a whole. So M5 Ultra vs RTX 5090
The problem here is that they aren't equivalent platforms - no amount of money will buy you an RTX 5090 with 512 GB of RAM.
If you want to buy the 16x RTX 5090s you need to get to the same 512 GB of RAM, we're talking $100,000 just in GPUs, industrial levels of power/cooling/noise, and the interconnect bandwidth is still going to kill it for normal inferencing workloads.
Of course, that 16x cluster is an absolute beast in terms of raw compute - it's just not all that useful for the things people are trying to do with such machines today.
today at 8:41 AM
I'm trying to run it, so far no luck
today at 12:36 PM
This looks very interesting. Their new language modelling objective shows some large improvements over masked diffusion modeling.
However, they hardly compare against the conventional next-token-prediction objective, which suggests they don't beat it, except for highly non-linear tasks like sudoku. Unfortunately, machine learning papers often tend to focus exclusively on the positives without mentioning bad aspects.
today at 9:18 AM
I keep getting clickbait youtube videos about how "NCM" obsoletes "LLMs" and "Nvidia memory chips" (yes, it's that bad).
This makes me wonder what these people think an "LLM" even is. The name itself implies that it is large and about language. It does not imply that an "LLM" is inherently autoregressive or that it has any particular architecture even though most LLMs are transformers. Anything can be a large language model as long as it is large and models language.
NCM is basically a hierarchical diffusion transformer where the further layers take one big token and then generate two smaller tokens. So it is more like an evaluation strategy. You could apply this to any model architecture.
I personally think the name NCM itself is terrible because it contains the word "next" which inherently implies an ordering and somehow we are supposed to think that "LLM" which contains no mention of ordering, is constrained to ordered token production.
today at 10:49 AM
I think they think an LLM is just whatever another LLM tells them it is when they ask it to generate a script.
today at 12:39 PM
Hey, me too.
today at 11:52 AM
today at 9:25 AM
Can we apply any insights from image and video compression and partitioning to this kind of dimensional masking ?
today at 8:00 AM
[flagged]
today at 1:27 PM
[dead]
today at 10:06 AM
I'm not sure why but using X instead of the word "times" annoys me. Probably because one of the first things I got taught at university is that mathematical notation is not prose and shouldn't be used as such.
today at 10:32 AM
But what about “got taught” instead of “was taught” and the missing comma before “but”?
today at 10:37 AM
Well they didn't teach me English. Or in English for that matter.
And I don't think I've ever used the Oxford comma.
There was probably some rule about not beginning sentences with Or and And but you probably know better than I.
today at 12:43 PM
Different cultures have different norms and rules. I think you're saying that, in your native language, they taught you not to use mathematical notation in prose.
They didn't teach you that about English, though. It's quite common to write "21x" for "21 times" in English and is pretty much universally understood and accepted by native speakers.
today at 2:29 PM
The first time I heard anyone pronounce 2X as "two ex" instead of "two times" was Steve Jobs in one of his presentation.
today at 1:04 PM
It's a more general rule about when and where to use mathematical notation. Though style rules do differ in places.
For what it's worth I've not seen "12x" in formal contexts, but if that is proper then the times symbol still shouldn't be styled as part of the maths formula imho.
today at 10:45 AM
The Oxford comma is the comma before the conjunction in a list of three or more terms, e.g. "one, two, and three". I think your sentence actually didn't need a comma anway.
today at 11:07 AM
I wonder whether "10" in "10×" also counts as mathematical notation. Or if just "ten times" is allowed.
today at 12:59 PM
Technically yes, but quoting mathematical constants is definitely allowed. You could also say O(N) times for instance. The kind of language that is discouraged is writing "therefore efficiency increases by O(log(N)) times" as "∴ efficiency increases by O(log(N)) *".
For one it simply looks like you forgot to finish the formula.