today at 7:51 PM
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c
It seems implementing transformers is the new "hello world" of AI assisted coding.
today at 7:07 PM
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
today at 7:23 PM
Fpga is software-driven, model can verify its output easily.
Zero surprise it did it
today at 7:08 PM
it only cost them $10,000 in Claude API usage. :)
today at 7:12 PM
You get that much usage in a monthly $200 subscription.
today at 7:51 PM
So who's paying the cost?
today at 8:32 PM
Everyone else that doesn't use the entirety of the plan they pay for.
today at 8:36 PM
Retail and society as ANT shares get pulled into index and wealth funds
today at 8:24 PM
API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
today at 8:31 PM
People believe in some wild stuff.
today at 7:52 PM
Corporate customers
today at 7:34 PM
And a kitten.
today at 7:30 PM
This is cool
Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...
today at 7:42 PM
Good chance I'm not the only one thinking this: share the code
today at 8:45 PM
Repetitive "make it faster" prompting is essentially what all these "RSI" claims are
today at 8:31 PM
I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.
(unfortunately for a decent model it will also be price competitive with a big nvidia card)
today at 7:51 PM
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
today at 8:24 PM
Now do silicon. ;)