nostrebored
yesterday at 7:29 PM
150k TPM limit on public endpoint means that it's likely unusable for many coding tasks. When we've tried Cerebras in the past, our problem has always been rates. We'd love to not deal with dedicated and to have access to a more flexible rate pool.
Even trying it out, it seems like our account has gotten moved to some limbo where we can no longer add billing information.
```
Billing access restricted
Self-serve billing is not available on Enterprise accounts. Please contact your team for further questions.
```
We have no team (they removed themself from our slack channel after we talked about rate limits). Perplexingly, none of this even shows up in the request, which gives:
```
{"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"}
```
When the error is really about billing.
I always want to like Cerebras, but I get the vibe that as a tokens in tokens out consumer you are not valued at all.
Aurornis
yesterday at 9:29 PM
> 150k TPM limit on public endpoint means that it's likely unusable for many coding tasks.
I don't understand. How does that make it unusable? Is the limit shared by an entire team at once?
150,000 tokens per minute is a lot. You could start hitting that with a lot of concurrent requests in your session, but even throttled to 150k TPM it's still going to be faster than anything else you find.
I think the 128K context limit is the real ceiling. These models aren't amazing at long context, but once you account for a short input prompt, the input files, and headroom for a compaction summary, there isn't a lot left for the problem.
wild_egg
yesterday at 10:12 PM
It's a limit on input tokens. So that's 3 50k requests per minute. At Cerebras speeds, that's about 5 seconds of usage per minute.
I was very excited last year for their coding plan but seeing a burst of requests pulse and then sitting there watching the cooldown reset is really not a great time.
Even though each individual request was fast, the sessions were only maybe 10% faster on wall clock time since there was so much waiting time.
kristjansson
today at 2:02 AM
They made the coding plan a bit better toward the end, but it was pretty tough to use throughout.
Seems like an Amdahl’s law of inference economics? there’s so much compute relative to SRAM on the chip and shoreline bandwidth onto the chip that caching buys ~nothing? The contended resource is SRAM and a given token of context needs just as much as another.
vlovich123
today at 4:16 AM
That’s not what caching is for. Caching lets you resume with a pre computed KV cache saving you from having to ingest everything in the chat history as input on every single round trip. You still need caching regardless of SRAM or not as it saves a huge amount (and ever growing) of compute ingesting the preceding history every time you want a completion.
I don’t know why they don’t give a price discount. Maybe their hardware is incapable for some reason of saving/restoring the state? Or maybe they just haven’t built the infrastructure to do it?
amelius
yesterday at 10:27 PM
Can't you do something with multiple accounts?
You would lose caching (if they cache)
sandworm101
yesterday at 10:50 PM
Or just buy a 5060. This will run on most any 16gb card. Slower for sure but far cheaper than another subscription.
embedding-shape
yesterday at 11:54 PM
Or buy a raspberry pi with a SSD, about the same difference, if you're giving up on the 1500 tokens/s anyways.
Thats not the point if you choose Cerebras as provider.
gerdesj
yesterday at 10:29 PM
128k context is not a limit of the model, that's a limit of implementation:
"Context Length: 262,144 natively and extensible up to 1,000,000 tokens."
https://huggingface.co/Qwen/Qwen3.8-27B
TPM means Tokens per Minute.
datadrivenangel
yesterday at 10:05 PM
150k tokens per minute at 1.5k tokens per second means you can have like 3 users concurrently and that's not a lot.
conception
yesterday at 9:47 PM
150k by account. At 1.5k a second you hit it very quickly.
Exactly, it burns the tokens 3000x faster, which means the budget ($$$$$$) runs out so faster it will stop super quick, not able to perform long-duration work. At 27B parameter size, the intelligence is not able to accomplish work within a short amount time. Consequently, it become not usable.
gerdesj
yesterday at 10:25 PM
I (we) run Qwen3.8-27B-FP8 on a DGX Spark box - that's roughly ÂŁ4000 of hardware.
I did benchmark it in various ways and it runs quite well but it is a quantised jobbie and 1.5k t/s is also rather faster than anything I can possibly hope to achieve.
To run that model at those sorts of speeds is going to need some serious investment and you are going to have to pay for it.
conception
today at 2:50 AM
The problem is most providers hit tok/sec limits really fast. 1m/min is the default and the only place I can get 10m+ is from first party providers without a lot of upfront cash.
jacquesm
today at 12:50 AM
How fast is it?
kristjansson
today at 2:07 AM
With MTP and FP4 I max out at 30ish t/s on mine. Without MTP or in regimes where the drafter performs poorly it’s about 10 t/s. FP8 is about half that
Unusable is too stretch IMO, you can still use it in tiny tasks that’ll respond almost instantly
olivermuty
yesterday at 7:34 PM
Cerebras the tech is awesome, cerebras the company is a trainwreck
i hope groq wins if they start doing such things with consumer.
dd8601fn
yesterday at 10:23 PM
Is this the chatjimmy asic approach with a bigger model?
ericd
yesterday at 11:54 PM
No, the asic could only ever run one model/set of weights, no updates possible, ever. These are general purpose processors that can have their models updated. But the chips are enormous, with a substantial amount of on-die memory alongside the execution units, for a relatively insane amount of memory bandwidth.
I thought from what I read about the Taalas approach, the model architecture and overall size couldn't be changed, but model weight values could be updated after for further tuning.
Not as flexible as Cerebras though. And I'd love for someone who knows more to clue me in to the truth.
puppymaster
today at 3:17 AM
all the above. They just simply do not care about non enterprise customers. Today they announced qwen, guess what - it's also the same day they pulled Gemma off their shared tier. No migration notice and all developers are scrambling as we speak trying to migrate. They gave a soft head-ups on discord a week ago and when folks complained about zero-day migration they started saying 'you aren't suppose to build production app on shared tier'.
ricardobeat
yesterday at 8:49 PM
What kind of coding tasks would you expect to hit that limit? In my setup, on a very large codebase, it takes each agent 3-4 minutes at minimum to go past 100k tokens.
(note it's 150k uncached tokens, the total limit is 450k/min)
conception
today at 2:55 AM
So that’s about 400 tok/sec. Times that by 3, you get 100k in under a minute. That’s doing nothing special and just using your current setup.
nostrebored
yesterday at 8:59 PM
in my last tests with cerebras for coding tasks, most large tasks or anything greenfield would hit token limits. note that smaller models and the gpt-oss-120b style models they used to run are very prone to overthinking, so individual turns may be 3-10k tokens of just thinking + input + output.
i don't think it's quite apples-to-apples to compare to a frontier model or even a k3. the odds of success (file compiles? read the right context?) are lower and thinking is longer.
collin
yesterday at 8:29 PM
This was my experience a year ago on some other model they could run super fast. Routine coding tasks would hit the per-minute token limits.
Just the math there... 150k TPM... and 15k TPS means... you can run for 10 seconds every minute?
The basic math boggles the mind.
baegi
yesterday at 8:34 PM
Not sure how the rate limiting works, but it's 1.5k TPS, not 15k, so you could run it for 100s/min, which seems good enough to me
nostrebored
yesterday at 8:41 PM
iirc input (uncached) goes towards the limit as well
fc417fc802
yesterday at 9:21 PM
What's the tok/s when they process input?
ah, yes, that seems right
I was using it quite a while back, different model, different quotas, but for coding tasks it routinely hit quotas which made it quite difficult to actually use.
100s/min seems pretty poor actually with sub-agents etc.
fc417fc802
yesterday at 9:20 PM
It seems you forgot to account for the fact that cerebras uses a baker's minute which is 144 seconds instead of 60. (Seriously though what's the supposed issue here?)
RussianCow
yesterday at 9:47 PM
The issue is that all input (including context) counts towards that limit. So 10 requests with 50k of context will blow through the limit, even if little to no output was generated, which is incredibly easy to do with agentic workloads.
0xbadcafebee
yesterday at 8:21 PM
Yeah, their public service isn't a serious/competitive offering. They don't have the capacity to serve all the customers who might want to use them at that speed. The public service exists so they get some users on OpenRouter, and that shows them as #1 on speed, which proves their tech is very fast, which gets them billions in hardware sales/licensing. If you have big enough pockets they can probably dedicate capacity to you. But for reliably fast small models you might want to rent some GPUs.
LoganDark
today at 5:06 AM
GPUs can't reach these speeds. You could build a supercomputing cluster and not reach these speeds.