aabdi 12 hours ago

5x throughput more than b4, same 10x tokens/s as compared to standard.

looks like disagg/wide ep is applicable to cerebras chips as well. kind of amazing.

https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

  • oivaksef 10 hours ago

    They talk about Kimi - I would love to run that on cerebras, but it's not available from them directly. Is there a provider I can use this at?

hephaes7us 10 hours ago

It sounds like this might mean they could do real prefix caching?

Cerebras would be much more interesting to me if cache hits resulted in a commensurate savings on my end.

ismailmaj 5 hours ago

Glad they are upfront about their chips being a bad fit for prefill workloads, their previous article completely dodged the question: https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

  • imtringued 2 hours ago

    If by "bad fit" you mean they are too valuable to waste on prefill then yes, they're a bad fit. Other than that, disaggregated inference with heterogeneous architectures is actually the gold standard, because the two workloads are very different.

    Decode needs to have as many parallel user sessions as possible. Your bottleneck here is how much SRAM is being occupied by user context since the full context has to be read from start to finish. Putting the parameters into SRAM is critical, but with expert parallelism you don't actually have to worry all that much about whether the layer fits on a cerebras wafer scale engine in full or not, because you just need to put as many experts as can fit. Meanwhile attention in decode mode basically just runs two GEMV. The bottleneck here is reading K and V matrices from the SRAM. You must make sure that the user session fits entirely in SRAM for this to work.

    Meanwhile prefill? Meh. You can store the user sessions in HBM or even host DRAM or hell an NVMe SSD (lol) and it is fast enough. You need a few hundred MB of SRAM to hold what you're currently processing, but not much more. Switching between user sessions is not very costly. Chip designers here are mostly selling Synopsis or Cadence designed floating point units. This is a commodity market. Heck even AMD's XDNA NPU is better at prefill than their own data center GPUs, it's just undersized for data center workloads.

    • ismailmaj 1 hour ago

      You're understating the difficulty of prefill, you still need good interconnect to run the all-reduce step for large GEMMs and they don't have an NVLink equivalent.

      From that point of view, it's the decode step that is "easy". You just need a swarm of nodes with just enough interconnect to send a few vectors per request (top 6 for DeepSeek v4.1 Flash), that can be done with a swarm of RTX Pro 6000 without any NVLink involved sufficiently well.

speedping 8 hours ago

Cross-platform prefill/decode split is an incredible feat, kudos to them. I wonder how much time is spent transporting kv cache, and if the gains would be even higher for LLMs with ultra-compressed kv cache like the latest deepseek.

augment_me 12 hours ago

Have they solved the issue yet where they need to keep running ghost tokens to keep the board hot to prevent it from cracking, which causes massive energy draw in idle, and reduces lifetime by 3 times the estimate?

https://culperresearch.com/wp-content/uploads/2026/10/Culper...

  • eh8 12 hours ago

    The short seller report contains a bunch of errors:

    1. The CS-3 operates on average at <65c, well below its thermal limits. Considering the actual thermal parameters of the wafer, using the short seller's own temperature equation would imply the wafer can operate at 85c for 33 years

    2. "Ghost tokens" are not unique to Cerebras and the 3x lifespan reduction claim is based on extrapolating GPU figures which is a big logical leap

    3. "since the introduction of the CS-3 in 2024, less than half of one percent of all CS-3 wafers deployed have been decommissioned."

    https://cerebras.ai/blog/what-it-took-to-build-wafer-scale-a...

    Disclaimer: I work at Cerebras

    • augment_me 12 hours ago

      Very interesting! Thanks for answering with your take, and the link.

      Questions:

      1) What is the idle power draw of the chip? Is the 80% number wrong?

      Nvidia is idling at around 180W/15% of their max for B300. So saying that "it's not unique to Cerebras" is meaningless if we don't consider the power draw itself

      2) Decommissioned is not the same as RMA:d/refurbished/redeployed, so giving this number is kind of dodging the question IMO

      • wmf 11 hours ago

        None of these expensive chips should be idling.

        • augment_me 11 hours ago

          What a non-answer. And there should be perfect latency hiding, and there should always be predictable power-of-2 workloads, and all weights should always be homogenous, etc.

          • adgjlsfhk1 10 hours ago

            It's a reasonable answer. Consumer chips are idle 95% of the time, so idle power draw matters. AI datacenters are currently under capacity, so rather than optimizing for idle power draw, just have work to do.

            • augment_me 10 hours ago

              No it's not, it does not reflect reality. I have worked with both industry and public clusters. The average is around 50% idle-time, many university clusters sit at around 80% idle-time.

              The absolute best I have seen at a hyperscalar internal with predictable, constant workloads due to own projects, and cracked routing infra is 15% idle-time.

              • Tuna-Fish 7 hours ago

                There are both latency-sensitive (serving tokens to users) and non-latency sensitive (doing RLAIF requests to help train models) workloads, and there are enough of the latter chasing compute that it should not be hard to get near perfect utilization. The non-latency sensitive workloads do pay a lot worse.

                This might become a problem later, but right now, if you have AI compute idling, you are doing something very wrong.

              • imtringued 2 hours ago

                Even if they can't "idle" they can still turn off whole nodes. I'm not really sure what your point is.

    • lukeify 12 hours ago

      So which Cerebras statement is in error then?

      > First, in Cerebras’s preliminary IPO prospectuses filed through December 2025, the Company assigned its computer equipment useful lives of just 1.5 to 3 years – in-line with both our Arrhenius modeling and what many of the Company’s hardware customers have described to us.4 Just four months later, in the Company’s April 2026 S-1, Cerebras nearly doubled that range to 3 to 5 years – a change that significantly inflated reported margins. The Company appears to have never explained the reason for this change.

      Was the useful life of a wafer 1.5–3 years on your IPO filing or 3–5 years in your S-1?

      Hard to believe a company with such skilled engineers couldn't figure out the correct useful wafer life given existing industry data. Or (more likely) are we just looking at standard corporate BS?

      • wmf 11 hours ago

        That's answered in the blog.

      • augment_me 8 hours ago

        Both are wrong.

        On the physics of silicon lifetime, Cerebras is right and Culper's calculation is wrong. As Cerebras blog points out, Culper took a 5-7 year GPU economic depreciation schedule and divided it by a 3x thermal power-draw ratio to claim the wafers physically die in ~2 years.

        However in the blog, Feldman writes:

        > By the next update of our S-1, we had set out to build a private cloud; the reason was that AI native customers and developers asked us to. One consequence was that the mix of fixed assets we used internally had changed to include production systems we owned for the cloud business. We therefore modified our depreciation schedule to 3-5 years.

        However the timeline is:

        Mar 2024: CS-3 launches.

        Jun 2024: First confidential IPO filing (DRS) sets equipment useful life at 1.5–3 years.

        Aug 2024: Cerebras Cloud launches (followed by $383M in 2025 cloud CapEx).

        Dec 22, 2025: Updated DRS still certifies a 1.5–3 year useful life, 16 months after launching the cloud, contradicting the statement in the blog.

        Late Dec 2025: Signs the 750 MW OpenAI cloud deal (3.7-year average contract).

        Apr 17, 2026: Public S-1 (weeks before the May 2026 IPO) doubles useful life to 3–5 years, boosting 1H 2026 gross margins from 24.0% to 29.9%.

        ---

        So the blog is not having the right causal relationship here, if launching the developer cloud in August 2024 was the reason for extending useful life to 3–5 years, that change would have been in the December 2025 filing.

        Real reasons probably touch upon their deal with OpenAI, which had to be taken on OpenAI's terms because the IPO would not have happened with a 86% UAE-based G42 (which stalled the 2024 IPO over China ties). Cerebras needed a big US anchor customer to go public, and OpenAI opted to buy cloud capacity rather than buy and run the hardware itself(you can guess why). Once you have to capitalize massive owned cloud infra for a 3.7-year contract right before an IPO, stretching hardware depreciation from 1.5–3 years to 3–5 years is kind of free because all other cloud providers use a 3–6 year schedule, so it's easy to get an auditor to sign off on it.

    • latchkey 11 hours ago

      As far as I understand it, data can also be retrieved from the ghost tokens, so they are valuable information too. This offsets the cost of them.

  • bob1029 6 hours ago

    This reminds me of AMD when zen was taking off.

    If short sellers are so desperate that their crap is showing up on HN, it might be time to buy CBRS.

LoganDark 2 hours ago

I hope this will be affordable one day.

villgax 4 hours ago

Lol why would anyone want to manage multiple vendors for the remotest non-LLM work even. Not everything is an LLM

  • rinpatch 3 hours ago

    Why would you need to do that? The LLM providers integrate disaggregated inference in their stack, you just get a fast inference API for your project.