Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

100x defect tolerance: How we solved the yield problem

cerebras.ai

Post reply on HN
Print view View original

151–160 of 186 posts

Re: 100x defect tolerance: How we solved the yield problem

#151

Post by idiotsecant »

dogcomplex wrote: post #141
ChuckMcM wrote: post #2

I think this is an important step, but it skips over that 'fault tolerant routing architecture' means you're spending die space on routes vs transistors. This is exactly analogous to using bits in your storage for error correcting vs storing data. That said, I think they do a great job of exploiting this technique to create a "larger"[1] chip. And like storage it benefits from every core is the same and you don't nee…

Of course many people are going to collectively lose trillions, AI's a very highly hyped industry with people racing into it without an intellectual edge and any temporary achievement by any one company will be quickly replicated and undercut by another using the same tools. Economic success of the individuals swarming on a new technology is not a guarantee whatsoever, nor is it an indicator of the impact of the tech…

>it's gonna rewire the way everything else operates too.

This is the part that I think a lot of very tech literate people don't seem to get. I see people all the time essentially saying 'AI is just autocomplete' or pointing out that some vaporware ai company is a scam so surely everyone is.

A lot of it is scams and flash in the pan. But a few of them are going to transform our lives in ways we probably don't even anticipate yet, for good and bad.

Re: 100x defect tolerance: How we solved the yield problem

#152

Post by idiotsecant »

mschuster91 wrote: post #81
__Joker wrote: post #75

Earlier quoted context omitted.

"While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage" Can you please explain more why you think so ? Thank you.

It's a hype cycle with many of the hypers and deciders having zero idea about what AI actually is and how it works. ChatGPT, while amazing, is at its core a token predictor, it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GP…

All the big LLMs are no longer just token predictors. They are beginning to incorporate memory, chain of thought, and other architectural tricks that use the token predictor in novel ways to produce some startlingly useful output.

It's certainly the case that an LLM alone cannot achieve AGI. As a component of a larger system though? That remains to be seen. Maybe all we need to do is duct tape a limbic system and memory onto an LLM and the result is something sort of like an AGI.

It's a little bit like saying that a ball bearing can't possibly ever be an internal combustion engine. While true, it's sidestepping the point a little bit.

Re: 100x defect tolerance: How we solved the yield problem

#153

Post by idiotsecant »

Shorel wrote: post #147
dsign wrote: post #118

Earlier quoted context omitted.

Those glorified token predictors are the missing piece in the puzzle of general intelligence. There is a long way to go still in putting all those pieces together, but I don't think any of the steps left are in the same order of "we need a miracle breakthrough". That said, I believe that this is going one of two ways: we use AI to make things materially harder for humans, in a scale from "you don't get this job" to "…

I'm sure there are more missing pieces. We are more than Broca's areas. Our intelligence is much more than linguistic intelligence. However, and this is also an important point, we have built language models far more capable than any language model a single human brain can have. Makes me shudder in awe of what's going to happen when we add the missing pieces.

Yes, I sometimes wonder if what we're witnessing in our lifetimes is the next stage of the 'bootstrapping' of life into a more complex form. If we might be the mitochondria contributing our little piece to the cell that comes after.

Re: 100x defect tolerance: How we solved the yield problem

#154

Post by idiotsecant »

anon373839 wrote: post #106
KronisLV wrote: post #98

Earlier quoted context omitted.

> And just as every other hype cycle, this one will crash down hard. Isn't that an inherent problem with pretty much everything nowadays: crypto, blockchain, AI, even the likes of serverless and Kubernetes, or cloud and microservices in general. There's always some hype cycle where the people who are early benefit and a lot of people chasing the hype later lose when the reality of the actual limitations and the real…

> I don't think the current "AI" is special in any way As someone who loves to pour ice water on AI hype, I have to say: you can't be serious. The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years. It's very different from crypto, the main use case for which appe…

>the main use case for which appears to be money laundering.

You say tomato, I say freedom from the tyranny of fiat power structures.

Re: 100x defect tolerance: How we solved the yield problem

#155

Post by emkee »

kragen wrote: post #145
ChuckMcM wrote: post #117

Earlier quoted context omitted.

It could diagram a sentence it had never seen.

I did eventually get an LLM to produce what seems to be a correct diagram of a sentence it had never seen, but it took about ten tries. Grammatical analysis seems to have happened correctly every time, but getting to a usable diagram was difficult. (I know that it's generally rude to include LLM output in HN comments, but in this case I think it's essential supporting material to elevate the discussion of LLM capabil…

Having only taken one syntax class for fun in college, I find this pretty impressive. Generating syntax trees was never a trivial task for me (but I was just a CS major who needed a credit). Slightly related, but I have also never had ChatGPT successfully generate ASCII art, even with extensive conversation.

Re: 100x defect tolerance: How we solved the yield problem

#156

Post by myrmidon »

Shorel wrote: post #148
immibis wrote: post #131

Earlier quoted context omitted.

Why do you think that an AGI can't be a token predictor?

By analogy with human brains: Because our own brains are far more than the Broca's areas in them. Evolution selects for efficiency. If token prediction could work for everything, our brains would also do nothing else but token prediction. Even the brains of fishes and insects would work like that. The human brain has dedicated clusters of neurons for several different cognitive abilities, including face recognition,…

> Evolution selects for efficiency.

I think this is a poor argument here. From an evolutionary point of view, our brains are optimized to:

- Provide fine-motor control to craft weapons and tools (enhancing adaptibility and enabling us to hunt way outside our weight class)

- Communicate/coordinate effectively in small groups

- Do sensor processing and the above with a low energy budget

Our brains are *not* selected to be minimum-complexity intelligences, and a lot of what our brain does is completely useless for AGI building (motor control, sensor processing, ...).

Furthermore, the cost/complexity (from a evolutionary PoV) is a totally different beast from what complexity means to us.

Just consider flight as an example: A fruitfly is an insanely simple and straightforward beast, but to us, a biochemically fuelled, beating-wing design is still infeasibly complicated. If our approach to flight had been to ape after how nature does it in detail, we likely still would not have planes.

I do agree that todays LLMs still have clear architectural flaws that we need to overcome (online learning being a very glaring one), but, to pick up the flight analogy, we might well have the main wing structure already down, and we won't necessarily have to make those wings beat to get into the air...

Re: 100x defect tolerance: How we solved the yield problem

#157

Post by kragen »

emkee wrote: post #155
kragen wrote: post #145

Earlier quoted context omitted.

I did eventually get an LLM to produce what seems to be a correct diagram of a sentence it had never seen, but it took about ten tries. Grammatical analysis seems to have happened correctly every time, but getting to a usable diagram was difficult. (I know that it's generally rude to include LLM output in HN comments, but in this case I think it's essential supporting material to elevate the discussion of LLM capabil…

Having only taken one syntax class for fun in college, I find this pretty impressive. Generating syntax trees was never a trivial task for me (but I was just a CS major who needed a credit). Slightly related, but I have also never had ChatGPT successfully generate ASCII art, even with extensive conversation.

Yeah, I think it would be very challenging for most people. It did considerably better with Graphviz than with ASCII art, but it still had trouble with the transition from a perfectly correct and highly nuanced verbal grammatical analysis to Graphviz. I think this is pretty convincing evidence against ChuckMcM's implicit position. It's weaker evidence against mschuster91's explicit position because parsing is something computers have been doing for a long time, so it doesn't imply any new capabilities.

I'm pretty sure there are part-of-speech tagging parsers using fairly shallow statistics that could also have produced an equivalently good sentence diagram. https://corenlp.run/ seems to produce a correct parse, though in a different format.

Re: 100x defect tolerance: How we solved the yield problem

#158

Post by immibis »

Shorel wrote: post #148
immibis wrote: post #131

Earlier quoted context omitted.

Why do you think that an AGI can't be a token predictor?

By analogy with human brains: Because our own brains are far more than the Broca's areas in them. Evolution selects for efficiency. If token prediction could work for everything, our brains would also do nothing else but token prediction. Even the brains of fishes and insects would work like that. The human brain has dedicated clusters of neurons for several different cognitive abilities, including face recognition,…

What do our brains do that isn't token prediction?

They receive information about photons and air vibrations and control muscles, okay. If a human brain was hooked up the way ChatGPT was, only to text input and output, would that make it not intelligent?

Re: 100x defect tolerance: How we solved the yield problem

#159

Post by ryao »

jjk166 wrote: post #62
ryao wrote: post #58

> Take the Nvidia H100 – a massive GPU weighing in at 814mm2. Traditionally this chip would be very difficult to yield economically. But since its cores (SMs) are fault tolerant, a manufacturing defect does not knock out the entire product. The chip physically has 144 SMs but the commercialized product only has 132 SMs active. This means the chip could suffer numerous defects across 12 SMs and still be sold as a flag…

Redundant cores lead to a fault tolerant chip.

ECC memory is fault tolerant. It repairs issues on the fly without disabling hardware. This on the other hand is merely redundant to handle manufacturing defects. If they make a mistake and ship a bad core that malfunctions at runtime, it is not going to tolerate that.

Re: 100x defect tolerance: How we solved the yield problem

#160

Post by Retric »

idiotsecant wrote: post #151
dogcomplex wrote: post #141

Earlier quoted context omitted.

Of course many people are going to collectively lose trillions, AI's a very highly hyped industry with people racing into it without an intellectual edge and any temporary achievement by any one company will be quickly replicated and undercut by another using the same tools. Economic success of the individuals swarming on a new technology is not a guarantee whatsoever, nor is it an indicator of the impact of the tech…

>it's gonna rewire the way everything else operates too. This is the part that I think a lot of very tech literate people don't seem to get. I see people all the time essentially saying 'AI is just autocomplete' or pointing out that some vaporware ai company is a scam so surely everyone is. A lot of it is scams and flash in the pan. But a few of them are going to transform our lives in ways we probably don't even ant…

I’m not so sure it’s going to even do that much. People are currently happy to use LLM’s, but the outputs aren’t accurate and don’t seem to be improving quickly.

A YouTuber watch regularly includes questions they asked Chat GPT and very single time there’s a detailed response in the comments showing how the output is wildly wrong from multiple mistakes.

I suspect the backlash from disgruntled users is going to hit the industry hard and these models are still extremely expensive to keep updated.

Post reply on HN