121–130 of 149 posts
Can anyone name a company which used such absurd language to describe themselves and then actually delivered something valuable? There must be one.
Earlier quoted context omitted.
It's led by the e/acc [1] founder, BasedBeffJezos [2]. He has a huge cult following. It's turned into a lot of Twitter memes and shitposting [3]. [1] https://en.wikipedia.org/wiki/Effective_accelerationism [2] https://twitter.com/BasedBeffJezos [3] https://knowyourmeme.com/memes/cultures/eacc-effective-accel...
Honestly, as interesting as the the chip sounds, I'm admittedly kind of biased against the company's probability of success simply because the founder is basically the #1 e/acc meme account/shitposter on Twitter. Like, it's hard to take someone seriously when they spend tons of time shitposting on Twitter, it's even harder when it's revealed that they're behind one of the most popular shitposting accounts within a ni…
Today, it's a very different situation. Now we have Elon Musk, who is really good at what he does, and also tweets a lot...
Hard time believing this is legit given how much time the CEO spends goofing around on social media. If it were possible to short startups, this would be a top candidate.
honestly, it would be too early to say this. Considering the people who invested in this startup, its better to assume CEO is capable. If he is not able to deliver in reasonable timeline then, we all are free to blame him for posting things on SM. actually many knows his company because he is goofing around on SM especially e/acc stuff.
This whole pitch sounds like the usual quantum computing babble.
I’m not a fan of Extropic, but I’m seeing a lot of misconceptions here. They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs. See here for similar things: https://arxiv.org/abs/2108.09836 There’s a company called Normal Computing that did somet…
It did make me curious however, if we dropped the requirement that operations return correct values in favor of probably correct values - would we see any material computing gains in hardware? Large neural models are intrinsically error correcting and stochastic. I’m unfortunately not familiar enough with hardware to weigh in.
What you want is low precision with stochastic rounding. Graphcore's IPUs have that and it's a really great feature. It lets you use really low precision number formats but effectively "dithers" the error. Same thing as dithering images or noise shaping audio.
Earlier quoted context omitted.
It did make me curious however, if we dropped the requirement that operations return correct values in favor of probably correct values - would we see any material computing gains in hardware? Large neural models are intrinsically error correcting and stochastic. I’m unfortunately not familiar enough with hardware to weigh in.
The trouble is if you use actual randomness then you lose repeatability which is an incredibly useful property of computers. Have fun debugging that! What you want is low precision with stochastic rounding. Graphcore's IPUs have that and it's a really great feature. It lets you use really low precision number formats but effectively "dithers" the error. Same thing as dithering images or noise shaping audio.
Big challenge would be whether any gains could compete with the economy of scale from NVidia.
Earlier quoted context omitted.
The use of “full-stack” was the first thing I noticed. Everyone, please stop using that term. I’m pretty sure, with a high degree of certainty, you don’t know what it means. If you do, there’s a merit badge waiting for you. And can we please stop using “hallucinations” to describe output. Yes, it may look like your tool dropped acid, but that’s not what it is.
full-stack means the ic can take any ticket. do the details beyond that matter?
Earlier quoted context omitted.
This interview makes their product seem like BS. First, they literally cannot simply explain the problem or solution. Regardless, their pitch is that they're building a more power efficient probability distribution sampler. No one in AI research thinks that's a bottleneck. edit: btw the bottleneck in AI algos is matrix multiply and memory bandwith.
My take on the Garry Tan interview (which seems pretty clear, regardless of whether this is snake oil or not) is that Extropic are building low power analog chips because we're hitting up against the limits of Moore's Law (limit's of physics in reducing transistor size), and at the same time the power consumption for LLM/AI training and inference is starting to get out of hand. So, their solution is to embrace the st…
I wouldn't want to write this off because you get the feeling these guys are on to something that could be hugely important (ignoring quantum this thermodynamic that) - but surely it feels like they need to get to the point a lot faster e.g. "We're taking a new approach to building chips for AI because transistors can't get any smaller." I really don't know what they gain by convoluting the point and it's pretty hard…
Quantum computing people have been selling this exact spiel (including the convoluted talking points) for decades and it keeps working at getting funded. It has not produced any results for the rest of us, though.
Earlier quoted context omitted.
My take on the Garry Tan interview (which seems pretty clear, regardless of whether this is snake oil or not) is that Extropic are building low power analog chips because we're hitting up against the limits of Moore's Law (limit's of physics in reducing transistor size), and at the same time the power consumption for LLM/AI training and inference is starting to get out of hand. So, their solution is to embrace the st…
I'm sorry this may come off as rude, not my intention: The Gary Tan interview explicitly says those things, I'm not sure that's really your "take".
Earlier quoted context omitted.
Computationally, yes, those are the bottlenecks. But I would also add supervised training data, as we can never get enough of that and it is one of few things that increases in compute are (to my mind, you could argue that by scaling unsupervised training further we could do away with it, but I am not yet convinced) not able to solve.
Their startup is addressing computing bottlenecks so that's what I addressed. Supervised training dat isn't a bottleneck on LLMs, Diffusion models, or any of the hot areas at the moment.
Still, you are correct that Extropic is looking at the computation rather than data. But, I wanted to chime in so as the discussion here would not leave the impression that we are still in the days of pure unsupervised scaling.