Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Extropic is building

extropic.ai

Post reply on HN
Print view View original

111–120 of 149 posts

Re: What Extropic is building

#111

Post by HarHarVeryFunny »

HarHarVeryFunny wrote: post #109
blovescoffee wrote: post #27

Earlier quoted context omitted.

This interview makes their product seem like BS. First, they literally cannot simply explain the problem or solution. Regardless, their pitch is that they're building a more power efficient probability distribution sampler. No one in AI research thinks that's a bottleneck. edit: btw the bottleneck in AI algos is matrix multiply and memory bandwith.

My take on the Garry Tan interview (which seems pretty clear, regardless of whether this is snake oil or not) is that Extropic are building low power analog chips because we're hitting up against the limits of Moore's Law (limit's of physics in reducing transistor size), and at the same time the power consumption for LLM/AI training and inference is starting to get out of hand. So, their solution is to embrace the st…

Just watched a few minutes of the Lex interview, and have to say Verdon gives off a totally different vibe there, and seems to be talking gibberish about quantum computing.

However, the idea of using analog matrix multiply is reasonable, and has already been done by at least one company:

https://mythic.ai/products/m1076-analog-matrix-processor/

Re: What Extropic is building

#112

Post by ks2048 »

swalsh wrote: post #66
binoct wrote: post #51

I really hope this was an experiment in using gen AI: “Create a website for a new company that is building the next generation of computing hardware to power AI software. Make sure it sounds science-y but don’t be too specific.”

Why make such a low effort pessimistic comment. What happened to HN?

I think pointing out BS is an important part of a useful forum.

Re: What Extropic is building

#113

Post by zooq_ai »

nfw2 wrote: post #6
jdulay19 wrote: post #2

Could someone smarter than me explain if this is a big deal or just hype? The work sound promising, but I wonder how long it would take to build and validate.

I have a suspicion that a lot of people are nodding along because they don't want to seem like the village idiot.

on the other hand, HN is filled with self-proclaimed critics that they dismiss everything and display their utter lack of imagination -- like AI, Metaverse, (success of) Snapchat, AirPods before

Re: What Extropic is building

#114

Post by ein0p »

sfnrm wrote: post #99
ein0p wrote: post #76

People need to read Hamming’s old papers in which he very clearly explains why analog circuits are not viable at scale. This is also why the brain uses spikes rather than continuous signals. The issue is noise, interference, and attenuation. There’s no way to get around this. If they have invented a way, I’d like to see it. But until it’s demonstrated, I’d take such things with a large grain of salt.

Sounds interesting. Do you have a link? (or at least a title?)

Not at the moment, but I do recall he has a chapter on this in his book “The Art of Doing Science and Engineering”, which I also recommend. He uses very long transmission lines to explain this, but the same thing applies at the nano scale, and perhaps to an even greater extent due to the much noisier environment and higher frequencies.

Re: What Extropic is building

#115

Post by winwang »

pclmulqdq wrote: post #97
winwang wrote: post #88

Earlier quoted context omitted.

Skimmed the litepaper. Has the flavor of: you can do "simulated" annealing by literally annealing. I like the idea of using raw physics as a "hardware" accelerator, i.e. analog computing. fwiw, quantum computing can be seen as a form of analog computing. I do think that a "better rng" can be interesting and useful in and of itself. Thanks for the Normal Computing post, it felt more substantial.

I make a better RNG right now ( https://arbitrand.com ). We experimented with doing ML training with it, but it's not clear that it trains any better than a non-broken PRNG. It might be fun to feed the output into stable diffusion and see how cool the pictures are, though.

Cloud RNG number streaming is interesting but costly, no? I did have the idea to serve truly random numbers via a quantum computer (trivial by just preparing the simplest state and measuring). Anything else can't be said to be truly random.

Re: What Extropic is building

#116

Post by lumost »

deepnotderp wrote: post #60

I’m not a fan of Extropic, but I’m seeing a lot of misconceptions here. They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs. See here for similar things: https://arxiv.org/abs/2108.09836 There’s a company called Normal Computing that did somet…

It did make me curious however, if we dropped the requirement that operations return correct values in favor of probably correct values - would we see any material computing gains in hardware? Large neural models are intrinsically error correcting and stochastic.

I’m unfortunately not familiar enough with hardware to weigh in.

Re: What Extropic is building

#117

Post by pclmulqdq »

winwang wrote: post #115
pclmulqdq wrote: post #97

Earlier quoted context omitted.

I make a better RNG right now ( https://arbitrand.com ). We experimented with doing ML training with it, but it's not clear that it trains any better than a non-broken PRNG. It might be fun to feed the output into stable diffusion and see how cool the pictures are, though.

Cloud RNG number streaming is interesting but costly, no? I did have the idea to serve truly random numbers via a quantum computer (trivial by just preparing the simplest state and measuring). Anything else can't be said to be truly random.

You don't need a quantum computer to sample noise from quantum processes.

It's a prohibitively expensive way to go, and depending on how you built the quantum computer, it may be more susceptible to interference and non-quantum noise than using good circuits and custom systems.

Re: What Extropic is building

#119

Post by throwawaymaths »

deepnotderp wrote: post #60

I’m not a fan of Extropic, but I’m seeing a lot of misconceptions here. They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs. See here for similar things: https://arxiv.org/abs/2108.09836 There’s a company called Normal Computing that did somet…

So it sounds like this startup is explicitly not using foundation models?

Is there any evidence that such a probabilistic model can run better than a state of the art model?

Or alternatively what would it take to convert an existing model (let's say, an easy one like llama2-7b) into an extropic model?

Re: What Extropic is building

#120

Post by arduanika »

Eliezer wrote: post #93
arduanika wrote: post #85

Earlier quoted context omitted.

Is this sarcasm? (Genuinely can't tell.) And also, are you the real Eliezer?

No, not sarcasm, and I am Eliezer Yudkowsky. I was around on the old Extropians mailing list starting in 1996, and their leadership did not talk like this. Max More (the founder of Extropianism) was a careful thinker then, and I haven't heard anything different about him more recently than that. "Extropy" is a term that was previously coined by a group of fairly nice people to describe themselves, and so far as I kno…

Oh wow, hello!

Seems like it was quite a time to be online. I mostly know of it through this version of events. Not sure how accurate you'd find it:

https://aiascendant.substack.com/p/extropias-children-chapte...

The word "extropy" itself seems to go back several decades before the mailing list, if I'm reading correctly here: https://en.wikipedia.org/wiki/Extropianism. Still, I wouldn't be surprised if many/most of the original mailing list members found this usage a corruption.

Post reply on HN