I really hope this was an experiment in using gen AI:
“Create a website for a new company that is building the next generation of computing hardware to power AI software. Make sure it sounds science-y but don’t be too specific.”
What Extropic is building
51–60 of 149 posts
Re: What Extropic is building
#52interesting that a company w/ no public repositories has 1.1k github followers https://github.com/extropic-ai
To be fair it isn't very common to detail proprietary hardware in github repos. And any code for such novel processors would be fascinating but useful only for theory rather than practice at the moment. The lack of open code is a missing merit badge rather than a demerit.
Re: What Extropic is building
#53So, basically this seems to be a way to replace PRNGs with real randomness with some knobs so you can adjust the distribution. Let's assume for the sake of argument that this can replace every single PRNG call in inference and training, how much savings in cost/energy/run time would there actually be?
This is a quantum computing company, specifically for quantum ML.
Re: What Extropic is building
#54so far the only thing they’ve built is more posts
TBF that's not a bad place to be in the current hype cycle. Better than releasing and being permanently written off as yet-another-ChatGPT-wrapper.
Re: What Extropic is building
#55smells like snake oil. will probably end up becoming a cryptocurrency scam? or some other grift? time will tell.
>Extropic is also building semiconductor devices that operate at room temperature to extend our reach to a larger market.
funny stuff
Re: What Extropic is building
#56I have no idea about the merits of this approach, but I found this interview with the founders a lot more sensical than the linked article: https://twitter.com/Extropic_AI/status/1767203839818781085
This was definitely easier to follow.
Since they're building a special-purpose accelerator for a certain class of models, what I'd like to see is some evidence that those models can achieve competitive performance (once the hardware is mature). Namely, simulate these models on conventional hardware to determine how effective they are, then estimate what the cost would be to run the same model on Extropic's future hardware.
Re: What Extropic is building
#57Could someone smarter than me explain if this is a big deal or just hype? The work sound promising, but I wonder how long it would take to build and validate.
I don't know about the AI aspect, but this sounds perhaps related to probabilistic finance simulations (Black-Scholes, Heston, etc). I've heard rumors that these types of simulations account for an obscene amount of compute at AWS
Re: What Extropic is building
#58Could someone smarter than me explain if this is a big deal or just hype? The work sound promising, but I wonder how long it would take to build and validate.
I have a suspicion that a lot of people are nodding along because they don't want to seem like the village idiot.
nods vigorously
Re: What Extropic is building
#59I wouldn't want to write this off because you get the feeling these guys are on to something that could be hugely important (ignoring quantum this thermodynamic that) - but surely it feels like they need to get to the point a lot faster e.g. "We're taking a new approach to building chips for AI because transistors can't get any smaller." I really don't know what they gain by convoluting the point and it's pretty hard…
Quantum computing people have been selling this exact spiel (including the convoluted talking points) for decades and it keeps working at getting funded. It has not produced any results for the rest of us, though.
Re: What Extropic is building
#60I’m not a fan of Extropic, but I’m seeing a lot of misconceptions here.
They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs.
See here for similar things: https://arxiv.org/abs/2108.09836
There’s a company called Normal Computing that did something similar: https://blog.normalcomputing.ai/posts/2023-11-09-thermodynam...