Live data from Hacker News

The First Fully General Computer Action Model

si.inc

61–70 of 85 posts

Re: The First Fully General Computer Action Model

#61
The mouse cursor binning special case is starting to look like how animals perceive, where we detect patterns and develop predictive models over time in how they are going to act, and that confidence leads to more deeply encoding those patterns for lower energy usage. Obviously the mouse cursor is a hand-rolled example in a controlled 2d environment, but it makes me wonder what efficiencies lie in identifying patterns in 3d environments once you construct an accurate enough 3d scene out of the images you have.

Do you have other examples of special cases you're looking at? Any 3d ones?

Re: The First Fully General Computer Action Model

#64

Neel, this is really cool. How long have you been working on this, and where did you guys get inspiration from? Did you work on vlms earlier or something like that? Just curious. Also, thanks for choosing a technical blog post for presenting this information.

thanks! got a lot of inspiration from VPT https://arxiv.org/abs/2206.11795 is a great paper, would recommend a read

we all have various backgrounds, me particularly i did a lot of material science x ai research and just fundamental architecture research before

Re: The First Fully General Computer Action Model

#65

This is one of those hacker news posts that you stumble upon and see 2 genius ideas within the span of as many paragraphs. Thanks again for sharing the diffusion based labeling algorithm. Truly demonstrates a mastery and understanding of what diffusion is capable of.

thanks! i definitely love diffusion + pushed for it, as a non-causal generative method i think its pretty unique
Post reply on HN