Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agents that “self-reflect” perform better in changing environments

hai.stanford.edu

Post reply on HN
Print view View original

31–40 of 46 posts

Re: AI agents that “self-reflect” perform better in changing environments

#31

Post by ftxbro »

tnecniv wrote: post #29
ftxbro wrote: post #12

Earlier quoted context omitted.

OK here is the arxiv https://arxiv.org/abs/2306.15934 called "Curious Replay for Model-based Adaptation" and from the abstract it says "we present Curious Replay -- a form of prioritized experience replay tailored to model-based agents through use of a curiosity-based priority signal" and "DreamerV3 with Curious Replay surpasses state-of-the-art performance on Crafter" here is the crafter benchmark https://github.com…

That’s standard. Me and others in my PhD cohort have had experiences where we saw so many minor inaccuracies in the copy we only fixed things that were flat out wrong, otherwise we’d have rewritten the whole article. It’s the result of a combination of non-experts having a 30 minute conversation with you then writing based off their notes a week later and the fact that their job is to hype up research so that it gets…

yes it's better now that the hn mods have changed the headline

Re: AI agents that “self-reflect” perform better in changing environments

#32

Post by uoaei »

mandmandam wrote: post #24
Schnitzkitz wrote: post #21

Earlier quoted context omitted.

By pointing to rational or moral failures, you already imply that we are supposed to act in a certain way. If there are people who are the worst, it begs the question of what a good human is, and who or what we should actually follow. Clearly, we don't think that raw power is what makes someone good, because otherwise these worst people on the pedestals would be by default good people, through all the power they have…

> By pointing to rational or moral failures, you already imply that we are supposed to act in a certain way. Don't keep such an open mind that your brain falls out. > If it is irrational that history repeats itself, do you think that it would be rational if history progressed towards some goal It has, often. For example, 50 years ago a bunch of fossil fuel executives decided it would be best to let the planet burn, s…

Rationality, and by extension rationalism, refuses to investigate the question of whether the axiomatic assumptions upon which the rational conclusions are based are valid.

So of course superyachts are perfectly rational. But of course they are far from reasonable.

Re: AI agents that “self-reflect” perform better in changing environments

#34

Post by PaulDavisThe1st »

mandmandam wrote: post #24
Schnitzkitz wrote: post #21

Earlier quoted context omitted.

By pointing to rational or moral failures, you already imply that we are supposed to act in a certain way. If there are people who are the worst, it begs the question of what a good human is, and who or what we should actually follow. Clearly, we don't think that raw power is what makes someone good, because otherwise these worst people on the pedestals would be by default good people, through all the power they have…

> By pointing to rational or moral failures, you already imply that we are supposed to act in a certain way. Don't keep such an open mind that your brain falls out. > If it is irrational that history repeats itself, do you think that it would be rational if history progressed towards some goal It has, often. For example, 50 years ago a bunch of fossil fuel executives decided it would be best to let the planet burn, s…

This is a major question in philosophy, not just some random aside in an HN comment thread.

Most famously, Hegel believed that human history trends & tends towards the perfection of human nature and society. Many other philosophers and philosophies fundamentally disagree with Hegel, and assert that history has no teleological purpose built into it.

Perhaps acknowledge the depth and history of this question before throwing out some quick asides about it?

Re: AI agents that “self-reflect” perform better in changing environments

#35

Post by selalipop »

kromem wrote: post #14
neuronerd1 wrote: post #11

Earlier quoted context omitted.

Author here, a key thing is that we didn't prescribe that the mechanism of exploration was the same, but rather we found that the AI agent explored poorly (i.e. unlike animals) until we included Curious Replay. Interestingly, we found that the benefits of Curious Replay also led to state of the art performance on Crafter.

It's very cool work. I've been wondering for a while at what the next steps in adding 'inefficiencies' to AI processing would look like, commenting the other day to a friend that what's needed in the next 18 months is getting AI to be able to replicate the Eureka moments in the shower where latent information is reconstructed in parallel to processing tangential topics. Going from "attention is all you need" to "atte…

> getting AI to be able to replicate the Eureka moments in the shower where latent information is reconstructed in parallel to processing tangential topics.

I've been playing with this part specifically and it's really amazing stuff.

Having the model concurrently model internal monologue and output for a task, but allowing the internal monologue to be as focused or unfocused as the model sees fit.

You end up with situations where you have it working on a naming task for example, and the model starts imagining the warmth of a coffee cup on the desk, or traffic building up outside for a future appointment with a non-existent person, and then returns back to the task at hand with non-obvious tangents that it'd probably never have uncovered if it was only predicting on tokens related to the original goal of naming something.

It gets even more interesting when you inject variability into the process via the API (for example, telling it to use certain letters pulled from an RNG inside the next iteration of internal monologue).

Re: AI agents that “self-reflect” perform better in changing environments

#38

Post by ly3xqhl8g9 »

Perhaps one would drop the quotes around self-reflect if one would implement something more akin to a Markov blanket [1], blankets within blankets, model ourselves modelling the world.

[1] 2018, "The Markov blankets of life: autonomy, active inference and the free energy principle", https://royalsocietypublishing.org/doi/10.1098/rsif.2017.079...

Re: AI agents that “self-reflect” perform better in changing environments

#39

Post by TechnologyPast »

dang wrote: post #27
ftxbro wrote: post #2

So from this hacker news title I definitely thought it was saying that when you give some AI agents a self reflection like maybe by putting an internal monologue loop then they unlock an emergent animal-like exploration behavior. But this is not what happened. Instead, some guys told AI agents to explore in the way that the guys think that animals explore. "Stanford researchers invented the “curious replay” training…

(Submitted title was "“Self-reflecting” AI agents explore like animals". We changed it in keeping with the HN guidelines - https://news.ycombinator.com/newsguidelines.html .)

Hi dang. Can you whitelist some URLs for commenting from a new account? Like wikipedia.org and libquotes.com

Looks like you shadowbanned this account. Maybe for posting a URL in the first comment.

Re: AI agents that “self-reflect” perform better in changing environments

#40

Post by sethammons »

I'm not an AI expert or even novice nor am I a neuroscientist, but I have been thinking about how I interact with the world.

My current imagining says that novelty and unexpected inputs drive our immediate understanding of the world around us. To have expectations you have to have to have a model. When that model breaks and is adjusted you have a novel experience and the model can be updated. This feedback loop is critical.

Example: other day I was grilling food and my digital food thermometer was on the metal prep area near the hot griddle. As I was walking away I reached for it, grabbed it, and expected to pick it up. However! I didn't know it had a magnet and it gave me back unexpected stimulus.

I immediately jerked my hand away and several thoughts happened near instantly. My thoughts went from I burned my hand to no, no pain, maybe a really bad burn, to no, no heat, no sizzling of flesh, to oops, wrong stimulus, something resisted, resisted how, it slid but wouldn't pick up easy, ah, a magnet.

The researchers here are right, I expect. You need curiosity and some goal, but you need to constantly tune the input for expectations and tweak the (mental) model of the world.

How many times do you, for a split second, totally misinterpret what you see or feel but near instantly self correct? Better AI will require putting forth it's initial result and then validating the result with feedback. The more unexpected the feedback the more novel the experience and more learning that can happen.

Post reply on HN