Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Factorio Learning Environment – Agents Build Factories

jackhopkins.github.io

Post reply on HN
Print view View original

141–150 of 225 posts

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#141

Post by tavavex »

eterm wrote: post #127
tavavex wrote: post #114

Earlier quoted context omitted.

I've played other factory-building games, but not Factorio, so I'm not familiar with the bus-building paradigm. I feel like you're saying that buses would incentivize bad practices, but at the same time I don't see what would make them inherently bad. Whenever I saw screenshots of Factorio, I thought that buses were more of a logistics tool, a way to cable-manage the delivery of stuff from one place to another. Is th…

Buffer is bad precisely because it increases the lag between having a gap in provision, and that gap being obvious to the player. With no buffering, as soon as your demand for steel is greater than than production of steel, then the bottleneck is immediate and obvious. The solution is also immediate and obvious: Build more steel. Buffering, in particular belt buffering, and in particular busses, contribute to mask th…

That makes sense. I guess I just had a different approach when I played the other games. The way I organized in other factory games is that the considerations of input and output were things that I thought of upfront - I never eyeballed and then tried to estimate the production speed based on how fast my resources were drained. I might be overplanning, or maybe Factorio encourages a far more chaotic approach, but I always treated factories as black boxes that take X/s of certain items and outputted X/s results. Knowing precisely how many items per second I have on any individual belt is the most essential piece of knowledge to me, so I never relied on buffering and always made sure to build consumer factories that never overwhelmed producer factories. This means that the visual indication of the buffer draining would only signal some building mistake to me, rather than a design mistake.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#142

Post by HideousKojima »

andai wrote: post #133
Starlord2048 wrote: post #120

[flagged]

Fascinating. I was thinking how the factory should be communicated to the model, and represented "internally". Images aren't the right solution (very high bandwidth for no real benefit). An ASCII grid of the game's tiles (more likely, a small chunk of it) is orders of magnitude better, but you still don't need to simulate every tile in a conveyor. It's just a line, right? So the whole thing is actually a graph! That…

>An ASCII grid of the game's tiles (more likely, a small chunk of it) is orders of magnitude better, but you still don't need to simulate every tile in a conveyor. It's just a line, right? So the whole thing is actually a graph!

Until you accidentally feed a different material into your belt and need to clean it up

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#143

Post by scottmsul »

scottmsul wrote: post #138

There was a HN post here not too long ago about a team that used reinforcement learning to train an agent to beat pokemon red. They mentioned how they had to tweak the cost function to give small rewards for exploring and big rewards for completing "essential tasks" like beating gyms. I wonder if this same approach could be used here in factorio? Using the pokemon red analogy the main "essential tasks" in Factorio ar…

Also I should add, being a Factorio veteran with 2-3k hours in this game, I think the goal of making the "largest possible factory" is too vague and not the right metric. When Factorio players make large megabases, they don't go for "size" per se, but rather science research per minute. The metric you should be telling the agents is SPM, not "largest" base!

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#144

Post by noddybear »

scottmsul wrote: post #138

There was a HN post here not too long ago about a team that used reinforcement learning to train an agent to beat pokemon red. They mentioned how they had to tweak the cost function to give small rewards for exploring and big rewards for completing "essential tasks" like beating gyms. I wonder if this same approach could be used here in factorio? Using the pokemon red analogy the main "essential tasks" in Factorio ar…

In FLE, you have access to milestones representing the first time a new entity was created, but coming up with a stratification of rewards for different degrees of automation would be really interesting. Join us!

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#145

Post by martbakler »

infogulch wrote: post #125

Interesting to see only a handful of complex scenarios. I've always suspected ML game agents need hundreds of tiny puzzles with hundreds of variations each to learn game mechanics properly. Like: The factory is not powered, place the missing power pole(s) The factory is missing items, place the missing belt(s) Craft and place these 200 assembly machines The assembly machine is not running for some reason, fix it The…

We are thinking of something like this (a curriculum approach) for further training. The reason why we didn't want to do this for current work, where the emphasis is on evaluations, is that the "difficulty level" of different tasks is quite subjective and hence we would need to make arbitrary decisions that could affect the evals (i.e which tasks would follow which scenarios, how to ensure sufficient coverage across all difficulty levels etc)

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#146

Post by chompychop »

zoogeny wrote: post #140

This reminds me of how I'll judge LLM ability when it can compete in a 1v1 dark souls battle. Even Factorio is pretty slow compared to what is possible in modern games.

It's edging pretty close to this - multimodal LLMs can already beat early bosses in soulslike games: https://arxiv.org/html/2409.12889v2

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#147

Post by martbakler »

grayhatter wrote: post #118
martbakler wrote: post #92

Earlier quoted context omitted.

Currently it's a text-only modality environment but we are planning to support vision in the future. We did run a couple of tests and saw that including screenshots of the game state did not improve performance on the off-the-shelf models. As the complexity of the game state grew and the screenshots were filled with more entities, the models got even more confused and started hallucinating directions, entities etc or…

> As the complexity of the game state grew and the screenshots were filled with more entities, the models got even more confused and started hallucinating directions, entities etc or weren't capable of troubleshooting factories with apparent mistakes (i.e missing transport belt, wrongly rotated inserter). We think it's because [...] I think you just described a research paper that would advance sota. Less describing…

Sounds almost like a visual "needle in a haystack" type of work, that could be quite interesting!

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#148

Post by devit »

Seems like it might be more effective to use the LLMs to write a program that plays Factorio rather than having them pick the next action given a game state.

Also in general I think the issue with Factorio is that you can just find an "optimal" factory design and build order and just follow it every time; perhaps starting with a suboptimal building layout already present and restrictions like being unable to change them or build others of the same type could help.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#149

Post by noddybear »

devit wrote: post #148

Seems like it might be more effective to use the LLMs to write a program that plays Factorio rather than having them pick the next action given a game state. Also in general I think the issue with Factorio is that you can just find an "optimal" factory design and build order and just follow it every time; perhaps starting with a suboptimal building layout already present and restrictions like being unable to change t…

This is exactly how FLE works, the agent writes a program that executes its policy.

I think you bring up a good point, we could create tasks where the goal is to optimise a static factory, starting from a kernel of functionality like 'steam engine power supply' etc.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#150

Post by mlsu »

Starlord2048 wrote: post #120

[flagged]

Yes!

The way I think of it is this. Yes, the LLM is a "general reasoner." However, it's locked in a box, where the only way in and out is through the tokenizer.

So there's this huge breadth of concepts and meanings that cannot be fully described by words (things like, spatial reasoning, smells, visual relationships, cause/effect physical relationships etc). The list of things that can't be described by words is long. The model would be capable of generalizing on those, it would optimize to capture those. But it can't, because the only thing that can fit through the front door is tokens.

It's a huge and fundamental limitation. I think Yann Lecunn has been talking about this for years now and I'm inclined to agree with him. This limitation is somewhat obscured by the fact that we humans can relate to all of these untokenizable things -- using tokens! So I can describe what the smell of coffee is in words and you can immediately reconstruct that based on my description, even though the actual smell of coffee is not encoded in the tokens of what I'm saying at all.

Post reply on HN