I wonder if anyone has done something similar with Dwarf Fortress
Not seen one yet, but the ASCII representation of the game would be ideal for an LLM benchmark.
+/- tokenization mishaps (c.f. strawberry).
Show HN: Factorio Learning Environment – Agents Build Factories
171–180 of 225 posts
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#172Wait are you basically telling me that now I can play factorio with code?
The mod system uses lua, so that was always possible.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#173> [LLMs] yet are unable to operate effectively in constrained environments, reflecting limitations in error analysis This reflects my experience with gen-LLM coding, where LLMs keep trying to do the same thing in a loop.
We once saw GPT4o spend something like 100 repeated interactions trying the action known not to work (before snapping out of it). My intuition here is that this is a result of target fixation - the more repetitions of something it does, the more likely it is to keep repeating it, because it occupies more of the context.
Gemini Pro does this constantly.
* It'll output a broken script * I tell it what's wrong and how to fix it * It tells me I'm absolutely right and that it will correct it * It outputs a script with the exact same brokenness
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#174its funny how video games are the hardest benchmark that humanity has for ai
DeepMind went from playing Pong to protein folding in a short number of years. There are much harder things for AI to do than playing video games. Also see: self driving cars.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#175Earlier quoted context omitted.
It actually is engineering wise quite trivial but the underlying question is which modality is the best to elicit spatial reasoning capabilities from the current general models. We tried (very anecdotally) a couple of months ago to get an agent to reason over a couple of ascii representations of factories and the results weren't very promising. It seems the models struggle with creating an accurate internal spatial r…
Did you try providing 2D vectors of where each object relates to every other object? Seems like the most obvious way. In my experience the current generation of models are very poor at spatial reasoning even when given accurate coordinate based location assignments of each object. But I suspect when a model can build the whole relationship of all objects by being given those spatial relationships in a vector they wil…
We did discuss this at some point but didn't end up trying it out. I think it's quite an interesting avenue and worth a shot, my intuition also says that the spatial capabilities will improve if the model has more access to relative info and doesn't need to infer it from absolute coordinates
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#176There was a HN post here not too long ago about a team that used reinforcement learning to train an agent to beat pokemon red. They mentioned how they had to tweak the cost function to give small rewards for exploring and big rewards for completing "essential tasks" like beating gyms. I wonder if this same approach could be used here in factorio? Using the pokemon red analogy the main "essential tasks" in Factorio ar…
This is interesting, one of our findings was that the Claude was capable of essential tasks & simple automation (i.e iron gear wheel factory in lab-play) but didn't even try to do it during the "build the biggest factory" game episodes. So the models can do these essential tasks but when given a general goal, i.e "complete the game", they don't have a good level of long-term planning to even try to attempt them. Often they just did un-coordinated small-scale constructs without attempting to scale up existing factories
That was also one of our goals, to find out how do the models act when given a very vague and general objective
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#177Their first key insight is interesting. It says that coding ability predicts model performance in the game. I wonder if it also predicts performance of human players in some way?
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#178Seems like it might be more effective to use the LLMs to write a program that plays Factorio rather than having them pick the next action given a game state. Also in general I think the issue with Factorio is that you can just find an "optimal" factory design and build order and just follow it every time; perhaps starting with a suboptimal building layout already present and restrictions like being unable to change t…
This is exactly how FLE works, the agent writes a program that executes its policy. I think you bring up a good point, we could create tasks where the goal is to optimise a static factory, starting from a kernel of functionality like 'steam engine power supply' etc.
But it seems like it's being used to generate short snippets that in the examples seem to be equivalent to command lists as opposed to generating a full program that actually plays the whole game by itself.
The model could also then be fed back the results of running the program and iteratively change it as needed.
I.e. prompt first with "Write a program that can play Factorio automatically given an interface and a set of goals in , and produces text output that can help determine whether the program is working correctly and whether tasks are performed efficiently and goals are reached as fast as possible"
And then with "the program was run and produced this text output: Determine any possible bugs, avenues of improvements or missing output information and modify the program accordingly, printing the new version".
And iterate until there doesn't seem to be an improvement anymore.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#179There was a HN post here not too long ago about a team that used reinforcement learning to train an agent to beat pokemon red. They mentioned how they had to tweak the cost function to give small rewards for exploring and big rewards for completing "essential tasks" like beating gyms. I wonder if this same approach could be used here in factorio? Using the pokemon red analogy the main "essential tasks" in Factorio ar…
Also I should add, being a Factorio veteran with 2-3k hours in this game, I think the goal of making the "largest possible factory" is too vague and not the right metric. When Factorio players make large megabases, they don't go for "size" per se, but rather science research per minute. The metric you should be telling the agents is SPM, not "largest" base!
ahhh another factorio addict :)
Curious, how long was your first play through (assuming in v1.x lanching the first rocket)
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#180Earlier quoted context omitted.
Regarding the 2d image - the issue is that these frontier models don't tend to support supplemental image inputs, and the ones that do aren't sufficiently well trained on (high precision) Factorio visuals to add that much information.
I see, integrating image inputs can be very challenging in this case as the models work with text input. I was not even thinking about the full isometric image, but just some simple 2D map where each pixel can be color-coded based on the entity type. I guess the problem is that these maps would look like nothing the models were trained on, so as you say, it might not provide any value. The reason I was suggesting thi…
Indeed I think the trade-off here is the more "pure factorio" types of images we give to the agents, the more likely it is that they've seen it during training (from google etc), however the signal-to-noise ratio is low and hence the current models get confused as the map complexity (amount of entities) and level of detail grows. If we start to create custom images, we can reduce the unneeded noise, but then risk giving something completely OOD to the agent (unless we train a visual encoder) and the performance also tanks