Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Factorio Learning Environment – Agents Build Factories

jackhopkins.github.io

Post reply on HN
Print view View original

211–220 of 225 posts

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#211

Post by noddybear »

vessenes wrote: post #86
noddybear wrote: post #59

Earlier quoted context omitted.

Agents don't have access to screenshots, as we are purely evaluating text-only models. All reasoning is conducted over object representations of the game (with positions etc). I have anecdotally tried using screenshots to help models debug their factories, but without training a custom CNN/ViT on the Factorio UI, the visual outputs miss critical things (e.g gaps in transport belts). That said, we have demonstrated vi…

Noddy, what’s “fair game” for this benchmark? e.g. do you wish to provide frontier models with a text goal, tooling info, and leave it at that? Or do you wish to have agent architectures compete? It seems to me like tiering the goal setting, layout and implementation are all separate tasks that would benefit from different agents.

The idea is for us to track all frontier models using the basic agent (goal, tooling info), and then offer another leaderboard for different agent architectures (with retrieval etc).

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#212

Post by Gasp0de »

scottmsul wrote: post #138

There was a HN post here not too long ago about a team that used reinforcement learning to train an agent to beat pokemon red. They mentioned how they had to tweak the cost function to give small rewards for exploring and big rewards for completing "essential tasks" like beating gyms. I wonder if this same approach could be used here in factorio? Using the pokemon red analogy the main "essential tasks" in Factorio ar…

Did you read the page? Because they did give rewards per item produced, and more complex items gave higher rewards.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#213

Post by noddybear »

devit wrote: post #178
noddybear wrote: post #149

Earlier quoted context omitted.

This is exactly how FLE works, the agent writes a program that executes its policy. I think you bring up a good point, we could create tasks where the goal is to optimise a static factory, starting from a kernel of functionality like 'steam engine power supply' etc.

But it seems like it's being used to generate short snippets that in the examples seem to be equivalent to command lists as opposed to generating a full program that actually plays the whole game by itself. The model could also then be fed back the results of running the program and iteratively change it as needed. I.e. prompt first with "Write a program that can play Factorio automatically given an interface and a s…

If I understand you correctly, this approach is sort of supported in FLE - the agents can create functions that encapsulate more complex logic. However, interaction is still synchronous/turn-based. I think to do what you propose, you will need to create event listeners that can trigger the agents program whenever appropriate.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#214

Post by noddybear »

jharohit wrote: post #196
noddybear wrote: post #21

Earlier quoted context omitted.

So the core insight was that we can take over the Factorio console remotely using RCON over TCP. From this, we implemented a server-side library of tools that run inside the game. We then implemented a client-side Python library that can invoke these tools - resulting in a Python API for the game. A nice side effect is that creating new tools is really easy, and they can be hot-loaded into running game servers (unlik…

this is cool! How could one extend that to a broader set of games? E.g another one where you can run larger simulations on behaviour are procedural games like No Man's Sky

This specific approach relied on: a) availability of multiplayer servers, and b) a remotely accessible console.

I know Minecraft works in the same way - but I’m not sure about RPGs like NMS.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#215

Post by martbakler »

jxjnskkzxxhx wrote: post #203

I don't understand - were these models post-trained to play factorio? A) If so, how is that possible given that e.g. Claude doesn't have public weights? B) If not, how would the agent know what the API does? Even if it's "guessing" from the English meaning of the API commands (e.g. place_entity_next_to places entity next to something), how would it know what the recipes are? If it's trying and learning we go back to…

To also jump in here, regarding tools the agents had access to function signatures (i.e tool docstrings, input and output types) and for each tool a small "manual", which described what the tool does, how it affects the game state and a small number of examples where using this tool would be useful (for instance, how to use place_entity_next_to to put an inserter next to an existing chest)

Overall as Jack said, no post-training was done at all but all agents had a complete API description (tools, entities, research) in their context so the results indicate to some level how well can modern agents use a completely OOD API with decent level of documentation

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#218

Post by infogulch »

noddybear wrote: post #210
infogulch wrote: post #137

Earlier quoted context omitted.

Yes each scenario will need someone to design it, but you can get a lot of mileage out of each. E.g. consider the "place the missing power pole" scenario: manually build a factory with a few dozen machines connected to a couple steam engines with 20 power poles, then you can generate 400 playable puzzles/scenarios by deleting 1-2 power poles from the working starting point. Humans would find all of these to be equiva…

Oh super interesting! Create 10 scenarios containing working factories, and ‘drop out’ entities to break the factory in different ways. great idea.

Yes exactly! This approach can generate hundreds of "fix the problem"-type tests very easily. With some creative thinking I suspect you can use variations to stack multipliers on other types of tests as well.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#219

Post by noosphr »

noddybear wrote: post #208
noosphr wrote: post #192

>We evaluate six frontier language models across both settings: Claude 3.5-Sonnet, GPT-4o, GPT-4o-Mini, Deepseek-v3, Gemini-2-Flash, and Llama-3.3-70B-Instruct. While I appreciate the effort and creativity that went into this there are a lot of much simpler dynamic benchmarks that can let you saturate the planning capabilities of non-reasoning models. Something as simple as giving a list of flight connections between…

This is true - there are simpler benchmarks that can saturate planning for these models. We were motivated to create a broader spectrum eval, to test multiple capabilities at once and remain viable into the future.

That's fair enough, but you should test other frontier model types to see if the benchmark makes sense for them.

For example the shortest path benchmark is largely useless when you look at reasoning models - since they have the equivalent of scratch paper to work through their answers the limitation became their context length rather than any innate ability to reason.

Re: Show HN: Factorio Learning Environment – Agents Build Factories

#220

Post by jxjnskkzxxhx »

noddybear wrote: post #206
jxjnskkzxxhx wrote: post #203

I don't understand - were these models post-trained to play factorio? A) If so, how is that possible given that e.g. Claude doesn't have public weights? B) If not, how would the agent know what the API does? Even if it's "guessing" from the English meaning of the API commands (e.g. place_entity_next_to places entity next to something), how would it know what the recipes are? If it's trying and learning we go back to…

These models were not post-trained - all off-the-shelf. We can fit about 128 pairs maximum in the context, but this performed the same as 32, which we ultimately decided on (for cost, latency purposes). Encoding the input/outputs to make them shorter degraded performance. It seems that descriptive names is helpful for pretrained models because they have an intuition on what they do.

Follow up. Do you have an hypothesis why Claude performs much better than the rest at these tasks?

Is it just because Clause is the best at coding and the API is code? (not very interesting). Maybe if the API required the llms to write in poems, the best LLM at poetry would win...

Or is it because whatever makes claude good at coding, also makes it good at mathematical-like tasks. This is more interesting, as it would show some transfer learning. It would also suggest if you're doing training for a specific task, you would also benefit from training adjacent tasks e.g. if you're training for maths you could benefit from training coding. I believe this is actually true for humans.

And would you know how to check whether if any of the above hypothesis is correct?

Post reply on HN