Everytime a paper like this comes out, I always have 1 question - How do they control the game using the LLMs? How does the control-feedback loop work? WHat tools, software and APIs they use to do it on Mac or Windows?
Show HN: Factorio Learning Environment – Agents Build Factories
11–20 of 225 posts
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#12Incredible idea and execution, very interesting results. Genuinely: what a time to be alive!
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#13For the real frontier benchmark, at the edge of what humans will put up with, install Pyanodon's mod. The scale of it puts a real strain on your organizational skills. Overbuilding, underbuilding, or bad planning can all cause significant pain down the line as the factory risks becoming an unmanageable, tangled mess with no sane capacity for expansion. It's a real test of executive function and organization. For some…
and i thought gleba was hard
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#14> [LLMs] yet are unable to operate effectively in constrained environments, reflecting limitations in error analysis
This reflects my experience with gen-LLM coding, where LLMs keep trying to do the same thing in a loop.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#15> 1. Coding skill predicts performance
> Models with stronger coding abilities (Claude 3.5-Sonnet, GPT-4o) achieved higher Production Scores and completed more lab tasks. Claude outperformed others with a PS of 293,206 and 28 milestones, progressing beyond early-game resource extraction.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#16Everytime a paper like this comes out, I always have 1 question - How do they control the game using the LLMs? How does the control-feedback loop work? WHat tools, software and APIs they use to do it on Mac or Windows?
OP made the framework available https://github.com/JackHopkins/factorio-learning-environment
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#17its funny how video games are the hardest benchmark that humanity has for ai
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#18Everytime a paper like this comes out, I always have 1 question - How do they control the game using the LLMs? How does the control-feedback loop work? WHat tools, software and APIs they use to do it on Mac or Windows?
[deleted]
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#19its funny how video games are the hardest benchmark that humanity has for ai
A video game is a very well-defined problem, and usually comes with simple metrics for success – health, time, or in Factorio’s case, ultimately science per minute (or per minute played, for AIs?). Real world problems are much harder to define, they are embedded in a very complex ecosystem, and it’s not clear at all what to optimize for.
Re: Show HN: Factorio Learning Environment – Agents Build Factories
#20"put the right signals into my train network"
Not even humans can pass this benchmark.