Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What was the last task where only a frontier model could do it?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 36 posts

Re: Ask HN: What was the last task where only a frontier model could do it?

#21

Post by garethsprice »

This latest crop of frontier models (Fable, Sol) are very good at broad exploratory directions like "audit for bugs" or "find places in this codebase that I custom-implemented functionality that has since become available in the libraries I am using" in ways that prior generations would fail or not return as high quality results.

I did a recent head-to-head comparison between Qwen 3.6 35B running locally and Fable on the above and while Qwen did an admirable job finding a good handful of things, Fable returned many more, and more serious, issues. This included many more issues that crossed system boundaries in a multi-service repo.

In terms of one-shot "follow these detailed instructions" I think we are at a point where frontier models are overkill, especially at the price. For broad fan-out explorations, I think we are just getting started.

Re: Ask HN: What was the last task where only a frontier model could do it?

#23

Post by Gamemaster1379 »

talloaktrees wrote: post #3

I have been experimenting with dissassembly and rewrite of 90s dos games in Rust. It's not something I have experience in so I rely heavily on the models setting the direction and automating the work. It's something where the level of intelligence has a huge impact. I still haven't succeeded, it's mostly an experiment for me to test the edges of model capabilities and see how they tackle such a hard problem.

You have to reach set hard principles for disassembly. I've been experimenting with this for about 6 months now, starting with 4.6 in February where it would do a hacky half ass buggy job of an NES game.

I got a shitty PSX recompiler together in March. Been refining it since. This week along I stood up 5 different psx games and added widescreen to 4 of them and added an in memory injection English translation to another to avoid having to fight the assembly.

Re: Ask HN: What was the last task where only a frontier model could do it?

#24

Post by marak830 »

Gamemaster1379 wrote: post #23
talloaktrees wrote: post #3

I have been experimenting with dissassembly and rewrite of 90s dos games in Rust. It's not something I have experience in so I rely heavily on the models setting the direction and automating the work. It's something where the level of intelligence has a huge impact. I still haven't succeeded, it's mostly an experiment for me to test the edges of model capabilities and see how they tackle such a hard problem.

You have to reach set hard principles for disassembly. I've been experimenting with this for about 6 months now, starting with 4.6 in February where it would do a hacky half ass buggy job of an NES game. I got a shitty PSX recompiler together in March. Been refining it since. This week along I stood up 5 different psx games and added widescreen to 4 of them and added an in memory injection English translation to anot…

If you did a write up of that I would be very interested. I assume a lot of others would be as well.

Re: Ask HN: What was the last task where only a frontier model could do it?

#25

Post by talloaktrees »

bubblegumcrisis wrote: post #4
talloaktrees wrote: post #3

I have been experimenting with dissassembly and rewrite of 90s dos games in Rust. It's not something I have experience in so I rely heavily on the models setting the direction and automating the work. It's something where the level of intelligence has a huge impact. I still haven't succeeded, it's mostly an experiment for me to test the edges of model capabilities and see how they tackle such a hard problem.

Out of curiosity - is this a "couple hours" sort of thing, or a "tens of hours" or "hundreds of hours" or other?

hundreds.

likely my approach is often not optimal as I'm not experienced in this field

Re: Ask HN: What was the last task where only a frontier model could do it?

#26

Post by talloaktrees »

Gamemaster1379 wrote: post #23
talloaktrees wrote: post #3

I have been experimenting with dissassembly and rewrite of 90s dos games in Rust. It's not something I have experience in so I rely heavily on the models setting the direction and automating the work. It's something where the level of intelligence has a huge impact. I still haven't succeeded, it's mostly an experiment for me to test the edges of model capabilities and see how they tackle such a hard problem.

You have to reach set hard principles for disassembly. I've been experimenting with this for about 6 months now, starting with 4.6 in February where it would do a hacky half ass buggy job of an NES game. I got a shitty PSX recompiler together in March. Been refining it since. This week along I stood up 5 different psx games and added widescreen to 4 of them and added an in memory injection English translation to anot…

Would be curious to learn more from you if you have any other guidance or details.

Re: Ask HN: What was the last task where only a frontier model could do it?

#27

Post by yassineklear »

I don't agree with the people who say that frontier models are "overkill"; sure, they may be expensive (API usage especially), but the subscription plans are fair for the insane value they provide. I have a session where, if I had to pay for API usage, I would have paid close to $300.

= I had a lower‑tier model that I couldn't trust to make things happen for me; I somehow got the intuitive feel of what it could and couldn't do, so I use the "frontier" models for those tasks—things that include complex AI integration, weird responsiveness edge cases, or just when I find myself begging the lower‑tier models to just fix something again and again.

Re: Ask HN: What was the last task where only a frontier model could do it?

#28

Post by talloaktrees »

talloaktrees wrote: post #26
Gamemaster1379 wrote: post #23

Earlier quoted context omitted.

You have to reach set hard principles for disassembly. I've been experimenting with this for about 6 months now, starting with 4.6 in February where it would do a hacky half ass buggy job of an NES game. I got a shitty PSX recompiler together in March. Been refining it since. This week along I stood up 5 different psx games and added widescreen to 4 of them and added an in memory injection English translation to anot…

Would be curious to learn more from you if you have any other guidance or details.

I found the blog post: https://1379.tech/i-built-a-ps1-static-recompiler-with-no-pr...

Re: Ask HN: What was the last task where only a frontier model could do it?

#29

Post by bubblegumcrisis »

talloaktrees wrote: post #25
bubblegumcrisis wrote: post #4

Earlier quoted context omitted.

Out of curiosity - is this a "couple hours" sort of thing, or a "tens of hours" or "hundreds of hours" or other?

hundreds. likely my approach is often not optimal as I'm not experienced in this field

[dead]

Re: Ask HN: What was the last task where only a frontier model could do it?

#30

Post by Gamemaster1379 »

talloaktrees wrote: post #26
Gamemaster1379 wrote: post #23

Earlier quoted context omitted.

You have to reach set hard principles for disassembly. I've been experimenting with this for about 6 months now, starting with 4.6 in February where it would do a hacky half ass buggy job of an NES game. I got a shitty PSX recompiler together in March. Been refining it since. This week along I stood up 5 different psx games and added widescreen to 4 of them and added an in memory injection English translation to anot…

Would be curious to learn more from you if you have any other guidance or details.

I've been in heavy build mode for a while so my docs are a bit dated. But the repo is here: https://github.com/mstan/psxrecomp/ and at the bottom of it a Discord community I started for like minded individuals. Happy to chat there in the interim to get things rolling.
Post reply on HN