>The competition is turning into "who can afford to run enough agents, with enough context, for long enough."
This will basically become true for everything.
Frontier AI has broken the open CTF format
331–340 of 502 posts
Re: Frontier AI has broken the open CTF format
#332Must I beg to have an acronym spelled out a least once, the first time it's used? Even if you assume 90% of readers already know, the other 10% (including me, in this case) will thank you, it doesn't take much effort, and it expands the reach of your communication or idea. Exceptions for cases where the acronym is just so well known that a lot of people don't even know what it stands for even though they know the con…
Let’s reduce this to absurdity:
I think you only wanted clarification of CTF (Capture the Flag) and not AI (Artificial Intelligence) and not GPT-4 (Generative Pre-Trained Transformer version 4) and not CLI (Command Line Interface) and not MCP (Model Context Protocol) and not LLM (Large Language Model)
Quoting TFA (The Fucking Article): “just adapt bro”
lol at the BGP example
Re: Frontier AI has broken the open CTF format
#333It's not only CTFs. I strongly believe being a programmer at a gamejam like Ludum Dare, or hackathons is pretty much over.
Ludum Dare 59 just wrapped up last week, and both first and second place were won by developers using "Agentic" coding tools, something the community there is still discussing:
https://ldjam.com/events/ludum-dare/59/setidream/about-ai-ar...
For what it's worth, the non-AI-coded entries were still quite good relative to the winners, so it's not so obvious that AI use confers an unbeatable advantage.
Re: Frontier AI has broken the open CTF format
#334Earlier quoted context omitted.
Wonderful teachers that give unreliable information with total confidence?
I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.
I think they are less reliable. For factually verifiable facts LLMs are doing worse than 90% for me. I've been told some incorrect things by educators, but at a much lower rate.
Re: Frontier AI has broken the open CTF format
#335Must I beg to have an acronym spelled out a least once, the first time it's used? Even if you assume 90% of readers already know, the other 10% (including me, in this case) will thank you, it doesn't take much effort, and it expands the reach of your communication or idea. Exceptions for cases where the acronym is just so well known that a lot of people don't even know what it stands for even though they know the con…
i try not to over feed tangents but this is precisely how i feel every time i speak to someone who is recently enlisted in the military. i have to constantly stop them and be like “i have no idea what you just said” over and over and over again. it’s like trying to make sense of a random bowl of alphabet soup.
Re: Frontier AI has broken the open CTF format
#336This is happening to other forms of competitive programming too. The most recent AIs have problem solving skills rivaling top humans, and so if AI can't be easily banned, the competition is dominated by AI agents. I thought code golf would take longer for AIs because there's so little training data (it's more niche), but we're seeing AIs starting to match expert humans there too. Sucks because golf has been my favori…
Code golf is well-suited for AI because you have a easily verified objective (minimize code size while passing tests) and can run an LLM in a loop to churn away at it.
Re: Frontier AI has broken the open CTF format
#337Must I beg to have an acronym spelled out a least once, the first time it's used? Even if you assume 90% of readers already know, the other 10% (including me, in this case) will thank you, it doesn't take much effort, and it expands the reach of your communication or idea. Exceptions for cases where the acronym is just so well known that a lot of people don't even know what it stands for even though they know the con…
We live in the goddammed future. Huamnity's knowledge is at your fingertips. Right clicking the Nth word of the article and putting in any semblance of effort to learn on your own is too much to ask?
I don't know everything, there's tons of stuff I don't know about, but when I'm at my web browser, the least I can do about something is ask Google about a word or phrase or subject that isn't familiar instead of being spoonfed information like I'm a baby.
Re: Frontier AI has broken the open CTF format
#338Earlier quoted context omitted.
Which acronym do you mean? CTF? I think that acronym, just like BGP, is more well known by itself than what it stands for. More generally, not every piece of writing is meant for every audience. Like if someone writes a blog post about CTFs aimed at people who like CTFs, nobody in the target audience needs to have CTF explained to them. Ultimately HN is a link aggregator, but sometimes its a bit like eavesdropping on…
Best practice in writing about technical concepts is to spell out acronyms like this on their first use. There is a ton of stuff I learn about here on HN that I didn't know anything about before. It doesn't help that the linked article never bothers to explain this either.
When I encounter new terms, I look them up. Just like any other new word. Been doing it since I was a kid with a dictionary. Now, it’s too easy not to. There is literally no excuse.
Re: Frontier AI has broken the open CTF format
#339>If adaptation means accepting that the scoreboard is now an AI orchestration benchmark, then we should say that honestly instead of pretending the old competition still exists. This is like someone complaining that making machine parts has been ruined: Skillful craftsmen used to make them by hand using manual tools! Nowadays the CAD/CAM/CNC cheaters have almost completely automated the whole thing. How is the next g…
[deleted]
Re: Frontier AI has broken the open CTF format
#340>If adaptation means accepting that the scoreboard is now an AI orchestration benchmark, then we should say that honestly instead of pretending the old competition still exists. This is like someone complaining that making machine parts has been ruined: Skillful craftsmen used to make them by hand using manual tools! Nowadays the CAD/CAM/CNC cheaters have almost completely automated the whole thing. How is the next g…
Just parachuting in to reflexively throw the "Luddite" label at someone lamenting the decline of a niche community they've enjoyed participating in and contributing to is certainly ... a choice.
Within the framework of your analogy, it's like responding to someone active in DIY maker groups suddenly dealing with an influx of influencers in meetups showing off Chinese junk from Etsy to post on Tiktok, and accusing them of being a Luddite blinded by their zealous hatred of mass production -- both strangely abrasive and also fairly nonsensical except as a "mass production supporter" social signifier.
Not to mention, in the article they specifically describe themselves as a heavy user of frontier models for security research ever since the release of Opus 4.5, calling them "useful within the field". In fact I don't see any actual criticism of AI/LLMs anywhere whether for security research, programming or anything else, except for making competitive CTFs no longer viable.
What does it take to avoid the "Luddite" brand? Using AI themselves and praising AI as useful (to the point of having a lopsided advantage over humans) isn't enough? Do they also need to say "I haven't written a line of code in 6 months/it's easily a 100x multiplier for my job" every time they mention it too?