Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI has broken the open CTF format

kabir.au

Post reply on HN
Print view View original

91–100 of 502 posts

Re: Frontier AI has broken the open CTF format

#91

Post by jack_pp »

jofzar wrote: post #71
IanCal wrote: post #65

Earlier quoted context omitted.

If it helps I understand the second much better and feels less clickbaity and includes more info. I do agree with the points you made about the confusion although I find frontier a term used in this area a lot, “frontier AI models have” would probably resolve that.

Imo frontier is too niche and specific, if you know what a frontier model means then it's fine, but if you don't then it's negative/detrimental to the title. "new" does the same thing and is probably just a better descriptor then frontier

if you are on HN and have no idea what "frontier model" would mean maybe it's time you found out.

Re: Frontier AI has broken the open CTF format

#93

Post by Retr0id »

hsbauauvhabzb wrote: post #28
kevinsimper wrote: post #18

You could make it offline and with provided laptops only, just like with the competitive CS2 scene.

Ctfs need preparation and unconstrained internet, even if you block domains it’s possible to tunnel out

Unconstrained internet is nice, but I don't think it's a hard requirement. Just tricky to enforce, even in-person.

Re: Frontier AI has broken the open CTF format

#94

Post by copx »

>If adaptation means accepting that the scoreboard is now an AI orchestration benchmark, then we should say that honestly instead of pretending the old competition still exists.

This is like someone complaining that making machine parts has been ruined: Skillful craftsmen used to make them by hand using manual tools!

Nowadays the CAD/CAM/CNC cheaters have almost completely automated the whole thing. How is the next generation of craftsmen going to learn how to craft a gear by hand when the process of gear making has been reduced to pressing start on a CNC machine?!

See what I mean? Sorry, I think this article is just Luddite. I can empathize with the pain of your beloved craft basically being rendered obsolete by new technology, but the process can neither be stopped nor is it bad in general.

The manual skills you trained with CTF puzzles are now simply no longer relevant . (Field-specific) "AI orchestration" is the new cyber securtiy skill if LLMs really have become so good at this, and what the author used to do manually then has the same value as being able to craft a gear by hand.

Re: Frontier AI has broken the open CTF format

#95

Post by black_knight »

hoyd wrote: post #49

«That feedback loop is breaking. If the visible scoreboard is dominated by teams using AI, a beginner is pushed toward using AI before they have built the instincts the AI is replacing. That is an anti-pattern. It prevents active learning, and active struggle is the bit that actually teaches you. It is also completely demotivating to put in real effort and see no visible progress because the ladder above you has been…

I see this with beginner programming students at university. They get AI to help them with assignments, with the intention of learning, but ultimately they do not get the understanding they would have if they had done the assignment themselves. Then they are at a deficit for learning more advanced topics.

My fear is that they never get to the level they need to be at to create good software even with the help of AI. So, although an expert with AI can create great software, that is not where we end up. In stead we will have vibe coded messes by people who barely have any grasp of what is going on.

Re: Frontier AI has broken the open CTF format

#96

Post by Gigachad »

Retr0id wrote: post #90
Gigachad wrote: post #86

Earlier quoted context omitted.

We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.

> I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI. If you remove the "without AI" and the end, I've been hearing similar anecdotes about fizzbuzz for years (isn't the whole point of fizzbuzz to filter out those candidates?)

While this is true, it seems undeniable that if you use AI to do everything for you, you will never learn the skills. I'm seeing a massive amount of developers submitting stuff for review and admitting they have no idea how it works and they just generated it.

Re: Frontier AI has broken the open CTF format

#98

Post by Jenk »

IanCal wrote: post #65
chrismorgan wrote: post #53

Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…

If it helps I understand the second much better and feels less clickbaity and includes more info. I do agree with the points you made about the confusion although I find frontier a term used in this area a lot, “frontier AI models have” would probably resolve that.

If the title simply said "AI is out-performing humans at CTF" then none of this confusion exists. Nothing is "broken," we don't need to be superfluous with "frontier," and the point is still there.

Re: Frontier AI has broken the open CTF format

#100

Post by ViscountPenguin »

parasti wrote: post #81

I can't help but draw parallels with video games. Aimbots in competitive multiplayer games is a well defined issue: it's considered cheating and frowned upon, players caught cheating are banned from the game. Tool-assisted speedruns (TAS) where a player attempts a world record at completion in a single-player game is another face of the same concept (computers help you win), but one that is socially accepted as long…

The biggest difference would be the fact that you can discover video game cheating through some kind of trace. Speed running communities go pretty hardcore on that kind of thing nowadays.

It's a lot harder to detect cheating when your only trace is how fast someone submitted the string CTF{DUck1e_Pwned}

Post reply on HN