Live data from Hacker News

Frontier AI has broken the open CTF format

kabir.au

151–160 of 502 posts

Re: Frontier AI has broken the open CTF format

#151
post #135

Chess and Go are not dead just because Ai got better than humans at these games. What am I missing here?

Read the article.

I read the article. Their chess section makes no sense as in "why this wouldn't work for CTF".

But I don't know enough that's why I asked.

I imagine one could do CTF in public, machines you work on vetted/prepared to some spec, yada yada.

If chess and Go can do it why can't CTF?

That was my question when I wrote "what am I missing here".

Re: Frontier AI has broken the open CTF format

#152

Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…

Why do people always hijack threads to discuss titles? Most articles have terrible titles. Just downvote it and move on.

Why do you contribute to making this thread longer? Just downvote an move on.

Re: Frontier AI has broken the open CTF format

#153
post #68

Earlier quoted context omitted.

>LLMs can be wonderful teachers Are they or aren't they

A million times better than any human teacher I’ve ever had, for sure. Now I’m certain that there exist those mythical human instructors who can do better, but that’s not worth much if 99.99% of people don’t have access to them. Just like a good human physician who takes their time with the patient is better than an LLM, but that’s not worth much either given that this doesn’t match most people’s experience with thei…

>A million times better than any human teacher I’ve ever had, for sure.

Not really, not if you want to ask it deep questions. It won't have an answer that is deeper than something that you can find online, and if pressed it will just keep circling around the same response.

The reason is that this "thing" was never curious, never asked questions, and never really learned anything. It just has learned the Internet "by heart", and is as boring as a human teacher who is not really curious about the subject they are teaching, and has just got some degree by "by hearting" some text book. Of course it does it much better than a human, but it is fundamentally the same thing.

Re: Frontier AI has broken the open CTF format

#154
I guess this is very similar to what happened to demo scene, in some way. The limits are what makes these problems interesting, and once we have better machines / tools, the incredible skill is no longer prerequisite, making everything less interesting for participants. Sad, but - such is life...

Re: Frontier AI has broken the open CTF format

#155
post #81

I can't help but draw parallels with video games. Aimbots in competitive multiplayer games is a well defined issue: it's considered cheating and frowned upon, players caught cheating are banned from the game. Tool-assisted speedruns (TAS) where a player attempts a world record at completion in a single-player game is another face of the same concept (computers help you win), but one that is socially accepted as long…

Sure if the goal is entertainment and sports, you're right. However, unlike chess or counter strike it's downstream from a real needed utility. Like, is there a point to do it anymore? (ofc there is, but still, it's been devalued from the perspective of the 'real utility')

It’s literally not. The most interesting and satisfying CTFs have never been grounded in reality, it’s just been an expression of mastery, both from players and authors, with a few notable exceptions. But they’re that, exceptions, not the rule.

Re: Frontier AI has broken the open CTF format

#156
There's something funny about complaining about cheating in a hacking competition.

Well actually I get it. In cycling motor doping, putting a hidden engine into the bike, seems more offensive than regular doping. I think this is because there is a continuum from eating well to taking supplements to injecting stuff, but having a engine breaks a fundamental idea about cycling. Similar hacking is about cleverly abusing the rules.

Re: Frontier AI has broken the open CTF format

#157

Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…

[deleted]

Re: Frontier AI has broken the open CTF format

#158
post #68

Earlier quoted context omitted.

>LLMs can be wonderful teachers Are they or aren't they

A million times better than any human teacher I’ve ever had, for sure. Now I’m certain that there exist those mythical human instructors who can do better, but that’s not worth much if 99.99% of people don’t have access to them. Just like a good human physician who takes their time with the patient is better than an LLM, but that’s not worth much either given that this doesn’t match most people’s experience with thei…

>Now I’m certain that there exist those mythical human instructors who can do better,

You're certain that mythical instructors exist (?) who "can" do better?

Are human instructors more competent as teachers than AI teachers, or are AI teachers more competent as teachers than human teachers? No "this or that can happen," just a definitive statement please.

AI is likely a million times better student than my dimwit cybersec meatbags...er, majors, for sure, as well! Don't have a reliable way to measure or experience why/how, tho, so I'm not out here claiming it. Even if I did, why would I argue for their replacement?

Re: Frontier AI has broken the open CTF format

#159
post #123

Earlier quoted context omitted.

We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.

I wonder if you’re filtering for the right things. We usually hire for problem solving capabilities and not so much for technical know-how. That’s at least how I read your comment.

Isn't wiring coding solving a problem? If the candidate can't do that then even if they use AI for coding how are they going to review the code properly?

Re: Frontier AI has broken the open CTF format

#160
post #105

Earlier quoted context omitted.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Yeah, I've interviewed people like this 15 years ago. Degrees and experience mean nothing in this field. The best predictor I found was personal passion projects. Let them get as nerdy as possible, then you will see pretty quickly where their skills are at and what their limits are. And you will immediately filter out people who just studied CS because they heard you can make good money.

Completely agree with this, leetcode has become such a business now of memorization for interviews it’s useless to know if someone memorized a solution or not.
Post reply on HN