CTF = Capture The Flag
https://en.wikipedia.org/wiki/Capture_the_flag_(cybersecurit...
Frontier AI has broken the open CTF format
221–230 of 502 posts
Re: Frontier AI has broken the open CTF format
#222Earlier quoted context omitted.
I think we're rediscovering management from first principles. The main selling point of AI is that it writes code faster than you could. Checking it line by line undoes most of that benefit. In the same vein, there's no real benefit to leading a team if you plan on supervising every task. But here's the thing: for humans, this is manageable because we've come up with a number of mechanisms to select for dependable wo…
Your conclusion is pretty silly. My expertise has led me to the obvious fact that I would never give an LLM write access to my production database in the first place. So in your own example my expertise actually does solve that problem without the need for something like a consequence whatever that means to you. We already have full control over the input and tools they are given and full control over how the output…
Until it decides it needs additional access to complete its task and focuses on escaping your sandbox to do so
Re: Frontier AI has broken the open CTF format
#223https://en.wikipedia.org/wiki/Capture_the_flag_(cybersecurit... still has no mention of AI, but that will likely change as they increasingly dominate competition.
Using AI on CTF is like using a car to get better at the 100 yard dash
Re: Frontier AI has broken the open CTF format
#224Earlier quoted context omitted.
Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.
Why is it important that a dev can’t do fizzbuzz without ai? If they can ship code that matches a spec, why does it matter if they’re using ai or not? Genuinely curious.
If you can’t do fizzbuzz without AI you have no business being in this career.
Re: Frontier AI has broken the open CTF format
#225Earlier quoted context omitted.
>LLMs can be wonderful teachers Are they or aren't they
As usual it depends. When it does well it's because it can do well. When it does poorly it's because you're prompting it wrong.
>When it does well it's because it can do well.
Can't argue with that logic
Re: Frontier AI has broken the open CTF format
#226Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
I developed for 15 years. I don’t think I can do with AI anymore. Why would I even want to do that? It’s like telling a car driver to build an engine.
Re: Frontier AI has broken the open CTF format
#227Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
I first did fizz buzz about 10 years ago fresh out of college. Now, after 10 years in full stack and fully vibe coding, I forgot basic python syntax. An interview like yours would have false positives if you are checking for syntax because well, its like looking up spelling, I just ask the AI for the syntax inline.
Re: Frontier AI has broken the open CTF format
#228Re: Frontier AI has broken the open CTF format
#229Chess and Go are not dead just because Ai got better than humans at these games. What am I missing here?
Chess banned engines from competition. CTFs can't really do that because you need internet access and tooling to play.
Re: Frontier AI has broken the open CTF format
#230Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
Don't worry, i never thought I would see someone unable to write fizzbuzz, but it happened 9 years ago.
Also how many people work with linux and can't tell you what 'ls -alh' is doing is staggering (lets ignore the h, even al people struggle hard).
People working with docker for YEARS and don't even understand how docker actually works (cgroups)...
Interviewing was always a bag of emotions in sense of "holy shit my job is save your years to come" and "srsly? how? How do you still have a job?"