121–130 of 502 posts
This left a strange feeling. The article reads as extremely bleak. But from a different perspective this is extremely bullish for AI.
The intent for most CTFs is to provide a meaningful challenge that concerns a single topic without introducing noise that wastes time. Of course a training exercise is easier to complete for an LLM.
Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
We usually hire for problem solving capabilities and not so much for technical know-how.
That’s at least how I read your comment.
Earlier quoted context omitted.
Bizarre font and poor contrast, yep. The text itself being exceedingly long for no obvious reason doesn’t help.
Poor contrast? White on black? And if you think it was too long, what part would you have shortened? I never knew about the scene and found it interesting to read this personal take on it.
According to Pikka, the paragraph text is Taupe Grey (#92908a) on a Liquorice (#111110) background. That's... pretty far from black and white.
Earlier quoted context omitted.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.
Earlier quoted context omitted.
Wonderful teachers that give unreliable information with total confidence?
Like humans.
We can all agree that both human "experts" and LLMs can sometimes be right, and sometimes be confidently wrong.
But that doesn't imply that they're equally fit for purpose. It just means that we can't use that simple shortcut to conclude that one is inferior to the other.
So where do we go from here?
Earlier quoted context omitted.
Ctfs need preparation and unconstrained internet, even if you block domains it’s possible to tunnel out
Unconstrained internet is nice, but I don't think it's a hard requirement. Just tricky to enforce, even in-person.
Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…