I can't help but draw parallels with video games. Aimbots in competitive multiplayer games is a well defined issue: it's considered cheating and frowned upon, players caught cheating are banned from the game. Tool-assisted speedruns (TAS) where a player attempts a world record at completion in a single-player game is another face of the same concept (computers help you win), but one that is socially accepted as long as runs are clearly labelled as TAS.
Frontier AI has broken the open CTF format
81–90 of 502 posts
Re: Frontier AI has broken the open CTF format
#82Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
Wonderful teachers that give unreliable information with total confidence?
Off the top of my head: DOMS being little crystals in muscles, tongue having separate areas for each type of taste, food pyramid, blue blood in the veins, the appendix being useless, body temperature doesn't change disregarding whether it's exposed to cold or to heat, and a whole lot of stuff related to politics and history I'd rather just omit (I don't live in the US).
All things I learned in school which were wrong information.
Not to mention, the current state of education is far worse. I don't think most realize how low the bar is.
Re: Frontier AI has broken the open CTF format
#83Re: Frontier AI has broken the open CTF format
#84Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
I found this interview [0] on the subject of AI in CS education on the Oxide & Friends podcast very illuminating.
Of course, Brown University CS != All education, but interesting angle nevertheless.
[0] Episode webpage: https://share.transistor.fm/s/31855e83
Re: Frontier AI has broken the open CTF format
#85Earlier quoted context omitted.
If it helps I understand the second much better and feels less clickbaity and includes more info. I do agree with the points you made about the confusion although I find frontier a term used in this area a lot, “frontier AI models have” would probably resolve that.
Imo frontier is too niche and specific, if you know what a frontier model means then it's fine, but if you don't then it's negative/detrimental to the title. "new" does the same thing and is probably just a better descriptor then frontier
But then you're not acting as a billboard promoting AI. Isn't that partly the point?
Re: Frontier AI has broken the open CTF format
#86Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
Re: Frontier AI has broken the open CTF format
#87Earlier quoted context omitted.
>LLMs can be wonderful teachers Are they or aren't they
They can be incredible. One on one teaching with an infinitely patient teacher who can generate interactive problems on the fly, for dollars a month? Wild. A year of paid ChatGPT would pay for about 9 hours of cheap tutoring here.
That's not going to work out the way you think it will when a student won't even know how to ask questions.
Re: Frontier AI has broken the open CTF format
#88Earlier quoted context omitted.
A portion could require astral projection and computers can't do that. Or maybe just a VR mini-game like the 90s always imagined.
bringing CTF solutions into the real world is a really good idea! I didn't even think of this until you mentioned it. we have very powerful simulation tools so something like "project a pattern at these angles" wouldn't really work as you could simulate that. I guess something cool is that we can make simulating the solution very expensive, but in real world it would be free since it's analog... As long as simulation…
Maybe CTF is dead, but there are plenty of fun problems in the real world -- ask any scientist, engineer, or medical researcher.
There are a million places where a computer can interact with a non-digital system in a loop.
- Tune an FPGA, or a whole data-center, or just a physical computer.
- Make a drone fly somewhere.
- Design a selective toxin (or anti-toxin).
Or, you know, get more people to click on adds. All totally possible to automate.
Re: Frontier AI has broken the open CTF format
#89Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
Wonderful teachers that give unreliable information with total confidence?
They'll also encourage and praise you even when you're heading down the wrong path until you think you've uncovered the secret of the universe or proven that established science was wrong this whole time when really you've just been bullshitting with an engagement bot.
Re: Frontier AI has broken the open CTF format
#90Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
> I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
If you remove the "without AI" and the end, I've been hearing similar anecdotes about fizzbuzz for years (isn't the whole point of fizzbuzz to filter out those candidates?)