Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI has broken the open CTF format

kabir.au

Post reply on HN
Print view View original

131–140 of 502 posts

Re: Frontier AI has broken the open CTF format

#131

Post by koolala »

dguest wrote: post #88
himata4113 wrote: post #37

Earlier quoted context omitted.

bringing CTF solutions into the real world is a really good idea! I didn't even think of this until you mentioned it. we have very powerful simulation tools so something like "project a pattern at these angles" wouldn't really work as you could simulate that. I guess something cool is that we can make simulating the solution very expensive, but in real world it would be free since it's analog... As long as simulation…

Maybe CTF is dead, but there are plenty of fun problems in the real world -- ask any scientist, engineer, or medical researcher. There are a million places where a computer can interact with a non-digital system in a loop. - Tune an FPGA, or a whole data-center, or just a physical computer. - Make a drone fly somewhere. - Design a selective toxin (or anti-toxin). Or, you know, get more people to click on adds. All to…

Using real-life calculators to add? Calculate the Flag. I don't think it is dead at all. It's like mixing in board game / escape room / science / engineeer/ medical research elements.

Re: Frontier AI has broken the open CTF format

#132

Post by StrauXX »

mort96 wrote: post #26
amingilani wrote: post #22

I don’t think CTFs are dead, they’ll just evolve. The difficulty level will need to be increased or the rules locked down. Just like sports and racing persist despite the existence of performance enhancing drugs and rocket technology. I just did a CTF where I was in the top 10. It was the first CTF I completed and I used AI because the rules permitted it. That said, I couldn’t solve all challenges. But yes, it was si…

Increasing the difficulty level is a terrible solution. The problem with CTFs isn't that they're too easy. Making them harder just makes them even less accessible to people who don't cheat. It'd be like seeing people who put hidden electric motors in their bikes during Tour de France and conclude, "oh we just need longer distances and steeper hills".

LLMs don't tend to help much when solving challenges beyond their skill level. Either they one-shot a challenge, or thei are almost useless as a companion for them.

Re: Frontier AI has broken the open CTF format

#133

Post by bernds74 »

dguest wrote: post #76
entropyneur wrote: post #67

Earlier quoted context omitted.

I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.

I'll always remember my middle school science teaching telling us that nuclear fusion violates conservation of mass because the 2 protons in a pair of hydrogen nuclei combine to make helium with 4 nucleons. It's not true, but that's not the point. But he was a great teacher anyway. He was engaging and kept the kids in line and learning. I eventually learned the truth, and most of my classmates forgot about it. Teachi…

I had a chemistry teacher who told us that hydrogen reacts violently with oxygen, and this is how the hydrogen bomb works.

Re: Frontier AI has broken the open CTF format

#134

Post by Retr0id »

StrauXX wrote: post #129
Retr0id wrote: post #93

Earlier quoted context omitted.

Unconstrained internet is nice, but I don't think it's a hard requirement. Just tricky to enforce, even in-person.

It is a hard requirement. Once you reach higher levels of challenges you spend most of your time reading through RFCs, web sepcs, Github issues, mailing lists, papers, random bugtrackers and library/framework code. There is no way to create a whitelist for that. Besides, a firewall won't stop good hackers.

Normal CTF workflows can involve a lot of research but that's not the point. You can design self-contained challenges with offline solving in mind, and bundle any truly necessary docs/src/etc. with the challenge download.

Re: Frontier AI has broken the open CTF format

#136

Post by oofbey »

CoastalCoder wrote: post #127
victorbjorklund wrote: post #63

Earlier quoted context omitted.

Like humans.

I think we should go a little deeper on this idea. We can all agree that both human "experts" and LLMs can sometimes be right, and sometimes be confidently wrong. But that doesn't imply that they're equally fit for purpose. It just means that we can't use that simple shortcut to conclude that one is inferior to the other. So where do we go from here?

I’ve always thought of the definition of “expert” as reliably knowing the difference between what is known, what is speculated but unproven, and what is unknown. People claim expertise in all sorts of things that they aren’t experts in. But true experts should not be wrong. They should qualify levels of certainty. This definition certainly works in the sciences.

Re: Frontier AI has broken the open CTF format

#137

Post by 3form »

zem wrote: post #116
3form wrote: post #103

Earlier quoted context omitted.

This one feels less sinister than some other things at least to me, personally. You can reasonably doubt that the conservation of mass is violated and find out the truth based on that. But understanding more complex biology or historical context for some things? Granted, many of these things seem to be low stakes, but I'm sure there are some there are not (sex ed comes to mind).

to be fair, fusion does violate conservation of mass, just not the way the teacher explained it. the loss of mass is where the energy comes from.

Yes, together with mass-energy equivalency it would form a coherent argument, and then also a correct one - but the thing is that if incomplete, it still might sound funky enough to you to research it if you care.

I think it helps that it's a very narrow field to look at, compared to fuzzy and big-picture view of social studies, for example. So much room to be confidently wrong... And sadly I can't think of a solution, LLMs or not.

Re: Frontier AI has broken the open CTF format

#138

Post by daymanstep »

bernds74 wrote: post #133
dguest wrote: post #76

Earlier quoted context omitted.

I'll always remember my middle school science teaching telling us that nuclear fusion violates conservation of mass because the 2 protons in a pair of hydrogen nuclei combine to make helium with 4 nucleons. It's not true, but that's not the point. But he was a great teacher anyway. He was engaging and kept the kids in line and learning. I eventually learned the truth, and most of my classmates forgot about it. Teachi…

I had a chemistry teacher who told us that hydrogen reacts violently with oxygen, and this is how the hydrogen bomb works.

I had a chemistry teacher who insisted that the fissile isotope of Uranium was U-238 not U-235. I challenged him on this multiple times and he refused to budge on this. I get that it's a simple mistake to make (it seems like U-238 is bigger so intuitively ought to be less stable) but he could have just looked it up and he didn't, I guess he was just so confident about it that he thought there was no way he could have been wrong about it.

Re: Frontier AI has broken the open CTF format

#139

Post by oldsecondhand »

entropyneur wrote: post #67
daymanstep wrote: post #50

Earlier quoted context omitted.

Wonderful teachers that give unreliable information with total confidence?

I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.

That's an American problem though. In most of Europe you need a masters degree to teach highschool and that involves at least an undergrad level of understanding the subjects you will teach.

E.g. in Hungary I had a university CS professor that originally wanted to be a highschool teacher and a highschool physics teacher that originally wanted to be researcher. Their choice of degree didn't determine which outcome they got. The researcher and teacher curriculum had an 80%+ overlap.

Re: Frontier AI has broken the open CTF format

#140

Post by aaron695 »

chrismorgan wrote: post #53

Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…

[dead]
Post reply on HN