Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI has broken the open CTF format

kabir.au

Post reply on HN
Print view View original

181–190 of 502 posts

Re: Frontier AI has broken the open CTF format

#181

Post by KomoD »

jsoaoxhd wrote: post #130
chrismorgan wrote: post #53

Meta: this was submitted with the article’s title “The CTF scene is dead” which I found very easy to understand. It has just been updated to use the subtitle’s first sentence, “Frontier AI has broken the open CTF format”. I find that much harder to grasp, rather like a garden-path sentence. My immediate thoughts were that “Frontier” was a company name, and that there was some file format named CTF. If you don’t know…

Why do people always hijack threads to discuss titles? Most articles have terrible titles. Just downvote it and move on.

You can't downvote a submission.

Re: Frontier AI has broken the open CTF format

#182

Post by tardedmeme »

mr_mitm wrote: post #161
zem wrote: post #116

Earlier quoted context omitted.

to be fair, fusion does violate conservation of mass, just not the way the teacher explained it. the loss of mass is where the energy comes from.

Yes, there is no law of conservation for mass like there is for energy. Fusion is a good example for why it's not conserved. The teacher was right.

There actually is a law of conservation of mass (it's the same law, because mass is energy) and it only appears violated if you forget about the particles that are zooming away at the speed of light. Of course the mass of a system changes if mass can flow in and out.

Re: Frontier AI has broken the open CTF format

#184

Post by ekidd »

josh2600 wrote: post #165
IanCal wrote: post #105

Earlier quoted context omitted.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Why is it important that a dev can’t do fizzbuzz without ai? If they can ship code that matches a spec, why does it matter if they’re using ai or not? Genuinely curious.

> If they can ship code that matches a spec, why does it matter if they’re using ai or not?

I am perfectly capable of writing specs, and feeding them to 3 separate copies of Claude Code all by myself. Then I task switch between the tmux windows based on voice messages from the pack of Claudes. This workflow is fine for some things, and deeply awful for others.

Basically, if a developer is just going to take my spec and hand it to Claude Code, then they're providing zero value. I could do that myself, and frequently do.

The actual bottleneck is people who can notice, "The god object is crumbling under the weight of managing 6 separate concerns with insufficient abstraction." Or "Claude has created 5 duplicate frameworks for deploying the app on Docker. We need to simplify this down to 1 or we're in hell." I will happy fight to hire people who can do the latter work. But those people can all solve fizzbuzz in their sleep.

People who just "ship code that matches a spec" without understanding the technical details are providing close to zero value right now.

There is an interesting niche for people with deep knowledge of customer workflows who can prompt Claude Code. These people can't build finished products using Claude. But they can iterate rapidly on designs until they find a hit. Which we can then fix using people with deeper engineering knowledge and taste.

But if you're not bringing either deep customer knowledge or actual engineering knowledge, you're not adding much these days.

Re: Frontier AI has broken the open CTF format

#185

Post by tardedmeme »

mold_aid wrote: post #61
baq wrote: post #44

Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.

>LLMs can be wonderful teachers Are they or aren't they

As usual it depends. When it does well it's because it can do well. When it does poorly it's because you're prompting it wrong.

Re: Frontier AI has broken the open CTF format

#187

Post by crab_galaxy »

oldsecondhand wrote: post #139
entropyneur wrote: post #67

Earlier quoted context omitted.

I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.

That's an American problem though. In most of Europe you need a masters degree to teach highschool and that involves at least an undergrad level of understanding the subjects you will teach. E.g. in Hungary I had a university CS professor that originally wanted to be a highschool teacher and a highschool physics teacher that originally wanted to be researcher. Their choice of degree didn't determine which outcome the…

I think it’s pretty common for states to require a masters degree to maintain your teachers certification.

You also have to pass a standardized test specifically on subject matter in order to get your teaching certificate.

The undergrad degree I did was split into thirds, one for subject matter, one for teaching pedagogy, and one for teaching your subject matter.

Re: Frontier AI has broken the open CTF format

#188

Post by mr_mitm »

tardedmeme wrote: post #182
mr_mitm wrote: post #161

Earlier quoted context omitted.

Yes, there is no law of conservation for mass like there is for energy. Fusion is a good example for why it's not conserved. The teacher was right.

There actually is a law of conservation of mass (it's the same law, because mass is energy) and it only appears violated if you forget about the particles that are zooming away at the speed of light. Of course the mass of a system changes if mass can flow in and out.

Mass is not the same as energy. Mass can be converted to energy or has energy, but a photon, for example, is massless while carrying energy.

Re: Frontier AI has broken the open CTF format

#189

Post by amazingamazing »

baq wrote: post #44

Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence. We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.

> We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part.

No we have not.

Re: Frontier AI has broken the open CTF format

#190

Post by Octoth0rpe »

josh2600 wrote: post #165
IanCal wrote: post #105

Earlier quoted context omitted.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Why is it important that a dev can’t do fizzbuzz without ai? If they can ship code that matches a spec, why does it matter if they’re using ai or not? Genuinely curious.

> If they can ship code that matches a spec, why does it matter if they’re using ai or not?

The inability to write fizzbuzz strongly implies their inability to understand what they've shipped. Review is some significant portion of the job. Understanding of the product is also part of the job.

Specs are also in a sense, scaled down, fuzzy, natural language descriptions of a feature. The fuzziness is the source of a bugs, or at least a mismatch between the actual desired feature and what was written down at spec writing time. As such, just matching a spec is just the bare minimum that a good dev should be doing. They should be understanding what the spec is _not_ saying, understanding holes in their implementation, how their implementation enables or hinders the next feature and the next, next feature, etc. I don't think any of that is possible without understanding what was actually implemented.

Post reply on HN