Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier AI has broken the open CTF format

kabir.au

Post reply on HN
Print view View original

141–150 of 502 posts

Re: Frontier AI has broken the open CTF format

#141

Post by Trung0246 »

himata4113 wrote: post #21

I was writing an obfuscator recently, I just had the model deobfuscate and optimize the code back to original and I kept improving the obfuscator until it couldn't. The funny thing is that after all this I also ended up with a really strong deobfuscator and optimizer which is probably more capable than most commercial tools. The solution is just to make CTFs harder, but when do CTFs become too hard? Maybe the problem…

Interesting, what I just did recently is basically the same of this as I tried to push the limit of js obfuscator as much as possible by keep forcing gpt/claude deobfuscate final output then having gpt improve the tool to break the deobfuscator.

Do you publish it somewhere? Here's a sample my my js obfuscator output: https://gist.github.com/Trung0246/c8f30f1b3bb6a9f57b0d9be94d...

Re: Frontier AI has broken the open CTF format

#142

Post by artninja1988 »

lugu wrote: post #135
virtualritz wrote: post #60

Chess and Go are not dead just because Ai got better than humans at these games. What am I missing here?

Read the article.

https://news.ycombinator.com/newsguidelines.html

"Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that"."

Re: Frontier AI has broken the open CTF format

#143

Post by Gigachad »

baxtr wrote: post #123
Gigachad wrote: post #86

Earlier quoted context omitted.

We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.

I wonder if you’re filtering for the right things. We usually hire for problem solving capabilities and not so much for technical know-how. That’s at least how I read your comment.

Ultimately in a software development role you need both technical know how and problem solving capabilities.

This situation in particular was a React role so there is an expectation that when you list React as one of your skills on your resume then you know at least the basics of state, the common hooks, the difference between a reference to a value vs the value itself.

These days you can do a surprising amount with AI without knowing what you are doing, but if you don't have any clue how things work you'll very quickly run in to problems you can't prompt away.

Re: Frontier AI has broken the open CTF format

#144

Post by Gigachad »

IanCal wrote: post #105
Gigachad wrote: post #86

Earlier quoted context omitted.

We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Something that is for sure new is the AI interview cheating tools which listen in on the call and provide answers in an overlay invisible to screen sharing. The only way to deal with it would either be invasive spyware on the applicants computer or asking them to do the interview face to face.

Re: Frontier AI has broken the open CTF format

#145

Post by [deleted] »

vladms wrote: post #73
p-e-w wrote: post #68

Earlier quoted context omitted.

A million times better than any human teacher I’ve ever had, for sure. Now I’m certain that there exist those mythical human instructors who can do better, but that’s not worth much if 99.99% of people don’t have access to them. Just like a good human physician who takes their time with the patient is better than an LLM, but that’s not worth much either given that this doesn’t match most people’s experience with thei…

Did an LLM teach you a topic you did not feel like learning? For me the best human teachers were the ones that managed to make me interested on topics that I thought are boring/useless (many times my opinion being stupid, mostly due to lack of experience). So far with LLM I learn about things I know something (at least that they exist) and I am interested in, which is a small subset of things that one should learn du…

[deleted]

Re: Frontier AI has broken the open CTF format

#146

Post by nsvd2 »

Gigachad wrote: post #144
IanCal wrote: post #105

Earlier quoted context omitted.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Something that is for sure new is the AI interview cheating tools which listen in on the call and provide answers in an overlay invisible to screen sharing. The only way to deal with it would either be invasive spyware on the applicants computer or asking them to do the interview face to face.

Spyware wouldn't help at all because you could just put the AI between the computer and the monitor, for example, or use a VM.

Re: Frontier AI has broken the open CTF format

#148

Post by gedy »

sigmoid10 wrote: post #126
IanCal wrote: post #105

Earlier quoted context omitted.

Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.

Yeah, I've interviewed people like this 15 years ago. Degrees and experience mean nothing in this field. The best predictor I found was personal passion projects. Let them get as nerdy as possible, then you will see pretty quickly where their skills are at and what their limits are. And you will immediately filter out people who just studied CS because they heard you can make good money.

I agree, however there are so many interviewers who will still treat that as some softball criteria and insist that unless you "prepare" for an interview by memorizing leetcode you are 100% a faker and liar.

Re: Frontier AI has broken the open CTF format

#149

Post by yc-kraln »

dostick wrote: post #147

Unable to find what “CTF” means, since it doesnt look like referring to Capture The Flag gaming

It does--but a particular form of Capture The Flag where there is a computer system and the "capturing" is breaking in or exploiting a security issue in that system.

Re: Frontier AI has broken the open CTF format

#150

Post by akdev1l »

Levitz wrote: post #82
daymanstep wrote: post #50

Earlier quoted context omitted.

Wonderful teachers that give unreliable information with total confidence?

Off the top of my head: DOMS being little crystals in muscles, tongue having separate areas for each type of taste, food pyramid, blue blood in the veins, the appendix being useless, body temperature doesn't change disregarding whether it's exposed to cold or to heat, and a whole lot of stuff related to politics and history I'd rather just omit (I don't live in the US). All things I learned in school which were wrong…

My biology teacher in school once tried to teach us that winds created by God. Not like spiritually or something but that God literally made the wind I guess.

My “earth sciences” teacher also once tried to argue with me against the universal law of gravitation. (no, she was not referring to Special/General Relativity. She didn’t agree two objects in a vacuum fall at the same speed regardless of mass.

Post reply on HN