Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Capturing the Flag with GPT-4

micahflee.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Capturing the Flag with GPT-4

#11

Post by rfoo »

zitterbewegung wrote: post #10

I tried using gpt4 on the qualifier rounds for hackasat and I think you need to know the basics of what is a good answer (or maybe satellite red teaming is too obscure) but I never got a flag out of it .

Hehe, you can solve the two "Street" challenges with GPT-4, they are bog standard entry-level competitive programming challenge disguised as a reverse engineering challenge.

Re: Capturing the Flag with GPT-4

#13

Post by politelemon »

cjbprime wrote: post #6

Neat! My mind-blown moment with GPT-4 was realizing that it will often be able to tell you the output of the (unique, not available in training data) scripts it writes for you.

That is very interesting, which I'd like to try once GPT4 is widely available. I did try GPT3 with a CTF a few weeks ago, and it gave seemingly plausible code but outright incorrect answers.

Re: Capturing the Flag with GPT-4

#14

Post by erfgh »

cjbprime wrote: post #6

Neat! My mind-blown moment with GPT-4 was realizing that it will often be able to tell you the output of the (unique, not available in training data) scripts it writes for you.

Hum... maybe it runs the scripts and sees the output?

Re: Capturing the Flag with GPT-4

#15

Post by amrb »

I appreciate they say they need to learn more about 'exec' when asking GPT4, it also plays well into some of the reading strategies I've seen to get a high-level understanding then read the documentation with more general idea going in, could also lower frustration for new engineers so they push through to the end!

Re: Capturing the Flag with GPT-4

#16

Post by SargeZT »

erfgh wrote: post #14
cjbprime wrote: post #6

Neat! My mind-blown moment with GPT-4 was realizing that it will often be able to tell you the output of the (unique, not available in training data) scripts it writes for you.

Hum... maybe it runs the scripts and sees the output?

It does not. Regular ChatGPT without plugins does not have access to any tools. Throw it a script with some weird outputs and it'll definitely fail, every time. While the script 'evaluation' stuff can be pretty decently impressive, it is not actually executing anything.

Re: Capturing the Flag with GPT-4

#17

Post by gjadi »

I get that it's a way to find out what GPT4 is capable of, but IMHO that defeats the point of a game like CTF.

It's like playing with an aimbot. You may be beating the other players, but where's the fun?

Re: Capturing the Flag with GPT-4

#18

Post by blowski »

gjadi wrote: post #17

I get that it's a way to find out what GPT4 is capable of, but IMHO that defeats the point of a game like CTF. It's like playing with an aimbot. You may be beating the other players, but where's the fun?

I often use cheats and walkthroughs, to the chagrin of my gamer friends. I have fun though, so who cares?
Post reply on HN