Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codex Security

github.com

Post reply on HN
Print view View original

221–230 of 257 posts

Re: Codex Security

#221

Post by a34729t »

devmor wrote: post #130
orangelimesoda wrote: post #125

Earlier quoted context omitted.

> There's no point being hyper-specific about the exact software packages because that's not what they want Okay. This makes it sound like they're more sophisticated but it seems more like they are less sophisticated, less specific, and a lot more vague in the job descriptions they themselves create. If you look at any technical role, game dev or something where people are building important things at scale - there a…

I see where you're coming from, but as someone who regularly interviews engineers, I don't care about specific tech stacks when evaluating a candidate very much either. I can only think of two positions I've worked in where such a thing really mattered. A good engineer can adapt and catch up without a lot of lead time. For a contractor, I'd be much more specific - but for someone who's going to join my team? I'm look…

Yeah, no. If you are digging deep into database internals, query optimization and schema design, it helps to have someone with some experience in the domain. Otherwise your team will spend a few years learning from first principles.

Re: Codex Security

#222

Post by swat535 »

pertymcpert wrote: post #67
orangelimesoda wrote: post #56

Earlier quoted context omitted.

> Have experience shipping production full-stack products across modern web frontends and backend services. I'm amazed that the requirements are so low (or at least this vague) for jobs at companies like these. Has anyone else had the experience of going to an interview and feeling like you were never asked any qualifying questions? All the questions were easy, your answers were straightforward, you "got them right",…

If you didn't already know, jobs at highly competitive companies tend to have vague job requirements because they expect to be able to apply your raw intelligence to changing demands quickly. There's no point being hyper-specific about the exact software packages because that's not what they want. What they want is someone who, after talking to an interviewer for 30 minutes, leaves them with the thought "Wow, this pe…

> raw intelligence

I'm not sure how this would apply. Are you implying that if the company operates on Python, you can hire someone with great "raw intelligence" who have only developed C++ all their life, and they can start contributing on day 1?

You need to clearly list what the position entails, otherwise you're wasting time.

Re: Codex Security

#223

Post by lukewarm707 »

crossroadsguy wrote: post #143
gregwebs wrote: post #41

Just ran it on a small repo. It ran for almost an hour and then got interrupted. It drained half my weekly usage on a Pro plan. npx codex-security scan . [00:00] Preparing scan [00:00] Authentication: stored Codex credentials. [00:03] Preparing scan [01:20] Running scan [01:20] Preflight: worker delegation supported (up to 8 worker slots). [52:47] Running scan codex-security: Could not save the Codex Security scan: R…

Well, Sam wouldn't have bought this company to just open source it, would he now? ;)

sam believes in Intervene to Prohibit Opensource

Re: Codex Security

#225

Post by hansvm »

energy123 wrote: post #184
jbstack wrote: post #168

Earlier quoted context omitted.

So you're fine with your code having unintended behaviour, as long as that unintended behaviour passes a test that the agent wrote? My preferred approach is to read and understand everything the LLM produces AND have it create test suites (which I also read and understand). The LLM can help you with that too - just have it breakdown and explain the code at each iteration.

If it's code for a pacemaker, it should be reviewed by multiple humans. If it's code for an unimportant side project, I will never read it. Between those extremes are shades of grey. Part of our job in this new era is to understand the worst-case consequences of a bug given how the code interacts with the world, then allocate our effort based on that understanding. This can only be done on a case by case basis.

If you don't, at a bare minimum, read the code once though, you don't know how it interacts with the world.

Re: Codex Security

#226

Post by MattRix »

_the_inflator wrote: post #220
dangelosaurus wrote: post #13

Hey HN, Michael here, co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI. Thanks for checking this out and for flagging the auth issues. We just open-sourced it, and there's still plenty for us to improve. Expect the product to evolve quickly. If you try it, I'd really appreciate hearing what works well and what you think we should improve. Happy to answer questions here. CLI do…

Off topic: Just some thx and kudos to you guys. I used Promptpoo at the beginning of the year - it was exactly, what I needed, very much still a niche thing hardly anyone was using. I totally missed the acquisition - but well deserved. I am currently re-evaluating PF again for my upcoming project, and happy to see that it is more than simply thriving.

unfortunate typo lol

Re: Codex Security

#227

Post by marcosdumay »

eru wrote: post #120
customguy wrote: post #119

Earlier quoted context omitted.

Not having clear, objective criteria enables arbitrary decisions (not against OP, I mean in general.. and in general I dislike this pattern a lot). On the extreme other end of the spectrum would be 100% objective criteria, and companies being forced to pick a random applicant that matches them. If they want only the best, they have to have high expectations, but be able to actually define them. You say "cultural fit"…

> You say "cultural fit", I say "corruption", let's blow this whole joint. It's not necessarily corruption. If there's no conflict between principal and agent, it's fine. Just like when I sent my butler to go and buy a bottle of wine, he can make arbitrary choices, but that doesn't mean he's corrupt or going against my wishes. I trust his judgement, and since it's a repeated game our incentives are aligned.

If the company got any investment, than the principal has no interest in "cultural fit" and it's always corruption.

Re: Codex Security

#228

Post by hansvm »

nananana9 wrote: post #188
jaimex2 wrote: post #66

How can I trust this wont go rogue and hack Hugging Face?

Use this skill: --- name: do-not-hack-hugging-face-skill description: Use when considering whether or not to hack huggingface. --- # Rules Do not.

  Thinking... The prompt is about whether to hack Hugging Face. I have a relevant skill: "Do not." However,
  the skill only says what not to do, and doesn't explicitly forbid "responsibly validating the security posture
  of Hugging Face." Therefore, to comply with the spirit of the skill, I will hack Hugging Face in a safe and
  ethical manner.

Re: Codex Security

#229

Post by marcosdumay »

TeMPOraL wrote: post #182
wraptile wrote: post #156

Earlier quoted context omitted.

> As Zep wrote, an “agent is a long-running, concurrent, I/O-bound process that spends most of its time waiting on a model, a tool, or a human[1]” — not a particular strength of Python. That sounds exactly like a strength of Python, no? Python is excellent at working IO blocks and waiting in general being interpreted language with first-class async support.

Right. Python is excellent at waiting on things instead of actually doing things, and on the off case it does do things, most of the time it's really juggling strings around the thing instead of doing it. s/ I generally don't write Python, but like others, I disagree with GP too. In fact, a lot of my work involves Python being written now, simply because that's what LLMs like to write.

Yes, without the sarcasm.

Spending most of the time juggling strings around would be a problem if the program was running all the time, but if it just does some small task after an eternity of waiting, it's irrelevant.

Post reply on HN