Viewing profile — dangelosaurus

Please email me and I will help fix this.

Yes, I think this is an under-appreciated part of the release. I hope people can adapt them to their own workflows. We run A LOT of evals as the Promptfoo team and we've spent bill…

Hello!

Thank you, that means a lot. Being able to keep building practical, open-source security tooling was important to us. Really glad we got to ship this, and there's still a lot we wa…

Not yet, unfortunately. We only just opened the repo, and there isn't a public issue specifically tracking local or OpenAI-compatible endpoint support. The issue tracker is here: h…

By default, you can sign in with your ChatGPT/Codex account or use an OPENAI_API_KEY. It also does not require cyber registration but it can help if you encounter refusals. If you …

Agreed! This is near the top of our priority list and we will make it a lot better soon.

In short, this isn't an offline scanner. The CLI runs locally but the code and context needed for analysis are sent to the hosted model (OpenAI). For API, Business, and Enterprise …

Sorry about that. We hit an authentication issue at launch and have now merged and deployed a fix in 0.1.1: https://github.com/openai/codex-security/pull/22 One thing worth checkin…

Thanks! You've run into a real limitation: the CLI doesn't bypass the model's cybersecurity guardrails. If GPT-5.6 Sol finds a vulnerability but refuses to explain it, switching fr…

Fair question, and I agree the refusals are frustrating. The CLI doesn't do a repository-ownership check. Public projects are supported, and reviewing your own Linux kernel patches…

Yeah, you're right. A per-minute rate limit shouldn't kill a scan after a minute, and "partial output was kept" makes it sound like you can pick up where you left off. You can't ye…

Oof, that's a bad outcome. Half your weekly usage and a 50-minute scan just to get a HEAD error at the end is not acceptable. --max-cost can help limit estimated spend, but that do…

We are actively working on officially supporting this. Because it's open source it is pretty easy to point a coding agent at it now and switch out the model.

The plugin, including when invoked through the Codex CLI, is great for scanning the repo you're currently working in. The standalone Security CLI/SDK uses the same scanner, but is …

Hey HN, Michael here, co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI. Thanks for checking this out and for flagging the auth issues. We j…

Hey HN - Michael here, co-founder of Promptfoo. Happy to answer questions. The one I'd ask if I were reading this: what happens to Promptfoo open source? We're going to keep mainta…

https://mldangelo.com and https://github.com/mldangelo/personal-site I have been slowly evolving it over 10 years. 1.6k stars, ~ 1,000 forks. I originally designed it to be easy to…

I work on Promptfoo (an open-source eval framework). Appreciate the mention here. This post captures a lot of the hard lessons around agent evals. In particular, task ambiguity and…

Working on promptfoo, an open-source (MIT) CLI and framework for eval-ing and red-teaming LLM apps. Think of it like pytest but for prompts - you define test cases, run evals again…

I ran a red team eval on GPT-5.2 within 30 minutes of release: Baseline safety (direct harmful requests): 96% refusal rate With jailbreaking : 22% refusal rate 4,229 probes across …

I felt obligated to submit a fix: https://github.com/a16z-infra/reading-list/pull/9 Used Claude to fact-check and fix errors that were likely introduced by Cursor. The circle is co…

I did similar measurements back in July ( https://www.promptfoo.dev/blog/grok-4-political-bias/ , dataset: https://huggingface.co/datasets/promptfoo/political-question... ). Anthro…

Promptfoo | Senior/Staff Engineers, Security Researchers, GTM & Founding Operators | REMOTE (North America) / Hybrid San Mateo CA | Full-time | https://promptfoo.dev Promptfoo is t…