Potential issues in curl found using AI assisted tools
1–10 of 203 posts
Re: Potential issues in curl found using AI assisted tools
#2[deleted]
Re: Potential issues in curl found using AI assisted tools
#3Here are 55 closed PRs in the curl repo which credit "sarif data" - I think those are the ones Daniel is talking about here https://github.com/curl/curl/pulls?q=is%3Apr+sarif+is%3Aclos...
This is notable given Daniel Stenberg's reports of being bombarded by total slop AI-generated false security issues in the past: https://www.linkedin.com/posts/danielstenberg_hackerone-curl...
Concerning HackerOne: "We now ban every reporter INSTANTLY who submits reports we deem AI slop. A threshold has been reached. We are effectively being DDoSed. If we could, we would charge them for this waste of our time"
Also this from January 2024: https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-f...
Re: Potential issues in curl found using AI assisted tools
#4This is exactly what I'd want from an 'AI coding companion'.
Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look.
When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in the end just gives me a list of my own FIXME comments (lol), which tbh is quite underwhelming - a simple bash script could do that too.
ChatGPT is even less useful since it basically just spend a lot of time to tell me 'everything looking great yay good job high-five!'.
So far, traditional static code analysis has been much more helpful in finding actual bugs, but static analysis being clean doesn't mean there are no logic bugs, and this is exactly where LLMs should be able to shine.
If getting more useful potential-bugs-information from LLMs requires an extensively customized setup then the whole idea is getting much less useful - it's a similar situation to how static code analysis isn't used if it requires extensive setup or manual build-system integration instead of just being a button or menu item in the IDE or enabled by default for each build.
Re: Potential issues in curl found using AI assisted tools
#5Here are 55 closed PRs in the curl repo which credit "sarif data" - I think those are the ones Daniel is talking about here https://github.com/curl/curl/pulls?q=is%3Apr+sarif+is%3Aclos... This is notable given Daniel Stenberg's reports of being bombarded by total slop AI-generated false security issues in the past: https://www.linkedin.com/posts/danielstenberg_hackerone-curl... Concerning HackerOne: "We now ban every…
The models used have improved quite well since then, I guess his change of opinion shows that.
Re: Potential issues in curl found using AI assisted tools
#6This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…
Cursor BugBot is pretty good for this, we did the free trial and it was so popular with our devs that we ended up keeping it. Occasional false positives aside, it's very useful. It saves time for both the PR submitter and the reviewer.
Re: Potential issues in curl found using AI assisted tools
#7This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…
Suggestion: run a regex to remove those FIXME comments first, then try the experiment again.
I often use Claude/GPT-5/etc to analyze existing repositories while deliberately omitting the tests and documentation folders because I don't want them to influence the answers I'm getting about the code - because if I'm asking a question it's likely the documentation has failed to answer it already!
Re: Potential issues in curl found using AI assisted tools
#8Here are 55 closed PRs in the curl repo which credit "sarif data" - I think those are the ones Daniel is talking about here https://github.com/curl/curl/pulls?q=is%3Apr+sarif+is%3Aclos... This is notable given Daniel Stenberg's reports of being bombarded by total slop AI-generated false security issues in the past: https://www.linkedin.com/posts/danielstenberg_hackerone-curl... Concerning HackerOne: "We now ban every…
The models used have improved quite well since then, I guess his change of opinion shows that.
I think it's more about how people are using it. An amateur who spams him with GPT-5-Codex produced bug reports is still a waste of his time. Here a professional ran the tools and then applied their own judgement before sending the results to the curl maintainers.
Re: Potential issues in curl found using AI assisted tools
#9I really didn't expect a story about curl and AI to be positive for once.
Some history: https://hn.algolia.com/?q=curl+AI
Re: Potential issues in curl found using AI assisted tools
#10Here are 55 closed PRs in the curl repo which credit "sarif data" - I think those are the ones Daniel is talking about here https://github.com/curl/curl/pulls?q=is%3Apr+sarif+is%3Aclos... This is notable given Daniel Stenberg's reports of being bombarded by total slop AI-generated false security issues in the past: https://www.linkedin.com/posts/danielstenberg_hackerone-curl... Concerning HackerOne: "We now ban every…
The models used have improved quite well since then, I guess his change of opinion shows that.
It's probably also the difference of idiots hoping to cash out/get credit for vulnerabilities by just throwing ChatGPT at the wall compared to this where it seems a somewhat seasoned researcher is trialing more customized tools.