Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Potential issues in curl found using AI assisted tools

mastodon.social

Post reply on HN
Print view View original

41–50 of 203 posts

Re: Potential issues in curl found using AI assisted tools

#41

Post by airstrike »

mhitza wrote: post #39
flohofwoe wrote: post #4

This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…

In an application I'm working on, I use gpt-oss-20B. In a prompt I dump in the OWASP Top 10 web vulnerabilities, and a note that it should only comment on "definitive vulnerabilities". Has been pretty effective in finding vulnerabilities in the code I write (and it's one of the poorest-rated models if you look at some comments). Where I still need to extend this, is to introduce function calling in the flow, when "it…

Tool calling is the best lever for getting value out of LLMs

Re: Potential issues in curl found using AI assisted tools

#42

Post by nxobject »

tptacek wrote: post #16
1970-01-01 wrote: post #11

Notice it was 'a set of tools' They're using it correctly. It's a system of tools, not an autopilot.

It's weird that the discussion has collapsed down to "autopilots" vs. "abstention". I'm thrilled to be converging on an understanding that it instead "people who understand what they're trying to do" vs. "vibe coders".

In defense of the cynics, I get the impression in a situation where (a) there's a lot of company marketing hype in such a competitive market that begs cynicism, and (b) we're constantly learning the boundary of trained LLMs can actually do (and can't), as well as unusual emergent workflows, that really do make a difference.

Re: Potential issues in curl found using AI assisted tools

#43

Post by CharlesW »

flohofwoe wrote: post #4

This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…

> When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in the end just gives me a list of my own FIXME comments (lol), which tbh is quite underwhelming - a simple bash script could do that too.

Here's a technique that often works well for me: When you get unexpectedly poor results, ask the LLM what it thinks an effective prompt would look like, e.g. "How would you prompt Claude Code to create a plan to effectively review code for logic bugs, ignoring things like FIXME and TODO comments?"

The resulting prompt is too long to quote, but you can see the raw result here: https://gist.github.com/CharlesWiltgen/ef21b97fd4ffc2f08560f...

From there, you can make any needed improvements, turn it into an agent, etc.

Re: Potential issues in curl found using AI assisted tools

#44

Post by diggan »

SAI_Peregrinus wrote: post #40
Sharlin wrote: post #32

Earlier quoted context omitted.

This is a point I see discussed surprisingly little. Given that many (most?) programmers like designing and writing code (excluding boilerplate), and not particularly enjoy reviewing code, it certainly feels backwards to make the AI write the code and relegate the programmer to reviewing it. (I know, of course, that the whole thing is being sold to stakeholders as "LoC machine goes brrrr" – code review? what's that?)

Creativity is fun. AIs automate that away. I want an AI that can do my laundry, fold it, and put it away. I don't need an AI to write code for me. I don't mind AI code review, it sometimes has a valid suggestion, and it's easy enough to ignore most of the rest of the time.

> Creativity is fun. AIs automate that away.

I've been developing with LLMs on my side for months/about a year now, and feels like it's allowing me to be more creative, not less. But I'm not doing any "vibe-coding", maybe that's why?

The creative parts (for me) is coming up with the actual design of the software, and how it all fits together, what it should do and how, and I get to do that more than ever now.

Re: Potential issues in curl found using AI assisted tools

#45

Post by diggan »

mhitza wrote: post #39
flohofwoe wrote: post #4

This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…

In an application I'm working on, I use gpt-oss-20B. In a prompt I dump in the OWASP Top 10 web vulnerabilities, and a note that it should only comment on "definitive vulnerabilities". Has been pretty effective in finding vulnerabilities in the code I write (and it's one of the poorest-rated models if you look at some comments). Where I still need to extend this, is to introduce function calling in the flow, when "it…

> (and it's one of the poorest-rated models if you look at some comments).

Yeah, don't listen to "wisdom of the crowd" when it comes to LLM models, there seems to be a ton of fud going on, especially on subreddits.

GPT-OSS was piled on for being dumb in the first week of release, yet none of the software properly supported it at launch. As soon as it was working properly in llama.cpp, it was clear how strong the model was, but at that point the popular sentiments seems to have spread and solidified.

Re: Potential issues in curl found using AI assisted tools

#46

Post by Legion »

athorax wrote: post #20
dwedge wrote: post #12

Earlier quoted context omitted.

Yeah this is really fair play to Daniel Stenberg that he still approached these AI generated bug reports with an open mind after all the problems he's had.

I think the big difference is that these aren't AI generated bug reports. They are bugs found with the assistance of AI tools that were then properly vetted and reported in a responsible way by a real person.

Basically using AI the way we have used linters and other static analysis tools, rather than thinking it's magic and blindly accepting its output.

Re: Potential issues in curl found using AI assisted tools

#48

Post by bgwalter »

alganet wrote: post #35
simonw wrote: post #30

Earlier quoted context omitted.

Looks like you're reacting to the Hacker News title here, which is currently " Daniel Stenberg on 22 curl bugs found by AI and fixed" That's an editorialized headline (so it may get fixed by dang and co) - if you click through to what Daniel Stenberg said he was more clear: > Joshua Rogers sent us a massive list of potential issues in #curl that he found using his set of AI assisted tools. AI-assisted tools seems rig…

If the title changes, it is still a valid critique of the tools, how they might work, and a possible way of getting them for free. Also, think about it: of course I read Joshua's report. Otherwise, how could I have known the names of the products he used?

I don't think many people here are interested in how something works. They want to see the headline "Curl developer finally convinced by AI!" and otherwise drop anecdotes about Claude Code etc.

All comments that want to know more are at the bottom.

Re: Potential issues in curl found using AI assisted tools

#49

Post by lowbloodsugar »

When I read “we consider nread == 0 as reading a byte and we shouldn’t” I immediately think of all the things that look like bugs but are there because some critical piece of infrastructure relies on that behavior. AI isn’t going to know about that unless you tell it, and the problem is that there’s plenty of folks who have job security precisely because they don’t write that down.

Re: Potential issues in curl found using AI assisted tools

#50

Post by dingnuts »

diggan wrote: post #44
SAI_Peregrinus wrote: post #40

Earlier quoted context omitted.

Creativity is fun. AIs automate that away. I want an AI that can do my laundry, fold it, and put it away. I don't need an AI to write code for me. I don't mind AI code review, it sometimes has a valid suggestion, and it's easy enough to ignore most of the rest of the time.

> Creativity is fun. AIs automate that away. I've been developing with LLMs on my side for months/about a year now, and feels like it's allowing me to be more creative, not less. But I'm not doing any "vibe-coding", maybe that's why? The creative parts (for me) is coming up with the actual design of the software, and how it all fits together, what it should do and how, and I get to do that more than ever now.

I'm still faster than the cheap bots.

The creative part for me includes both the implementation and the design, because the implementation also matters. The bots get in the way.

Maybe I would be faster if I paid for Claude Code. It's too expensive to evaluate.

If you like your expensive AI autocomplete, fine. But I have not seen any demonstrable and maintainable productivity gains from it, and I find understanding my whole implementation faster, more fun, and that it produces better software.

Maybe that will change, but people told me three years ago that we would be at the point today where I could not outdo the bot;

with all due respect, I am John Henry and I am still swinging my hammer. The steam pile driving machine is still too unpredictable!

Post reply on HN