Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Potential issues in curl found using AI assisted tools

mastodon.social

Post reply on HN
Print view View original

81–90 of 203 posts

Re: Potential issues in curl found using AI assisted tools

#81

Post by refulgentis »

bgwalter wrote: post #65
refulgentis wrote: post #54

Earlier quoted context omitted.

[flagged]

[flagged]

I assumed based on your post and the post you replied to that it is literally impossible to prove any AI is involved, and I trust both of you on that.

Given that, I'm afraid all the interlocution I have to offer is the thing you commented on, the mind of a downvoter, i.e. positing that every downvoter must have details, including details we[1] can't find.

Past that, I'm afraid to admit I am having difficulty understanding how the slides are related, and I don't even know what Matasano is -- is that who owns fly.io? I thought they were "indie" -- I'm embarrassed to admit I thought Monsanto at first. I do know how much I've used AI to code, so I can vouch for tptacek's post.

[1] royal we, i.e. I trust you and OP so completely on what it findable vs. not findable that I trust we can't establish with 100% certainty any sort of AI-based thingy was used at all. To be clear, too, 100% is always too high of a bar, I mean to say we can even't establish at 90% confidence. Even 1% confidence. If all we have is their word to go on, it's impossible.

Re: Potential issues in curl found using AI assisted tools

#84

Post by tptacek »

refulgentis wrote: post #81
bgwalter wrote: post #65

Earlier quoted context omitted.

[flagged]

I assumed based on your post and the post you replied to that it is literally impossible to prove any AI is involved, and I trust both of you on that. Given that, I'm afraid all the interlocution I have to offer is the thing you commented on, the mind of a downvoter, i.e. positing that every downvoter must have details, including details we[1] can't find. Past that, I'm afraid to admit I am having difficulty understa…

Matasano was a software security company I cofounded in 2005 and sold to NCC Group in 2012. Super weird pull for this thread.

Re: Potential issues in curl found using AI assisted tools

#85

Post by bgwalter »

tptacek wrote: post #83
bgwalter wrote: post #65

Earlier quoted context omitted.

[flagged]

What does "even at Matasano" mean? Matasano hasn't existed for over 12 years.

My mistake. I confused Mataroa with Matasano:

https://ludic.mataroa.blog/blog/contra-ptaceks-terrible-arti...

Re: Potential issues in curl found using AI assisted tools

#86

Post by tptacek »

alganet wrote: post #58
refulgentis wrote: post #54

Earlier quoted context omitted.

[flagged]

Do you believe AI is at the core of these security analyzers? If so, why the personal story blogpost? You can just explain me in technical terms why is that so. Claiming to work for Google does not work as an authority card for me, you still have to deliver a solid argument. Look, AI is great for many things, but to me these products sounds like chocolate that is actually just 1% real chocolate. Delicious, but 99% no…

I had a conversation in a chat room yesterday about AI-assisted math tutoring where a skeptic said that the ability of GPT5 to effortlessly solve quotient differentials or partial fraction decomposition or rational inequalities wasn't indicative of LLM improvements, but rather just represented the LLMs driving CAS tools and thus didn't count.

As a math student, I can't possibly care less about that distinction; either way, I paste in a worked problem solution and ask for a critique, and either way I get a valid output like "no dummy multiply cos into the tan before differentiating rather than using the product rule". Prior to LLMs, there was no tool that had that UX.

In the same way: LLMs are probably mostly not off the top of their "heads" (giant stacks of weight matrices) axiomatically deriving vulnerabilities, but rather just doing a very thorough job of applying existing program analysis tools, assembling and parallel-evaluating large numbers of hypothesis, and then filtering them out. My interlocutor in the math discussion would say that's just tool calls, and doesn't count. But if you're a vulnerability researcher, it doesn't matter: that's a DX that didn't exist last year.

As anyone who has ever been staffed on a project triaging SAST tool outputs before would attest: it extremely didn't exist.

Re: Potential issues in curl found using AI assisted tools

#87

Post by Sharlin »

CuriouslyC wrote: post #72
SAI_Peregrinus wrote: post #40

Earlier quoted context omitted.

Creativity is fun. AIs automate that away. I want an AI that can do my laundry, fold it, and put it away. I don't need an AI to write code for me. I don't mind AI code review, it sometimes has a valid suggestion, and it's easy enough to ignore most of the rest of the time.

Is AI automating creativity away if you come up with an idea and have it actually implement it?

Yes, because ideas are not worth much if anything. If you have an idea of a book, or a painting, and have someone else implement it, you have not done creative work. Literally, you have not created the work, brought it to existence. The creator has done the creativity.

Re: Potential issues in curl found using AI assisted tools

#88

Post by tptacek »

bgwalter wrote: post #85
tptacek wrote: post #83

Earlier quoted context omitted.

What does "even at Matasano" mean? Matasano hasn't existed for over 12 years.

My mistake. I confused Mataroa with Matasano: https://ludic.mataroa.blog/blog/contra-ptaceks-terrible-arti...

I think you owe me a better apology than that. I disagree with your evaluation of the response to that post, strongly, but more importantly I didn't bring it up in the first place, and the claim you made about it (intentionally or not; awfully weird to land at my company's name) was personal and scurrilous.

Your call! I'm moving on.

Re: Potential issues in curl found using AI assisted tools

#89

Post by ako »

OGWhales wrote: post #79
CharlesW wrote: post #43

Earlier quoted context omitted.

> When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in the end just gives me a list of my own FIXME comments (lol), which tbh is quite underwhelming - a simple bash script could do that too. Here's a technique that often works well for me: When you get unexpectedly poor results, ask the LLM what it thinks an effec…

I've found this a really useful strategy in many situations when working with LLMS. It seems odd that it works, since one one think its ability to give a good reply to such a question means it already "understands" your intent in the first place, but that's just projecting human ability onto LLMS. I would guess this technique is similar to how reasoning modes seems to improve output quality, though I may misunderstan…

Works for humans the same? Even if you know how to do a complex project, it helps to first document the approach, and then follow it.

Re: Potential issues in curl found using AI assisted tools

#90

Post by dylan604 »

Sharlin wrote: post #32
flohofwoe wrote: post #4

This is exactly what I'd want from an 'AI coding companion'. Don't write or fix the code for me (thanks but I can manage that on my own with much less hassle), but instead tell me which places in the code look suspicious and where I need to have a closer look. When I ask Claude to find bugs in my 20kloc C library it more or less just splits the file(s) into smaller chunks and greps for specific code patterns and in t…

This is a point I see discussed surprisingly little. Given that many (most?) programmers like designing and writing code (excluding boilerplate), and not particularly enjoy reviewing code, it certainly feels backwards to make the AI write the code and relegate the programmer to reviewing it. (I know, of course, that the whole thing is being sold to stakeholders as "LoC machine goes brrrr" – code review? what's that?)

To me, it's the natural result of gaining popularity that enough people have started to use after the hype train rolled through and are now giving honest feedback. Real honest feedback can feel like a slap in the face when all you have had is overwhelming positive feedback from those aboard the hype train.

The writing has been on the wall with so called hallucinations where LLMs just make stuff up that the hype was way out over its skiis. The examples of lawyers being fined for unchecked LLM outputs being presented as fact type of stories will continue to take the shine off and hopefully some of the raw gungho nature will slow down a bit.

Post reply on HN