181–190 of 258 posts
For those of you who don't want to click into linked in, https://hackerone.com/reports/3125832 is the latest example of a invalid curl report
Recent toots on account has the news as well
Earlier quoted context omitted.
It is somebody's own skill. Photographers are not painters. People who do modular synths aren't guitarists. Technical DJing is quite different from tapping on a Spotify app on a smartphone. Just because you've exclusively exposed yourself to crude implementations doesn't mean sophisticated ones don't exist.
But you just missed the point. People aren't trying to push photographs into painted works displays People who do modular synths aren't typically trying to sell their music as country/rock/guitar based music. A 3D modeler of a statue isn't pretending to be a sculpturist. People pushing AI art are trying to slide it right into "human art" displays. Because they are talentless otherwise.
The portraiture artist industry was dramatically disrupted by the daguerreotype.
The automobile dried up the income of farrier and blacksmith along with ending the horsemanship industry.
The rise of synthesizers in the 80s greatly reduced the number of studio musicians.
And it's undeniable that the industry of commercial artists is currently being disrupted by AI.
But the decline of portraiture artist due to daguerreotypes doesn't mean, say Ansel Adams is dogshit.
We can acknowledge both the industrial ramifications and the labor and skill of the new forms without being dismissive of either. Auto repair is still a skill. Driving a car is still work even if there's no horses.
When mechanical looms replaced manual weavers during the luddite movement, it might have killed countless careers but it didn't kill fashion. Our clothing isn't simulacrum echos of the 1820s.
This is the transfer of a skill into a property. The transfer of a skill into a property changes it from something that must be rented from below to something that can be owned from above.
Property isn't a thing however, it's a chosen relationship between people about a thing. we could make different choices...
What in curl makes AI-based analysis completely ineffective?
The more positive take, and I think the biggest reason is that curl is just well made. But along the way, it most likely uses plenty of code analysis tools: static analysis, testing, coverage, fuzzing,... the classic. And I am sure these tools catch bugs before they are published. Is there an overlap between one of these tools and AI, can one substitute for the other?
Another possibility is that curl is "weird" enough to throw off AI-based code analysis. We won't change curl for that reason, but it may be good to know.
And yeah, it may just be that AI just sucks but only looking at one side of the equation is not very productive I think.
The article mentions spam and AI slop, it is a problem for sure, but the claim here is much stronger than "stop spamming me", it is "AI never worked". And I find it a bit surprising, because when I introduce an new category of tool on some code base I work with, AI or not, I almost always find at least a problem or two.
Reading the straw that broke the camel's back commit illustrates the problem really well: https://hackerone.com/reports/3125832 . This shit must be infuriating to dig through. I wonder if reputation systems might work here - you could give anyone who id's with an AML/KYC provider some reputation, enough for two or three reports, let people earn reputation digging through zero rep submissions and give someone like 10,…
What if the human marks it as spam but you're actually legit? Deposit another 2€ to have the platform (like Hackerone or whichever you're reporting via) give a second opinion, you'll get the 4€ back if you weren't spamming. What to do with the proceeds from spammers? The first X euros of spam reports go to upkeep of the platform, the rest to a good cause defined by the projects to whom the reports were submitted because they were the ones who had to deal with reading the slop so they get at least this much out of it
Raise deposit cost so long as slop volume remains unmanageable
This doesn't discriminate against people who aren't already established, but it may be a problem if you live in a low-income country and can't easily afford 20€ (assuming it ever gets to that deposit level). Perhaps it wouldn't work, but it can first be trialed at a normal cost level. Another concern is anonymity and payment. We hackers are often a paranoid lot. One can always support cash in the mail though, the sender can choose whether their privacy is worth a postage stamp
I handle reports for a one million dollar bug bounty program. AI spam is bad. We've also never had a valid report from an by an LLM (that we could tell). People using them will take any being told why a bug report is not valid, questions, or asks for clarification and run them back through the same confused LLM. The second pass through generates even deeper nonsense. It's making even responding with anything but "clo…
This sounds more like an influx of scammers than security researchers leaning too hard on AI tools. The main problem is the bounty structure. And I don’t think these influx of low quality reports will go away, or even get any less aggressive as long as there is money to attract the scammers. Perhaps these bug bounty programs need to develop an automatic pass/fail tester of all submitted bug code, to ensure the report…
I am more interested about the why than on bashing AI-based code analyzers. Without checking, I am sure that AI will be able to find all sorts of vulnerabilities in my untested, unpublished week-end projects. What in curl makes AI-based analysis completely ineffective? The more positive take, and I think the biggest reason is that curl is just well made. But along the way, it most likely uses plenty of code analysis…
> Is there an overlap between one of these tools and AI, can one substitute for the other?
AI is a crude facsimile of any tool, which is both why it's useful and why it's ineffective. In the case linked from the post, it's hallucinating function names and likely hallucinating the entire patch. This hallucination would be an annoyance for the submitter using an AI tool to discover potential security vulnerabilities, and is both an annoyance and waste of time for the maintainer who was given the hallucination in bad faith.
Earlier quoted context omitted.
Reputation systems for this kind of thing sounds like rubbing some anti-itch cream on bullet wound. I feel like the problem seems to me to be behavior, not a technology issue. Personally I can't imagine how miserable it would be for my hard-earned expertise to be relegated to sifting through SLOP where maybe 1 in hundreds or even thousands of inquiries is worth any time at all. But it also doesn't seem prudent to jus…
I consider myself a left leaning soyboy, but this could be the outcome of too "nice" of a discourse. I won't advocate for toxicity, but I am considering if we bolster the self-image of idiots when we refuse to call them idiots. Because you're right, this is fundamentally a people problem, specifically we need people to filter this themselves. I don't know where the limit would go.
Earlier quoted context omitted.
I had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings! This was like two weeks ago. These things suck.
Isn't there a website that builds git man pages this way? By just stringing together random concepts into sentences that seem vaguely like something Git would implement. I thought it was silly and potentially harmful the first time I saw it. Apparently, it may have just been ahead of the curve.
Earlier quoted context omitted.
Reminds me of when some LLM (might have been Deepseek) told me I could add wasm_mode=True in my FastHTML python code which would allow me to compile it to WebAssembly, when of course there is no such feature in FastHTML. This was even when I had provided it full llms-ctx.txt
I had Google's in-search "AI" invent a command line switch that would have been very helpful... if it existed. Complete with usage caveats and warnings! This was like two weeks ago. These things suck.
Earlier quoted context omitted.
Good god did they hallucinate the segmentation fault and the resulting GDB trace too? Given that the diffs don’t even apply and the functions don’t even exist, I guess the answer is yes - in which case, this is truly a new low for AI slop bug reports.
An real report would have a GDB trace that looks like that, so it isn't hard to create such a trace. Many of us could create a real looking GDB trace just as well by hand - it would be tedious, boring, and pointless but we could.
My complaint is: if you're trying to use an AI to help you find bugs, you'd sincerely hope that they would have *some* attempt to actually run the exploit. Having the LLM invent fake evidence that you have done so, when you haven't, is just evil, and should be resulting in these people being kicked straight off H1 completely.