Isn’t the real danger now not the ability to find security vulnerabilities, but rather, the ability of anyone to ask an LLM agent to rewrite your open source project in another language and thus work around whatever license your project has?
Open Source Isn't Dead
51–60 of 200 posts
Re: Open Source Isn't Dead
#52cofounder here going closed source does not mean we are not fighting fire with fire we are using a handful of internal AI vulnerability scanners for months now being open source simply reduces risk by 5x to 10x according to several security researchers we are working with https://cal.com/blog/continuous-ai-pentesting-vulnerability-...
Don’t get me wrong but if virtually all modern software infrastructure lives on top of open source and they’re mostly fine then I’d imagine that you can make a scheduling webapp secure independent to if it’s OSS or not.
It’s OK if there’s another reason for this transition, just be transparent about it and don’t treat your users as children.
Re: Open Source Isn't Dead
#53It's a good question - is blackbox hacking as effective as whitebox hacking, for AI agents? I've gotta assume someone at Anthropic is putting together an eval as we speak.
I don't really know, but I have a story which might prompt some conversation about it.
At $WORK we had a system which, if you traced its logic, could not possibly experience the bug we were seeing in production. This was a userspace control module for an FPGA driver connected to some machinery you really don't want to fuck around with, and the bug had wasted something like three staff+ engineer-years by the time I got there.
Recognizing that the bug was impossible in the userspace code if the system worked as intended end-to-end, the engineers started diving into verilog and driver code, trying to find the issue. People were suspecting miscompilations and all kinds of fun things.
Eventually, for unrelated reasons, I decided to clean up the userspace code (deleting and refactoring things unlocks additional deletion and refactoring opportunities, and all said and done I deleted 80% of the project so that I had a better foundation for some features I had to add).
For one of those improvements, my observation was just that if I had to write the driver code to support the concurrency we were abusing I'd be swearing up a storm and trying to find any way I could to solve a simpler problem instead.
Long story short, I still don't know what the driver bug was, but the actual authors must've felt the same way, since when I opted for userspace code with simpler concurrency demands the bug disappeared.
Tying it back to AI and hacking, the white box approach here literally didn't work, and the black box approach easily illuminated that something was probably fucky. Given that AI can de-minify and otherwise spot patterns from fairly limited data, I wouldn't be shocked if black-box hacking were (at least sometimes) more token-efficient than white-box.
Re: Open Source Isn't Dead
#54I decided to not open source my latest project but it has nothing to do with security concerns. My code is perfectly secure and bug-free. My concern is mostly financial. Most people would be in a better position to monetize my software than I am... Using AI to obfuscate the origin while appropriating all the key innovations. I wouldn't get any credit. Also, I'm not really interested in humans anymore. I have human fa…
> My code is perfectly secure and bug-free.
I mean, bold statement but statistically speaking it's almost certainly incorrect. I will say that, irrespective of whether source is open or closed, I would be deeply skeptical of a project that made this assertion.
Re: Open Source Isn't Dead
#55[dead]
Re: Open Source Isn't Dead
#56I have an open source project and started receiving a lot of security vulnerability reports in the last few months. A lot of them are extremely corner cases, but there were some legit ones. They're all fixed now. Closed source software won't receive any reports, but it will be exploited with AI. So I definitely agree with the message of this article.
Assembly is still source code so really it comes down to if the copy protection is obscuring the executable code to the point where the LLM is not able to retrieve it on its own. And if it can't someone motivated could give it the extra help it needs to start tracing how outside inputs gets handled by the application.
Re: Open Source Isn't Dead
#57> The reasoning provided by their CEO, Bailey Pumfleet, is that AI has automated vulnerability discovery at scale, That sounds like an excuse. The real reason is probably that it's hard to make a viable business out of developing open source.
It’s also now ridiculously easy to simply cherry pick from open source without actually “using” it.
“I need to do foo in my app. Libraries bar and baz do these bits well. Pick the best from each and let’s implement them here”
I’d not be surprised if npmjs.com and its ilk turn into more a reference site than a package manager backend soon.
Re: Open Source Isn't Dead
#58feels like people are arguing the wrong axis tbh
- it’s not open vs closed anymore, it’s more like bug finding going a few devs poking around to basically infinite parallel scanners
- so now you don’t get a couple of thoughtful reports, you get a many edge cases and half-real junk. fixing capacity didn’t change though
- closing the repo doesn’t really save you, it just switches from white-box to black-box… and that’s getting pretty damn good anyway
real problem is: vuln discovery scaled, patching didn’t. now everything is a backlog game
Re: Open Source Isn't Dead
#59Brilliant piece of content marketing:
1) Pulls you in with a catchy title, that at first glance seems like a dunk on Cal.com (whatever that is).
2) Takes the "we understand your pain" approach to empathize w/ Cal.com, so you feel like you're on the good vibes side.
3) Provides a genuine response to the actual problem Cal.com is dealing with. Something you can't dismiss out of hand.
4) But in the end of the day, the response aligns perfectly with the product they're promoting (a click away to the homepage!)
This mix of genuine ideas and marketing is quite potent. Not saying this is all bad or anything, just found it a bit funny. The mixed-up-ness is the point!
Re: Open Source Isn't Dead
#60I have an open source project and started receiving a lot of security vulnerability reports in the last few months. A lot of them are extremely corner cases, but there were some legit ones. They're all fixed now. Closed source software won't receive any reports, but it will be exploited with AI. So I definitely agree with the message of this article.
I don't follow. It seems obvious that there's more to gain for attackers using AI agents to exploit open source repositories, than there is for good samaritan defenders. In this new closed-source world (for Cal.com), there's nothing stopping them from running their own internal security agent audits, all whilst at least blocking the easiest method of finding zero-days - that is, being open source.
This really just seems like Strix marketing. Which is totally fair, but let's be reasonable here, any open-source business stands to lose way more by continuing to be open-source vs. relying on the benevolence of people scanning their code for them.