Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My minute-by-minute response to the LiteLLM malware attack

futuresearch.ai

Post reply on HN
Print view View original

171–180 of 194 posts

Re: My minute-by-minute response to the LiteLLM malware attack

#171

Post by minhduc0711 »

StrauXX wrote: post #164
rpodraza wrote: post #76

At this point I'd highly recommend everyone to think twice before introducing any dependencies especially from untrusted sources. If you have to interact with many APIs maybe use a proxy instead, or roll your own.

LiteLLM isn't a good choice for a proxy in any case. It introduces a lot of lag amd latency and the features are often half baked. To me, it looks like a vibecoded application without a product owner. And the code itself isn't very organized either. I evaluated it for a project a few months ago and will never use it for anything production. Theres a few much better alternatives out there.

Could you name some of these better alternatives?

Re: My minute-by-minute response to the LiteLLM malware attack

#172

Post by latexr »

Fibonar wrote: post #2

Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuring out what was going on in real time, unedited with only minor redactions. I didn’t need to recount my thought process after the fact. It’s the very same ones I wrote down to help Claude figure out what was happening. I’m an ML engineer by trade, so having Cla…

> I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache?

cURL had to stop the bug bounty program because they were inundated by slop reports of vulnerabilities which don’t exist.

https://github.com/curl/curl/pull/20312

It’s good that you found and reported something real, but that isn’t the norm.

Also, from the article:

> AI tooling has sped up not just the creation of malware but also the detection.

That’s an awful tradeoff. Detection is not a fix.

Re: My minute-by-minute response to the LiteLLM malware attack

#173

Post by latexr »

rgambee wrote: post #6
Fibonar wrote: post #2

Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuring out what was going on in real time, unedited with only minor redactions. I didn’t need to recount my thought process after the fact. It’s the very same ones I wrote down to help Claude figure out what was happening. I’m an ML engineer by trade, so having Cla…

I've heard stories lately of open source projects being inundated with vulnerability reports and PRs. But in this case, it seems like AI assistance was clearly a boon for root-causing and reporting this so quickly.

Yes, in this case. Chances are in the future others will be missed despite being reported, because the maintainers are either inundated or tired of all the other reports and can’t distinguish the real one.

Re: My minute-by-minute response to the LiteLLM malware attack

#174

Post by latexr »

rmunn wrote: post #159
joquarky wrote: post #139

Earlier quoted context omitted.

Don't think of a pink elephant.

Best way I've found not to think of a pink elephant is to choose to think of a green rabbit. Really focus on the mental image of the green rabbit... and voila, you're not thinking of, what was it again? Eh, not as important as this green rabbit I'm focusing on. How to translate that to LLM world, though, is a question I don't know the answer to. P.S. Obviously that won't prevent you from having that first mental flas…

> Best way I've found not to think of a pink elephant is to choose to think of a green rabbit.

Seems easy circumventable: “Don’t think of a green rabbit”. Now the past vividness of that image becomes a hindrance.

Re: My minute-by-minute response to the LiteLLM malware attack

#175

Post by kryptiskt »

__mharrison__ wrote: post #55
simonw wrote: post #49

Earlier quoted context omitted.

The package in question was live for 46 minutes. It generally takes longer than that for security partners to scan and flag packages. PyPI doesn't block package uploads awaiting security scanning - that would be a bad idea for a number of reasons, most notably (in my opinion) that it would be making promises that PyPI couldn't keep and lull people into a false sense of security.

I realize this is controversial (and many Python folks would claim anti ethical). But I keep wondering if requiring a small payment for registering and updating packages would help. The money could go to maintaining pypix as well as automated AI analysis. Folks who really couldn't afford it could apply for sponsorship.

I don't see how this would help in the least, what kind of criminal would be dissuaded by paying a small fee to set an elaborate scheme such as this in motion? This is not a spamming attack where the sheer volume would be costly. It doesn't even help to get a credit card on file, since they can use stolen CC numbers.

It's far more likely that hobbyists will be hurt than someone that can just write off the cost as a small expense for their criminal scheme.

Re: My minute-by-minute response to the LiteLLM malware attack

#177

Post by onepunchmob »

nubinetwork wrote: post #123

I have a hard time believing that Claude instantly figured out this was malware... I've fed it obfuscated JavaScript before, and it couldn't figure it out... and then there was the time I tried to teach it nftables... whooo boy...

They are really good at this, had codex discover similar malware from another supply chain attack months ago because my laptop was running hot. Actually crazy times we live in, I would certainly not be able to discover this without agent help.

Re: My minute-by-minute response to the LiteLLM malware attack

#178

Post by patrakov »

__mharrison__ wrote: post #55
simonw wrote: post #49

Earlier quoted context omitted.

The package in question was live for 46 minutes. It generally takes longer than that for security partners to scan and flag packages. PyPI doesn't block package uploads awaiting security scanning - that would be a bad idea for a number of reasons, most notably (in my opinion) that it would be making promises that PyPI couldn't keep and lull people into a false sense of security.

I realize this is controversial (and many Python folks would claim anti ethical). But I keep wondering if requiring a small payment for registering and updating packages would help. The money could go to maintaining pypix as well as automated AI analysis. Folks who really couldn't afford it could apply for sponsorship.

Not speaking on behalf of PSF, but to me, it looks like a no-go, as some packages are maintained, legitimately, by people from sanctioned countries, with no way to pay any amount outside their country.

Re: My minute-by-minute response to the LiteLLM malware attack

#179

Post by lq9AJ8yrfs »

thierrydamiba wrote: post #144
lq9AJ8yrfs wrote: post #48

Earlier quoted context omitted.

As a sometimes peripheral and sometimes primary program manager for vulnerability disclosure, for companies you nearly can't avoid, $0.02 follows. It's a signal vs noise thing. Most of the grief is caused by bottom feeders shoveling anything they can squint at and call a vulnerability and asking for money. Maybe once a month someone would run a free tool and blindly send snippets of the output promising the rest in e…

Isn’t this a great use of llms? Clone the repo in a sandbox and have the llm identify if the issues are real and the appropriate response based on severity level. Wouldn’t be perfect but would have caught something like this.

I don't think I've met an llm that is adversary resistant, and here are counterparties that are actively playing the field, to put it mildly.

The bug bounty service providers did an adequate job of filtering out junk reports. There was a survivorship bias, some of the bogus ones that got through had an uncanny ability to twist words.

Post reply on HN