Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

github.com

Post reply on HN
Print view View original

71–80 of 569 posts

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#71

Post by eoskx »

nickvec wrote: post #60
otabdeveloper4 wrote: post #55

LiteLLM is the second worst software project known to man. (First is LangChain. Third is OpenClaw.) I'm sensing a pattern here, hmm.

Not familiar with LangChain besides at a surface level - what makes it the worst software project known to man?

LangChain at least has its own layer for upstream LLM provider calls, which means it isn't affected by this supply chain compromise. DSPy uses LiteLLM as its primary way to call OpenAI, etc. and CrewAI imports it, too, but I believe it prefers the vendor libraries directly before it falls back to LiteLLM.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#72

Post by tomaskafka »

shay_ker wrote: post #58

A general question - how do frontier AI companies handle scenarios like this in their training data? If they train their models naively, then training data injection seems very possible and could make models silently pwn people. Do the labs label code versions with an associated CVE to label them as compromised (telling the model what NOT to do)? Do they do adversarial RL environments to teach what's good/bad? I'm ve…

Everyone’s (well, except Anthropic, they seem to have preserved a bit of taste) approach is the more data the better, so the databases of stolen content (erm, models) are memorizing crap.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#73

Post by nickvec »

eoskx wrote: post #68

This is bad, especially from a downstream dependency perspective. DSPy and CrewAI also import LiteLLM, so you could not be using LiteLLM as a gateway, but still importing it via those libraries for agents, etc.

Wow, the postmortem for this is going to be brutal. I wonder just how many people/orgs have been affected.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#74

Post by eoskx »

mikert89 wrote: post #31

Wow this is in a lot of software

Yep, DSPy and CrewAI have direct dependencies on it. DSPy uses it as its primary library for calling upstream LLM providers and CrewAI falls back to it I believe if the OpenAI, Anthropic, etc. SDKs aren't available.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#75

Post by MuteXR »

ipython wrote: post #69
MuteXR wrote: post #63

Earlier quoted context omitted.

You know that people can already write backdoored code, right?

But now you have compromise _at scale_. Before poor plebs like us had to artisinally craft every back door. Now we have a technology to automate that mundane exploitation process! Win!

You still have a human who actually ends up reviewing the code, though. Now if the review was AI powered... (glances at openclaw)

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#76

Post by eoskx »

nickvec wrote: post #73
eoskx wrote: post #68

This is bad, especially from a downstream dependency perspective. DSPy and CrewAI also import LiteLLM, so you could not be using LiteLLM as a gateway, but still importing it via those libraries for agents, etc.

Wow, the postmortem for this is going to be brutal. I wonder just how many people/orgs have been affected.

Yep, I think the worst impact is going to be from libraries that were using LiteLLM as just an upstream LLM provider library vs for a model gateway. Hopefully, CrewAI and DSPy can get on top of it soon.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#77

Post by cozzyd »

rdevilla wrote: post #50

It will only take one agent-led compromise to get some Claude-authored underhanded C into llvm or linux or something and then we will all finally need to reflect on trusting trust at last and forevermore.

The only way to be safe is to constantly change internal APIs so that LLMs are useless at kernel code

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#78

Post by alansaber »

Imustaskforhelp wrote: post #70
shay_ker wrote: post #58

A general question - how do frontier AI companies handle scenarios like this in their training data? If they train their models naively, then training data injection seems very possible and could make models silently pwn people. Do the labs label code versions with an associated CVE to label them as compromised (telling the model what NOT to do)? Do they do adversarial RL environments to teach what's good/bad? I'm ve…

I am pretty sure that such measures aren't taken by AI companies, though I may be wrong.

The API/online model inference definitely runs through some kind of edge safeguarding models which could do this.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#79

Post by eoskx »

postalcoder wrote: post #30

This is a brutal one. A ton of people use litellm as their gateway.

Not just as a gateway in a lot cases, but CrewAI and DSPy use it directly. DSPy uses it as its only way to call upstream LLM providers and CrewAI falls back to it if the OpenAI, Anthropic, etc. SDKs aren't available.
Post reply on HN