Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

github.com

Post reply on HN
Print view View original

201–210 of 569 posts

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#201

Post by saidnooneever »

syllogism wrote: post #193

Maintainers need to keep a wall between the package publishing and public repos. Currently what people are doing is configuring the public repo as a Trusted Publisher directly. This means you can trigger the package publication from the repo itself, and the public repo is a huge surface area. Configure the CI to make a release with the artefacts attached. Then have an entirely private repo that can't be triggered aut…

this kind of compromise is why a lot of orgs have internal mirrors of repos or package sources so they can stay behind few versions to avoid latest and compromise. seen it with internal pip repos, apt repos etc.

some will even audit each package in there (kind crap job but it works fairly well as mitigation)

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#203

Post by thibault000 »

sschueller wrote: post #29

Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.

[dead]

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#204

Post by vlovich123 »

cozzyd wrote: post #200
vlovich123 wrote: post #100

Earlier quoted context omitted.

Reflect in what way? The primary focus of that talk is that it’s possible to infect the binary of a compiler in a way that source analysis won’t reveal and the binary self replicates the vulnerability into other binaries it generates. Thankfully that particular problem was “solved” a while back [1] even if not yet implemented widely. However, the broader idea of supply chain attacks remains challenging and AI doesn’t…

I believe the issue is if an exploit is somehow injected into AI training data such that the AI unwittingly produces it and the human who requested the code doesn't even know.

That’s a separate issue and specifically not what OP was describing. Also highly unlikely in practice unless you use a random LLM - the major LLM providers already have to deal with such things and they have decent techniques to deal with this problem afaik.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#205

Post by amelius »

saidnooneever wrote: post #199
amelius wrote: post #113

Earlier quoted context omitted.

We need programming languages where every imported module is in its own sandbox by default.

just sandbox the interpreter (in this case), package manager and binaries. u can run in chroot jail and it wouldnt have accessed ssh keys outside of the jail... theres many more similar technologies aleady existing, for decades. doing it on a per language basis is not ideal. any new language would have to reinvent the wheel. better to do it at system level. with the already existing tooling. openbsd has plege/unveil,…

What if I wanted to write a program that uses untrusted libraries, but also does some very security sensitive stuff? You are probably going to suggest splitting the program into microservices. But that has a lot of problems and makes things slow.

The problem is that programs can be entire systems, so "doing it at the system level" still means that you'd have to build boundaries inside a program.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#206

Post by dist-epoch »

jFriedensreich wrote: post #86

We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…

This stuff already exists - mobile phone sandboxed applications with intents (allow Pictures access, ...)

But mention that on HN and watch getting downvoted into oblivion: the war against general computation, walled gardens, locked down against device owners...

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#207

Post by f311a »

samsk wrote: post #158
f311a wrote: post #134

Their previous release would be easily caught by static analysis. PTH is a novel technique. Run all your new dependencies through static analysis and don't install the latest versions. I implemented static analysis for Python that detects close to 90% of such injections. https://github.com/rushter/hexora

Interesting tool, will definitely try - just curious, is there a tool (hexora checker) that ensures that hexora itself and its dependencies are not compromised ? And of course if there is one, I'll need another one for the hexora checker....

There is no such tool, but you can use other static analyzers. Datadog also has one, but it's not AST-based.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#208

Post by fn-mote »

nazcan wrote: post #164
uyzstvqs wrote: post #153

Earlier quoted context omitted.

That's no solution. If you can't trust and/or verify dependencies, and they are malicious, then you have bigger problems than what a sandbox will protect against. Even if it's sandboxed and your host machine is safe, you're presumably still going to use that malicious code in production.

I'm supportive of going further - like restricting what a library is able to do. e.g. if you are using some library to compute a hash, it should not make network calls. Without sub-processes, it would require OS support.

Which exists: pledge in OpenBSD.

Making this work on a per-library level … seems a lot harder. The cost for being very paranoid is a lot of processes right now.

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#209

Post by detente18 »

detente18 wrote: post #99

LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09 2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt 3. The package is in quarantine on pypi - this block…

Update:

- Impacted versions (v1.82.7, v1.82.8) have been deleted from PyPI - All maintainer accounts have been changed - All keys for github, docker, circle ci, pip have been deleted

We are still scanning our project to see if there's any more gaps.

If you're a security expert and want to help, email me - krrish@berri.ai

Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised

#210

Post by detente18 »

Imustaskforhelp wrote: post #106
detente18 wrote: post #99

LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09 2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt 3. The package is in quarantine on pypi - this block…

> - Krrish Was your account completely compromised? (Judging from the commit made by TeamPCP on your accounts) Are you in contacts with all the projects which use litellm downstream and if they are safe or not (I am assuming not) I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.

It was the PYPI_PUBLISH token which was in our github project as an env var, that got sent to trivvy.

We have deleted all our pypi publishing tokens.

Our accounts had 2fa, so it's a bad token here.

We're reviewing our accounts, to see how we can make it more secure (trusted publishing via jwt tokens, move to a different pypi account, etc.).

Post reply on HN