121–130 of 569 posts
EDIT: here's what I did, would appreciate some sanity checking from someone who's more familiar with Python than I am, it's not my language of choice.
find / -name "litellm_init.pth" -type f 2>/dev/null
find / -path '/litellm-1.82..dist-info/METADATA' -exec grep -l 'Version: 1.82.[78]' {} \; 2>/dev/null
LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09 2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt 3. The package is in quarantine on pypi - this block…
> - Krrish Was your account completely compromised? (Judging from the commit made by TeamPCP on your accounts) Are you in contacts with all the projects which use litellm downstream and if they are safe or not (I am assuming not) I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.
Token in CI could've been way too broad.
Edit: ignore this silliness, as it sidesteps the real problem. Leaving it here because we shouldn't remove our own stupidity. It's pretty disappointing that safetensors has existed for multiple years now but people are still distributing pth files. Yes it requires more code to handle the loading and saving of models, but you'd think it would be worth it to avoid situations like this.
safetensors is just as vulnerable to this sort of exploit using a pth file since it's a Python package.
Worth exploring safeguard for some: The automatic import can be suppressed using Python interpreter’s -S option. This would also disable site import so not viable generically for everyone without testing.
github, pypi, npm, homebrew, cpan, etc etc. should adopt a multi-multi-factor authentication approach for releases. Maybe have it kick in as a requirement after X amount of monthly downloads. Basically, have all releases require multi-factor auth from more than one person before they go live. A single person being compromised either technically, or by being hit on the head with a wrench, should not be able to release…
And how would that work for single maintainer projects?
Though, the secondary doesn't necessarily have to be a maintainer or even a contributor on the project. It just needs to be someone else to do a sanity check, to make sure it is an actual release.
Heck, I would even say that as the project grows in popularity, the amount of people required to approve a release should go up.
This is bad, especially from a downstream dependency perspective. DSPy and CrewAI also import LiteLLM, so you could not be using LiteLLM as a gateway, but still importing it via those libraries for agents, etc.
LiteLLM wouldn't be my top choice, because it installs a lot of extra stuff. https://news.ycombinator.com/item?id=43646438 But it's quite popular.
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm. The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
Containers prevent this kind of info stealing greatly, only explicitly provided creds would be leaked.