I've been developing an alternative to LiteLLM. Javascript. No dependencies. https://github.com/johnhenry/ai.matey/
Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
161–170 of 569 posts
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#162We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm. The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
Thanks for the pointer! Love the premise project. Just a few notes:
- a security focused project should NOT default to train people installing by piping to bash. If i try previewing the install script in the browser it forces download instead of showing as plain text. The first thing i see is an argument
# --prefix DIR Install to DIR (default: ~/.smolvm)
that later in the script is rm -rf deleting a lib folder. So if i accidentally pick a folder with ANY lib folder this will be deleted.
- Im not sure what the comparison to colima with krunkit machines is except you don't use vm images but how this works or how it is better is not 100% clear
- Just a minor thing but people don't have much attention and i just saw aws and fly.io in the description and nearly closed the project. it needs to be simpler to see this is a local sandbox with libkrun NOT a wrapper for a remote sandbox like so many of the projects out there.
Will try reaching you on some channel, would love to collaborate especially on devX, i would be very interested in something more reliable and bit more lightweight in placce of colima when libkrun can fully replace vz
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#163LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09 2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt 3. The package is in quarantine on pypi - this block…
> - Krrish Was your account completely compromised? (Judging from the commit made by TeamPCP on your accounts) Are you in contacts with all the projects which use litellm downstream and if they are safe or not (I am assuming not) I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.
He would have to state he didn't in fact make all those commits and close the issue.
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#164We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
That's no solution. If you can't trust and/or verify dependencies, and they are malicious, then you have bigger problems than what a sandbox will protect against. Even if it's sandboxed and your host machine is safe, you're presumably still going to use that malicious code in production.
I'm supportive of going further - like restricting what a library is able to do. e.g. if you are using some library to compute a hash, it should not make network calls. Without sub-processes, it would require OS support.
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#165We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
That's no solution. If you can't trust and/or verify dependencies, and they are malicious, then you have bigger problems than what a sandbox will protect against. Even if it's sandboxed and your host machine is safe, you're presumably still going to use that malicious code in production.
Except that LiteLLM probably got pwned because they used Trivy in CI. If Trivy ran in a proper sandbox, the compromised job could not publish a compromised package.
(Yes, they should better configure which CI job has which permissions, but this should be the default or it won't always happen)
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#166LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far: 1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy... https://ramimac.me/trivy-teampcp/#phase-09 2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt 3. The package is in quarantine on pypi - this block…
The decision to block all downloads is pretty disruptive, especially for people on pinned known good versions. Its breaking a bunch of my systems that are all launched with `uv run`
> Its breaking a bunch of my systems that are all launched with `uv run`
From a security standpoint, you would rather pull in a library that is compromised and run a credential stealer? It seems like this is the exact intended and best behavior.
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#167Earlier quoted context omitted.
This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties. Which sounds great, but the way things work now tend to be the exact opposite of that, so there will be no trustable platform to run the untrusted code in. If the sandbox, or the operating system the sandbox run…
> This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties. For the most part you can. Just version pin slightly-stale versions of dependencies, after ensuring there are no known exploits for that version. Avoid the latest updates whenever possible. And keep aware of s…
Sure, and that is basically what sane people do now, but that only works until something needs a security patch that was not provided for the old version, and changing one dependency is likely to cascade so now I am open to supply chain attacks in many dependencies again (even if briefly).
To really run code without trust would need something more like a microkernel that is the only thing in my system I have to trust, and everything running on top of that is forced to behave and isolated from everything else. Ideally a kernel so small and popular and rarely modified that it can be well tested and trusted.
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#168How were they compromised? Phishing?
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#169[dead]
Re: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
#170Stuff like is happening too much recently. Seems like the more fast paced areas of development would benefit from a paradigm shift
Move Slow and Fix Things.