191–200 of 569 posts
if you have tips i am sure they are welcome. snark remarks are useless. dont be a sourpuss. if you know better, help the remediation effort.
Configure the CI to make a release with the artefacts attached. Then have an entirely private repo that can't be triggered automatically as the publisher. The publisher repo fetches the artefacts and does the pypi/npm/whatever release.
Thank you for posting this, interesting. I hope that everyone's course of action will be uninstalling this package permanently, and avoiding the installation of packages similar to this. In order to reduce supply chain risk not only does a vendor (even if gratis and OS) need to be evaluated, but the advantage it provides. Exposing yourself to supply chain risk for an HTTP server dependency is natural. But exposing yo…
> Remember that you are programmers and you can just program, you don't need a framework, you are already using the API of an LLM provider, don't put a hat on a hat, don't get killed for nothing. Programming for different LLM APIs is a hassle, this library made it easy by making one single API you call, and in the backstage it handled all the different API calls you need for different LLM providers.
That's what they pay us for
I'd get it if it were a hassle that could be avoided, but it feels like you are trying to avoid the very work you are being paid for, like if a MCD employee tried to pay a kid with Happy Meal toys to work the burger stand.
Another red flag, although a bit more arguable, is that by 'abstracting' the api into a more generic one, you achieving vendor neutrality, yes, but you also integrate much more loosely with your vendors, possibly loose unique features (or can only access them with even more 'hassle' custom options, and strategically, your end product will veer into commodity territory, which is not a place you usually want to be.
Earlier quoted context omitted.
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm. The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
Probably on the side of your project, but did you try SmolBSD? https://smolbsd.org > It's a meta-OS for microVMs that boots in 10–15 ms. It can be dedicated to a single service (or a full OS), runs a real BSD kernel, and provides strong isolation. Overall, it fits into the "VM is the new container" vision. Disclaimer: I'm following iMil through his twitch streams (the developer of smolBSD and a contributor to NetBSD)…
At a glance, it's a matter of compatibility, most software has first class support for linux. But very interesting work and I'm going to follow it closely
Thank you for posting this, interesting. I hope that everyone's course of action will be uninstalling this package permanently, and avoiding the installation of packages similar to this. In order to reduce supply chain risk not only does a vendor (even if gratis and OS) need to be evaluated, but the advantage it provides. Exposing yourself to supply chain risk for an HTTP server dependency is natural. But exposing yo…
Comparing this project to is-odd seems very disingenuous to me. My understanding is this was the only way you could use llama.cpp with Claude Code for example, since llama.cpp doesn't support the Anthropic compatible endpoint and doing so yourself isn't anywhere near as trivial as your comparison. Happy to be corrected if I'm wrong.
Back in the days of GPT-3.5, LiteLLM was one of the projects that helped provide a reliable adapter for projects to communicate across AI labs' APIs and when things drifted ever so slightly despite being an "OpenAI-compatible API", LiteLLM made it much easier for developers to use it rather than reinventing and debugging such nuances.
Nowadays, that gateway of theirs isn't also just a funnel for centralizing API calls but it also serves other purposes, like putting guardrails consistently across all connections, tracking key spend on tokens, dispensing keys without having to do so on the main platforms, etc.
There's also more to just LiteLLM being an inference gateway too, it's also a package used by other projects. If you had a project that needed to support multiple endpoints as fallback, there's a chance LiteLLM's empowering that.
Hence, supply chain attack. The GitHub issue literally has mentions all over other projects because they're urged to pin to safe versions since they rely on it.
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
We just can't trust dependencies and dev setups. I wanted to say "anymore" but we never could. Dev containers were never good enough, too clumsy and too little isolation. We need to start working in full sandboxes with defence in depth that have real guardrails and UIs like vm isolation + container primitives and allow lists, egress filters, seccomp, gvisor and more but with much better usability. Its the same requir…
We need programming languages where every imported module is in its own sandbox by default.
u can run in chroot jail and it wouldnt have accessed ssh keys outside of the jail...
theres many more similar technologies aleady existing, for decades.
doing it on a per language basis is not ideal. any new language would have to reinvent the wheel.
better to do it at system level. with the already existing tooling.
openbsd has plege/unveil, linux chroot, namespaces, cgroups, freebsd capsicum or w/e. theres many of these things.
(i am not sure how well they play within these scenarios, but just triggering on the sandboxing comment. theres plenty of ways to do it as far as i can tell...)
It will only take one agent-led compromise to get some Claude-authored underhanded C into llvm or linux or something and then we will all finally need to reflect on trusting trust at last and forevermore.
Reflect in what way? The primary focus of that talk is that it’s possible to infect the binary of a compiler in a way that source analysis won’t reveal and the binary self replicates the vulnerability into other binaries it generates. Thankfully that particular problem was “solved” a while back [1] even if not yet implemented widely. However, the broader idea of supply chain attacks remains challenging and AI doesn’t…