Viewing profile — tgaddair

In LoRAX v0.8 we've added native integration with Outlines, allowing you to guarantee your output always comes back in the structure of your choosing. But while structured generati…

Yes, we (LoRAX devs) saw that (we know the author pretty well). It's a useful addition, though quite a bit simpler than our level of support for multi-LoRA inference. We're plannin…

Thanks for the shoutout! We are adding the Outlines integration to LoRAX in the next two weeks, so stay tuned!

Hey, LoRAX dev here. This was one thing we spent a lot of effort optimizing. The TL;DR is that in most cases latency will be with 80% of the baseline latency with 0 adapters with a…