21–28 of 28 posts
Am I correct in understanding that LoRA is basically a way to cheaply create “delta” LLMs that apply onto the main large one to create a specialization? In other words, this would obviate all the vector DB stuff that people are doing right?
Awesome work! Here's a recent paper released yesterday, also focused on efficiently serving many LoRAs simultaneously: https://arxiv.org/abs/2311.03285 Really looking forward to these innovations becoming more widespread -- I expect we're very close to a world where training a LoRA on a one-off task like "review every HN post from the last 3 years and flag any of them that contain informed speculation about the archi…
I'm curious if there is a quality argument to be made: imagine needing to finetune k different classifiers...
Before this work, we could train a single multi-label classifier by pooling the training sets, and deploy as 1 LoRa
Now, we can have k distinct classifiers, and not risk them interfering with one another
Any sense of, in realistic scenarios, when the quality of k distinct LoRas would be better?
That name is easy to confuse with the unrelated LoRa and LoRaWAN.
I think this is one of the most important possible works for open source LLM's, really glad y'all pushed this forward! That's not hyperbole. Why is OpenAI able to charge so little for their API's? I have heard rival mega LLM company CEO's complain that OpenAI's prices would be a loss for their rivals. But I think it's still positive margin, and that they can charge low prices for API because they've invested more int…
Doesn’t OpenAI still operate at significant losses by using massive infusions of capital from Microsoft and other investors? If you are giving away half your product, it’s not surprising that they would be undercutting competition. Not a new strategy. Underprice to avoid or drive out competition and encourage lock-in, then increase prices when you no longer have competitors or your user base is large enough and relia…
There is a difference between pricing aggressively and pricing at a loss. Their pricing for gpt-3.5-turbo now matches leading public providers for Llama-70B ($1/million tokens). Rumors are that 3.5-turbo is actually a 20B model, but even let's assume that it is larger than 70B: OpenAI can still price more aggressively than Llama-70B providers because they have better throughput and utilization of the same hardware.
I think this is one of the most important possible works for open source LLM's, really glad y'all pushed this forward! That's not hyperbole. Why is OpenAI able to charge so little for their API's? I have heard rival mega LLM company CEO's complain that OpenAI's prices would be a loss for their rivals. But I think it's still positive margin, and that they can charge low prices for API because they've invested more int…
Interesting. I'm not so sure I really 'got' that part of finetunes / LoRA adapters before reading this comment. Makes me want to make one to take it for a spin, see what comes out the other side.