41–50 of 143 posts
Serious question - why doesn’t someone crowd source the funds to train a GPT scale model for open source? I assume it’s not just a matter of a ton of GPU instances?
Doesn't one need to have a bunch of "very good" data to train on? I'm under the impression that sourcing costs are large.
Earlier quoted context omitted.
Doesn't one need to have a bunch of "very good" data to train on? I'm under the impression that sourcing costs are large.
Yes, and the labor of the Reinforcement Learning from Human Feedback (RLHF) .. is anyone crowdsourcing this?
Earlier quoted context omitted.
That's part of why I'm interested in their motivation changing licenses. They obviously had a profit motive and a model that centered on collecting rents on the model and decided to back off from it. Will they keep maintaining their code and improving the models and continue releasing everything under Apache 2.0? I'm just saying I feel like we got a glimpse of what they're about and it wasn't pretty, so why build aro…
They dont need to do squat. Look at Stable Diffusion 1.5 and LLaMA: They are thriving, but the original implementations are ancient history, and Meta/StabilityAI/RunawayML have done precisely nothing. And to be blunt, their legality is very ugly, which already makes Falcon more attractive.
I mean, that’s true of SD 1.5 in the sense that what the original creators have done since is new versions (SD 2.0, 2.1, and currently SDXL, which is apparent another SD2-architecture model, and DeepFloyd.) 2.1 has also seen some community uptake, and XL likely will once it is released unless there’s something inhibiting that. DF seems to be slowed by different architecture and high resource cost, but I’ve seen posts about people integrating the DeepFloyd early stage models with other models from the SD ecosystem for the last stage upscaling and final rendering, so I wouldn’t be surprised to see it integrated in some of the community UIs as both an integrated workflow and with access to the individual models for mix-and-match workflows.
Earlier quoted context omitted.
Doesn't one need to have a bunch of "very good" data to train on? I'm under the impression that sourcing costs are large.
Yes, and the labor of the Reinforcement Learning from Human Feedback (RLHF) .. is anyone crowdsourcing this?
4bit Quantized versions that run on an A100-40G or 2x3090/4090 24GB: https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ Inference is very slow right now but it works!
The hardware requirements on these models is basically at a fixed floor, and the democratisation will come from cheaper, possibly specialised, hardware, not reduced requirements, right?
Earlier quoted context omitted.
They dont need to do squat. Look at Stable Diffusion 1.5 and LLaMA: They are thriving, but the original implementations are ancient history, and Meta/StabilityAI/RunawayML have done precisely nothing. And to be blunt, their legality is very ugly, which already makes Falcon more attractive.
> Look at Stable Diffusion 1.5 and LLaMA: They are thriving, but the original implementations are ancient history, and Meta/StabilityAI/RunawayML have done precisely nothing. I mean, that’s true of SD 1.5 in the sense that what the original creators have done since is new versions (SD 2.0, 2.1, and currently SDXL, which is apparent another SD2-architecture model, and DeepFloyd.) 2.1 has also seen some community uptak…
Deepfloyd is niche.
SDXL is indeed interesting, especially if its happy with 4/8 bit quant... we will see about that.
Nevertheless StabilityAI seems kinda disconnected from all the innovations going on in the community compared to, say, huggingface.
4bit Quantized versions that run on an A100-40G or 2x3090/4090 24GB: https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ Inference is very slow right now but it works!
I thought that 4090s were "nerfed" and nvlink support removed - https://www.windowscentral.com/hardware/computers-desktops/n...
In terms of building something that's usable (considering cost, speed, scale, etc) if comparing an OpenAI API call to these, it's difficult for me to see a current path where these have any viable application outside some niche scenario. From what I understand, even to run locally you/your team needs to be able to afford a machine with a 4090. These are super expensive in some countries. I played around with the smal…
Fine-tuning is a much better proposition than you’re giving it credit for. Papers are coming out demonstrating that 7B parameter models can outperform GPT-4’s quality when trained on a limited set of tasks. Yet, a 7B model offers comparatively cheap and fast inference. Furthermore, for a lot of use cases, few-shot prompting is infeasible because you need to supply 2-3k tokens worth of few-shot examples with every pro…
4bit Quantized versions that run on an A100-40G or 2x3090/4090 24GB: https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ Inference is very slow right now but it works!
How is it possible to run this model on 2x 4090s ? I thought that 4090s were "nerfed" and nvlink support removed - https://www.windowscentral.com/hardware/computers-desktops/n...