131–140 of 148 posts
I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.
Did you build a machine with 4x 3090 ? I looking for a way to build such a machine for ML training.
Earlier quoted context omitted.
It's free for noncommercial use. If you use it in your company, your company should pay the membership fee. afaik most openai competitors also use similar usage restriction (e.g. free for noncommercial or research use, contact us for commercial license).
This basically means "Get sued." There is no clear legal, definition of "noncommercial," and courts have gone all sorts of different way on what constitutes commercial use. This is where CC NC licenses imploded. A lot of places (hello, MIT!) intentionally use CC NC licenses to make things appear more open than they are.
I've been experimenting with code-llama extensively on my laptop, and from my experience, it seems that these models are still in their early stages. I primarily utilize them through a Web UI, where they can successfully refactor code given an existing snippet. However, it's worth noting that they cannot currently analyze entire codebases or packages, refining them based on the most suitable solutions using the most…
I think there is a decent chance SourceGraph will figure this all out. The most important thing at this point is figuring what context to feed. They can build up a nice graph of a codebase and I expect from there they can put in the best context and then boom. They might also be able to train a model more intelligently by generating training data from said graphs.
You can use/try code-llama with Cody https://sourcegraph.com/blog/cody-vscode-1.1.0-release#:~:te...
Earlier quoted context omitted.
Business model is bundling so you have a one stop shop for good quality models of every modality and cultural variants of them. These go on bedrock, on chip, on prem etc and our consulting partners take them to the end user. On the innovation side stable diffusion turbo does like 100 cats with hats per second and the video model outperforms runway, pika etc on blind tests. Stable audio was one of the time innovation…
> Stable LM zephyr is the best 3b chat model By what measure? Phi 2 seems better as far as I can tell from benchmarks and usage and has much more permissive license.
From first principles, Phi 2 is extremely unlikely to be better, it's a base model and doesn't know how to chat. (see README on HF repo and also "Responses by phi-2 are off, it's depressed and insults me for no reason whatsover?", https://huggingface.co/microsoft/phi-2/discussions/61)
re: Benchmarks, see https://huggingface.co/stabilityai/stablelm-zephyr-3b. Phi-2 wins on some, StableLM on others. For some reason the HF and Lmsys leaderboards don't show it, and I don't know why.
Phi-2's license just changed and you still need to finetune it yourself. $20/month is more than reasonable for commercial use IMHO, it's a game changer.
Until I can use a truly* chat finetuned Phi-2, StableLM remains a clear winner in my experience. It can do RAG, the only other small model I've seen do that is Mistral 7B, and Phi-2 acts like PaLM acted when I would play around with it internally at Google, when it was just a base model. Impossible to use but fun toy.
* there's a couple other there, but they don't seem to have enough fine-tuning...yet
Earlier quoted context omitted.
How do you use these models with your editor? (E. vscode or Emacs etc)
I run tabby [0] which uses llama.cpp under the hood and they ship a vscode extension [1]. Going above 1.3b, I find the latency too distracting (but the highest end gpu I have nearby is some 16gb rtx quadro card that's a couple years old, and usually I'm running a consumer 8gb card instead). [0] https://tabby.tabbyml.com/ [1] https://marketplace.visualstudio.com/items?itemName=TabbyML....
I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.
Did you build a machine with 4x 3090 ? I looking for a way to build such a machine for ML training.
BTW, depending on where you're at in your ML journey, Jeremy Howard from FastAI says you should focus more on using hosted instances like paperspace until you really need to get your own machine. Unless, of course, you enjoy linux sysadmin tasks. :) It can get really annoying trying to match the right version of CUDA with the version of Pytorch you're trying to get running for the newest model you're trying.
Earlier quoted context omitted.
This basically means "Get sued." There is no clear legal, definition of "noncommercial," and courts have gone all sorts of different way on what constitutes commercial use. This is where CC NC licenses imploded. A lot of places (hello, MIT!) intentionally use CC NC licenses to make things appear more open than they are.
If you make money using it, pay them. If you're using it for free, don't worry
That's the reason you see "IANAL" disclaimers all over the internet. Legal advice from non-lawyers can be problematic in many ways. Some jurisdictions, although not where I live, you can even go to jail for giving bad legal advice without being a licensed lawyer.
Earlier quoted context omitted.
Business model is bundling so you have a one stop shop for good quality models of every modality and cultural variants of them. These go on bedrock, on chip, on prem etc and our consulting partners take them to the end user. On the innovation side stable diffusion turbo does like 100 cats with hats per second and the video model outperforms runway, pika etc on blind tests. Stable audio was one of the time innovation…
I use Stable Diffusion family models for innovative art products. On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged ( https://github.com/comfyanonymous/ComfyUI/pull/298 ). On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient chec…