251–260 of 306 posts
Earlier quoted context omitted.
(Chat)GPT-4s practical coding abilities are now 100x because it can code, run the code, and reason about its performance mid-response. They must be using fine tunes for this so the overall model could well be better too
You can do that as well, under your complete control. That's a framework they put around the model.
I used the original 34B last night with 4 bit through accelerate and I was absolutely blown away. I got goosebumps because it finally felt like we have models we can run on consumer hardware (single 3090 in this case) and did not feel like a toy. I purposely broke some functions, it fixed it. I asked some complex questions and it answered it well. I’m excited for what’s to come. I wish there was a Phind instruct mode…
It really is good. Surprisingly it seems to answer instruct-like prompts well! I’ve been using it with Ollama ( https://github.com/jmorganca/ollama ) with prompts like: ollama run phind-codellama "write c code to reverse a linked list" To run this on an m1 Mac or similar machine, you'll need around 32GB of memory for the 4-bit quantized version since it's a 34B parameter model and is quite big (20GB).
> write c code to reverse a linked list
The number of times I had to do it in real production code amounts to zero.
The number of times I had to piece code from poorly documented external services, conflicting product requirements, and running on complex and weird environments has been ... well ... multiple times a day for the past 20+ years.
"instruct-like prompts" is what you give a very junior engineer out of college, and then you have to carefully review their code.
Their pricing page says 30 uses per day of (GPT-4). What exactly does that mean, they send my query to Open AI's GPT 4 and return the result?
Earlier quoted context omitted.
This project is sweet and I fortunately have a Mac M1 laptop with 64GB RAM to play with it. But I'd like to also be able to run these models on my Linux desktop with two GPU's (a 2080Ti and a 3080Ti) and a Threadripper. How difficult would it be to set some of these up on there?
Not hard with text-generation-ui! https://github.com/oobabooga/text-generation-webui I personally use llama.cpp as the driver since I run CPU-only but another may be better suited for GPU usage. But then it's as simple as downloading the model and placing it in the directory.
Earlier quoted context omitted.
Not hard with text-generation-ui! https://github.com/oobabooga/text-generation-webui I personally use llama.cpp as the driver since I run CPU-only but another may be better suited for GPU usage. But then it's as simple as downloading the model and placing it in the directory.
My Threadripper has 64 cores/128 threads and I'm wondering if any models can take advantage of CPU concurrency to at least mitigate some of the loss from not using a GPU (should one not be using a GPU)
Earlier quoted context omitted.
Perhaps it is because it is true and both of you are showing signs of great denial. Engineers will be replaced by those that use AI and it will only accelerate and there will be less of them hired.
Engineers will be replaced by those that use AI By that time not only engineers will get replaced. What do you do if I may ask ?
Earlier quoted context omitted.
That's what I've been using but still get that error.
Same here, would be interested to know if there's a solution! I created an issue, if you have an example prompt to add that would be helpful! https://github.com/jmorganca/ollama/issues/422
Earlier quoted context omitted.
Is this really the line we want to draw in the sand? It’s not open source because it can’t be trivially used by AWS and Google? It feels like the popularism is the point, not the complete lack of restrictions.
Well if we are talking about the point of open source, then really, the lack of restrictions is the point. Some commercial entities wish to redefine the term because it benefits their marketing, but is that the line we want to draw in the sand?