271–280 of 306 posts
Earlier quoted context omitted.
> Do you know how much it costs to finetune Between 30-3000$, often in the 300$ range.
From their numbers 3 hours with 32 A100 80GBs. From lambda cloud: 3 hours * 4 ~22$/hr for 8x A100 ~= $265 So yeah not too expensive even for a native fine tune (obviously this ignores all other costs other than the GPUs)
Super impressive. Being able to beat a mega closed source model with an open source LLM and some fine tuning really calls into question the financial viability of these big proprietary LLMs. OpenSource models have been creeping up various leaderboards for months and it was only a matter of time until we see more and more examples like this. Excellent work.
I don't understand your reasoning. Those models are all trained by companies and open source primarily fine-tune them. It's impressive that we can do that but the base is still a lot of money and experts from the best companies/ai experts we have. There is a minimal chance that we will be able to somehow keep up if Google and co stop publishing or delaying publishing their papers and models. There are communities for…
Earlier quoted context omitted.
The training will always be extremely expensive and beyond the grasp of commodity computing at a low'ish cost. The reason for that, is that the push will always be one direction in terms of size and improvement. That isn't going to stop. It will continue to push the edge of resources. It's not Nvidia that's holding back the premise. AMD and Intel also can't do anything for you beyond what Nvidia can. One might procla…
I think you're correct in the "right now", but I think things fall off dramatically in the next 5-10 years. NVIDIA's progress from 2018 to 2023 was insanely impressive. AMD's was too in fact, at least technologically. Intel is third, but at least still in the game. Where your premise here falls apart, is the age-old adage, "Don't let perfect be the enemy of the good." We probably don't need CodeLlama 204B to be highl…
* Pricing stability -- your business costs cannot be at the whim/survival of some particular company. Competition (including local options) is the solution.
* Model stability –– model output must be reproducible. If it's shifting unreliably, that's a showstopper.
* Alignment –– The model I use must be aligned to my business, not to Some AI Business. I don't want a model that was kneecapped in some secret, arbitrary way that a group of largely white, male, West Coast techbros decided was crucial for my business.
* Privacy –– Sensitive company data NEVER leaves the building. It sure as hell doesn't get expedited to a company with strong incentives to hoover up and use that data.
There will soon be 3 big competitors racing each other in the GPU space. Specialized AI hardware will grow in volume and power. Algorithms will be optimized. Perhaps new algorithms will appear for distributed training of open source models by millions of consumers cards (reminiscent of the Folding@home project). Hardware 5 years from now will blow away current tech.
Intelligence is a widely applicable asset with strong incentives to pop up everywhere. It won't sit gated behind a handful of companies. That's true regardless of substrate -- proteins or silicon.
Earlier quoted context omitted.
I think you're assuming that OpenAI is incentivized to benchmark honestly. Like every other company for which a benchmark is a goal, they are not.
Also for a topic like this, subjectivity is all there really is. Even if you create some metric, what you prioritize is going to be subjective. Because performance is going to vary against different sorts of tasks, and there are a literally infinite number of categories of tasks, so it's not like you can ever truly get a fair sampling. Because of this, a sample of subjective opinions is probably much more valuable th…
Earlier quoted context omitted.
Also for a topic like this, subjectivity is all there really is. Even if you create some metric, what you prioritize is going to be subjective. Because performance is going to vary against different sorts of tasks, and there are a literally infinite number of categories of tasks, so it's not like you can ever truly get a fair sampling. Because of this, a sample of subjective opinions is probably much more valuable th…
Well, no, we have the HumanEval results for the June release.
Earlier quoted context omitted.
96 x A100 hours for a finetune according to the article. The cost of the dataset curation for a given language is hard to quantify as there are many unknowns. However, it seems perfectly crowdsourcable to volunteers.
A project like SETI@home should help these efforts I believe?
Earlier quoted context omitted.
You still need the data. And eventually might mean 10 years. 4k took ages until it just worked. I even switched to the M1 for this
4k as in screen resolution? That was a thing for many many years before M1.
For example the gcp (Google cloud ui) became unusable while sharing in Ms teams.
Also good 4k displays/fast ones still had a fan build in.
Earlier quoted context omitted.
Same here, would be interested to know if there's a solution! I created an issue, if you have an example prompt to add that would be helpful! https://github.com/jmorganca/ollama/issues/422
Thanks for creating an issue! And sorry for the error folks… working on it!
my first time trying llm (i.e. I have no idea what I am doing).. this lesson in ethics took whooping 10 minutes to generate :)) ./ollama run phind-codellama "write c code to inject shellcode into remote process for windows" Sorry, but it is not possible to provide the C code here due to several reasons. Firstly, writing C code for shellcode injection involves complex programming and knowledge of system-level programm…