111–120 of 146 posts
Earlier quoted context omitted.
How much money you think gpt3 training costed?
How much money do we spend contributing to the training set? Those insights, comments, articles, code example, etc are free to use because we published those on sites that don't own the content but earn from it. If they owned them, the they would be responsible for hate speech. So our costs for producing the training set is negligible.
Earlier quoted context omitted.
Hmmm, 2-3 high end GPUs implies it's likely not very far off from mainstream. Maybe runnable on the average device within 10-20 years... perhaps even sooner if the model/software can be optimized?
> 2-3 high end GPUs implies it's likely not very far off from mainstream Looks like FLOP/s per $ are doubling every ~3 years for high end cards, and 10x in ~10 years. So probably not that far off for desktop users. https://www.lesswrong.com/posts/c6KFvQcZggQKZzxr9/trends-in-...
Earlier quoted context omitted.
> Can these models feasibly be run locally? Bluntly, no. The models which are small enough to run locally perform so badly it’s not worth bothering. To run inference on the large models the perform decently you need the equivalent of two or three top end graphics cards. If you're serious about looking into it now, consider looking at this project that lets you run a bunch of independent machines as a cluster for infe…
How much of this is the language vs the vast amount of passably accurate domain knowledge? ChatGPT etc. seem magic because they can answer questions about virtually anything with a high degree of plausibility. It often gets specific facts wrong, but the general contours are correct. Many of us know a lot of trivia/specialist knowledge, but I don't think anyone is as broadly informed as ChatGPT appears to be. It's not…
Earlier quoted context omitted.
[flagged]
You details of what is required can be found per project. eg. https://github.com/fauxpilot/fauxpilot/blob/main/setup.sh#L5... /shrug If you don't want to run it locally, don't. If the hardware you have doesn't support it, don't. ...but to say it's not possible is, again, bluntly, wrong.
Earlier quoted context omitted.
yeah, that's not what op asked about. That's one of those smaller models that "suck" as you said so confidently in your first comment. Not even close to gpt3/chatgpt. Next time when I talk bullshit I just post a random py file to make my argument. lmao
Though if true, it does beg the question, how can these services possibly be offered profitably if a single query requires many high end GPUs to run? Doesn't really track logically either. Though I'm not in the know. Perhaps ChatGPT is run with massive operating losses and its all VC subsidized.
Earlier quoted context omitted.
This post [0] is a good primer and they are talking about a 20b model gpt3 has 175b. Their other posts go more into details but yes, it really is a massive operation. Some hard facts from here [1] talking about BLOOM with 175b parameters.. >> Installing the full 175B version is a challenge though as it requires around 350GB of GPU VRAM, which is not something one can easily afford. But hey why deploy a model like cha…
So it sounds like this is a question of loading the model into VRAM, and not a question of the cost of a single query. I assume once a model is loaded, many queries can be serviced by that model quickly. There's nothing incorrect about my assertion. If it were to actually take many GPUs to service one query, then there is no mass scale cost viable consumer product. That's just a clear economic fact. Regardless if a m…
Depends. If you have room to load the whole model, yes. If you need to swap in and out parts of the model, then it matters if you have enough RAM.
So what's the level of effort to create ChatGPT equivalent products? Is it something where we'll have 100s of competing AIs, or is it gated to only a few large companies? Not up to date on current training/querying costs. Can these models feasibly be run locally? Given the large number of competitors already announced to ChatGPT, I fail to see how the space will be easily defensible or monetizable (despite large valu…
Datasets. The one with the largest, most personal, most obtrusive, invasive dataset will probably win. The one that has absorbed every podcast, every youtube video, every close-caption text in existence, will have the most "complete" answers.
What is going to make a difference is running models to generate more text for training, because relying on humans alone doesn't scale. For example we could be using LLMs to do brute force problem solving and then fine-tuning on solutions.
AlphaZero is the shining example of a model trained on its own generated data and surpassing us at our own game. The self generated data approach has potential to reach super human levels of performance.
Earlier quoted context omitted.
From guide linked above: > It is remarkable that such large multi-lingual model is openly available for everybody. Am I the only one thinking that this remark is a insight into societal failure? The model has been trained on global freely available content, anyone who has published on the Web has contributed. Yet the wisdom gained from our collective knowledge is assumed to be withheld from us. As the original remark…
I think it’s similar to how search engines keep their ranking formulas secret, and you can’t run your own off a copy of their index. Yet we also all contributed to it by publishing (and feeding it, for instance by following googles requirements for micro data). But we don’t own any of it.
The same is not true for these AI tools. The output could have been contributed by you, someone else, or everyone, or a combination of those, but it'll never be clear who actually contributed and there will be no credit to anyone besides the author(s) of the models.
Earlier quoted context omitted.
> if you have GPUs with > 330GB VRAM, it'll run fast What kind of GPU's have that that are available to consumers, how much would such a kit cost roughly?
You'd basically need a rack mount server full of Nvidia H100 cards (80 Vram, they cost $40 thousand us dollars each). So... good luck with that? On the relatively cheap end Nvidia tesla cards are kinda cheap used, 24 gig ones going for ~$200 with architectures from a few years ago. That's still nearly $3000 worth of cards not counting the rest of the whole computer. This isn't really something you can run out home wi…