Earlier quoted context omitted.
What's /g/?
https://4chan.org/g/ (enter at your own risk? I think it's kinda safe but still 4chan)
Message boards like this are so unreadable unless you have hours upon hours of spare time.
Fork of Facebook’s LLaMa model to run on CPU
141–150 of 178 posts
Re: Fork of Facebook’s LLaMa model to run on CPU
#142Earlier quoted context omitted.
Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.
Which is a good thing. So glad all that GPU compute is being used on cool stuff rather than running SHA-256 18 quintillion times
Definitely a good thing but FYI it hasn't been profitable/feasible to mine bitcoin (SHA-256) on GPU for many many years as ASIC based miners have completely taken over. I've talked about it plenty on HN but any way you slice it crypto is an unbelievable waste of resources in every possible way regardless.
What really (finally) more or less killed GPU mining was the Ethereum move to PoS (Proof of Stake).
Re: Fork of Facebook’s LLaMa model to run on CPU
#143Earlier quoted context omitted.
Which is a good thing. So glad all that GPU compute is being used on cool stuff rather than running SHA-256 18 quintillion times
> running SHA-256 18 quintillion times or games. People could have been studying or doing something more important than wasting time and energy. I get that it is entertainment, but so are board games and that don't require mining rare earth minerals or putting pressure on the grid as you can always play board games with candles on.
I'm of two minds on this. I'm not a gamer so part of me thinks gaming is a complete waste of time and resources. Then again, the same could be said about almost any hobby/pastime.
That said, gaming is what gave us GPUs (which have developed for gaming over the course of decades) so that we can now utilize them for more interesting and "productive" applications.
So, for me, in the end I'm happy the PC gaming industry and user base has been pushing GPU capability.
Re: Fork of Facebook’s LLaMa model to run on CPU
#144Earlier quoted context omitted.
For most part of the 20th, a bulk of the energy humanity was able to extract was used for industrialization. Now it seems that a vast bulk of the energy being extracted will go towards computation.
I doubt it frankly. Computation consumes a lot of energy, true, but it is dwarfed by how much energy we use in transportation and food production. Energy use per capita in most of the Global North is about 75,000kWh per year, https://ourworldindata.org/grapher/per-capita-energy-use That's like the average person running 27 NVIDIA A100s at max capacity at all times!
Yeah but every time we discover a new interesting thing to do with computers the requirements go up by several orders of magnitude. How many more orders of magnitude of energy can we spend on food production from here with current projections of world population to peak in 2100?
Re: Fork of Facebook’s LLaMa model to run on CPU
#145Could this fit into GitHub Codespaces's top VM?
The 65 billion model is 160 GB so no - unless you request larger storage spaces from github. 7 billion and 13 billion should fit though.
Re: Fork of Facebook’s LLaMa model to run on CPU
#146Earlier quoted context omitted.
Oh yes “”” Hackernews senator: “”Someone on the internet said meta aka Facebook is not considered a real data native, clean coder and high IQ company unless your new language model exceeds the elegance and slipperiness of mark Zuckerbergs (you) language output in senate hearings. he is smoother than a lake in the metaverse.“” Mark LLM: “ Yes, unfortunately, the media and our competitors are all over the idea that Met…
I have to say "he is smoother than a lake in the metaverse" is presumably accidental, based on the quality of the rest of that text, but it has to be one of the wittiest phrases ive seen LLMs come out with to date
I opened the twitch AI seinfeld stream once and stumbled into a conversation that went something to the effect of:
George: I really like that orange sweater
Jerry: Yeah, I just found black so depressing
George: Orange is such a great color! Orange is the new black.
...
Re: Fork of Facebook’s LLaMa model to run on CPU
#1470.35 words/s on my 11th gen i5 with 7B model (framework laptop) not so bad !
How long did you have to wait for it to load? On my machine it's been running for 15mins, I'm still waiting for a prompt...
Re: Fork of Facebook’s LLaMa model to run on CPU
#148Re: Fork of Facebook’s LLaMa model to run on CPU
#149Earlier quoted context omitted.
What's /g/?
https://4chan.org/g/ (enter at your own risk? I think it's kinda safe but still 4chan)
Better to direct people to the catalog: https://boards.4channel.org/g/catalog
Browsing page by page is not a good idea.
Search for "aicg" or visit https://boards.4channel.org/g/catalog#s=aicg to see the AI Chatbot General thread (a new one is created every time the previous one hits the reply limit).
Re: Fork of Facebook’s LLaMa model to run on CPU
#150Earlier quoted context omitted.
That's genuinely surprising. What sort of on-board compute do you typically have today?
I don't work in the field but just to kind of put it into perspective, a 12v 100A LiFePO4 battery has 1200 Watts capacity and weighs 30 pounds. A typical gaming PC (which to be fair, is more willing to trade power for performance) consumes about 600 Watts per hour. Problem for a Tesla? Not so much. Problem for a lightweight drone? Definitely.
ahhh the units in this post are making my eye twitch.