Live data from Hacker News

Fork of Facebook’s LLaMa model to run on CPU

github.com

81–90 of 178 posts

Re: Fork of Facebook’s LLaMa model to run on CPU

#81
post #74
post #21

Earlier quoted context omitted.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

For most part of the 20th, a bulk of the energy humanity was able to extract was used for industrialization. Now it seems that a vast bulk of the energy being extracted will go towards computation.

The singularity and fusion power are probably interlinked because of this. Once one happens the other will and in either order.

Re: Fork of Facebook’s LLaMa model to run on CPU

#82
post #58
post #23

Earlier quoted context omitted.

What we will get are specialized hardware, with not so open APIs anyway. With a bunch of people trailling behind with "it kind of works" open alternatives.

It sounds like you are complaining about capitalism :-) It's not so bad. Nvidia could come and say, "hey, I'm going to lock down your GPU so that you can only use it to render polygons in my whitelisted list of video-games, and then you pay us $$$$$$ to buy our 'datacenter' thingy for anything else." But if they do it, people will go and buy the competitor's product. And yes, probably their 4090 are being bought by s…

> hey, I'm going to lock down your GPU so that you can only use it to render polygons in my whitelisted list of video-games

You just described gaming consoles.

Re: Fork of Facebook’s LLaMa model to run on CPU

#83

Earlier quoted context omitted.

Which is a good thing. So glad all that GPU compute is being used on cool stuff rather than running SHA-256 18 quintillion times

They can’t both be cool?

One is a contest to waste the most resources, one has potential to actually have useful results

Re: Fork of Facebook’s LLaMa model to run on CPU

#85

Earlier quoted context omitted.

Which is a good thing. So glad all that GPU compute is being used on cool stuff rather than running SHA-256 18 quintillion times

They can’t both be cool?

Nothing cool in throwing away lots of resources for no reason. In fact there's substantial heating involved.

Re: Fork of Facebook’s LLaMa model to run on CPU

#86
post #58
post #23

Earlier quoted context omitted.

What we will get are specialized hardware, with not so open APIs anyway. With a bunch of people trailling behind with "it kind of works" open alternatives.

It sounds like you are complaining about capitalism :-) It's not so bad. Nvidia could come and say, "hey, I'm going to lock down your GPU so that you can only use it to render polygons in my whitelisted list of video-games, and then you pay us $$$$$$ to buy our 'datacenter' thingy for anything else." But if they do it, people will go and buy the competitor's product. And yes, probably their 4090 are being bought by s…

As you will find on my comment history, I am perfectly fine with commercial products and APIs.

Good luck with AMD.

Re: Fork of Facebook’s LLaMa model to run on CPU

#88
post #55

Earlier quoted context omitted.

Why is it a bad start? It could be venv as well, I suppose, I haven't used conda.

Conda is gonna work much, much, much better for these kinds of applications, as that's what it's mostly used for, i.e. scientific/numerical computing with C/C++ dependencies.

Conda is an abomination that will download 4Gig of unnecessary shite and carelessly dump it into your system, thereby ruining your existing configuration in the process.

Use it in a container or a VM unless you enjoy re-installing your system from scratch.

Or better still, don't use it al all and let it wither away: these kind of braindead projects need to be put down with extreme prejudice.

Re: Fork of Facebook’s LLaMa model to run on CPU

#89
post #74
post #21

Earlier quoted context omitted.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

For most part of the 20th, a bulk of the energy humanity was able to extract was used for industrialization. Now it seems that a vast bulk of the energy being extracted will go towards computation.

I doubt it frankly. Computation consumes a lot of energy, true, but it is dwarfed by how much energy we use in transportation and food production. Energy use per capita in most of the Global North is about 75,000kWh per year,

https://ourworldindata.org/grapher/per-capita-energy-use

That's like the average person running 27 NVIDIA A100s at max capacity at all times!

Post reply on HN