Live data from Hacker News

Fork of Facebook’s LLaMa model to run on CPU

github.com

111–120 of 178 posts

Re: Fork of Facebook’s LLaMa model to run on CPU

#111
post #21
post #15

The thing I like the most about the current AI wave is the pressure is putting on computing hardware. Yes, mobile phones with long battery lives are cool and all of that, but most cool things I like are locked behind huge computational requirements.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

GPU conspiracy or just the side-effect of the decline of Intel?

Re: Fork of Facebook’s LLaMa model to run on CPU

#112
post #110

Earlier quoted context omitted.

Conda is an abomination that will download 4Gig of unnecessary shite and carelessly dump it into your system, thereby ruining your existing configuration in the process. Use it in a container or a VM unless you enjoy re-installing your system from scratch. Or better still, don't use it al all and let it wither away: these kind of braindead projects need to be put down with extreme prejudice.

The download size is large but conda doesn't ruin any existing configuration unless you explicitly tell it to be your native python environment. Conda is set up as a self-contained set of independent environments. Why would your system care what's inside the Anaconda directory unless you explicitly add it to your PATH/bash?

I haven't touched that steaming pile of shite in a looong while, so - who knows - they might have managed to minimize the amount of havoc their wreak on their user's systems.

But ... I seem to recall ... Conda tries to install GPU drivers does is not? ... Is that not the case anymore?

Because if it still does, your theory about "Why would your system care" and all that doesn't really hold water.

Re: Fork of Facebook’s LLaMa model to run on CPU

#113
post #21

Earlier quoted context omitted.

Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.

GPU conspiracy or just the side-effect of the decline of Intel?

theres an economics theory of “supply creates its own demand”. we wanted to do ml, gpus were around for games, we repurposed them for ml, and ml architectures that benefit from gpus won the “hardware lottery” (influential paper from sara hooker in case you are unaware)

Re: Fork of Facebook’s LLaMa model to run on CPU

#114
post #15

The thing I like the most about the current AI wave is the pressure is putting on computing hardware. Yes, mobile phones with long battery lives are cool and all of that, but most cool things I like are locked behind huge computational requirements.

John Hopkins are working on organoids that will replace silicon GPUs for AI.

Here is an article from JHU on that topic - https://hub.jhu.edu/2023/02/28/organoid-intelligence-biocomp...

Re: Fork of Facebook’s LLaMa model to run on CPU

#115

Earlier quoted context omitted.

Why do you think Bitcoin does not have useful results?

Can you cite an useful result? I can't but I don't think that some people getting richer is useful.

https://www.elliptic.co/blog/live-updates-millions-in-crypto...

https://www.cnbc.com/2022/03/23/ukrainian-flees-to-poland-wi...

In general, the purpose of Bitcoin is not to get rich, but to have a currency that is universally accepted and not tied to a political party’s fiscal decisions.

Re: Fork of Facebook’s LLaMa model to run on CPU

#116

Earlier quoted context omitted.

Which is a good thing. So glad all that GPU compute is being used on cool stuff rather than running SHA-256 18 quintillion times

> running SHA-256 18 quintillion times or games. People could have been studying or doing something more important than wasting time and energy. I get that it is entertainment, but so are board games and that don't require mining rare earth minerals or putting pressure on the grid as you can always play board games with candles on.

Be careful. The gaming industry has successfully conditioned people into believing they need a $1500 GPU with the TDP of a microwave so they can play the next unfinished-at-release AAA title.

Re: Fork of Facebook’s LLaMa model to run on CPU

#118
post #96

Earlier quoted context omitted.

Agree. I work in robotics and we never have enough compute. I want to see us get to the point where the most advanced robot ever has all the compute it needs onboard, and that means huge growth in compute density and efficiency are needed.

That's genuinely surprising. What sort of on-board compute do you typically have today?

I don't work in the field but just to kind of put it into perspective, a 12v 100A LiFePO4 battery has 1200 Watts capacity and weighs 30 pounds. A typical gaming PC (which to be fair, is more willing to trade power for performance) consumes about 600 Watts per hour. Problem for a Tesla? Not so much. Problem for a lightweight drone? Definitely.

Re: Fork of Facebook’s LLaMa model to run on CPU

#119

Wondering how difficult this would be to get running on a m1 max?

another commenter posted a fork that does it https://news.ycombinator.com/item?id=35067469

per the readme it looks like there a few bugs to figure out in case anyone here is a pytorch expert

Post reply on HN