161–170 of 178 posts
It's useless before the model gets instruction and preference tunings. Won't even follow a simple ask, it will just assume it is a list of questions and generate more, or continue with slightly related comments. FB trained a LLaMA-I (instruction tuned) variant for sports, just to show they can, but I don't think it got released.
Earlier quoted context omitted.
that's literally all I did (plus switching the tensor type). I'd imagine people are posting and upvoting this not because it's actually interesting code but rather just because it runs unexpectedly fast on consumer CPUs and it's not something they considered feasible before.
How are you getting this to run fast? I'm on a top of the line M1 MBP and getting 1 token every 8 minutes.
Earlier quoted context omitted.
ahhh the units in this post are making my eye twitch.
I know Watts per hour is not the right way to phrase it but I feel it helps for those that don't know. Also, I just don't like saying Amp. Hours :)
Earlier quoted context omitted.
Agree. I work in robotics and we never have enough compute. I want to see us get to the point where the most advanced robot ever has all the compute it needs onboard, and that means huge growth in compute density and efficiency are needed.
That's genuinely surprising. What sort of on-board compute do you typically have today?
There is another reply to your comment that shares a lot of what I have experienced. You have so many pieces of code that need to run and a good handful of them are working on something like LIDAR point clouds with a million 3D points in them, plus some cameras running several different image recognition and segmentation algorithms, and you want to have fast cycle times, it just all adds up. Every serious robot I have ever worked on is maxing out its system, even ones at Google X with a full desktop CPU, a high end NVIDIA graphics card, and a couple secondary ARM CPUs.
The thing I like the most about the current AI wave is the pressure is putting on computing hardware. Yes, mobile phones with long battery lives are cool and all of that, but most cool things I like are locked behind huge computational requirements.
Crazy to me that as soon as one GPU wave is dying (crypto), another one is picking up slack.
Earlier quoted context omitted.
I know Watts per hour is not the right way to phrase it but I feel it helps for those that don't know. Also, I just don't like saying Amp. Hours :)
Watts per hour implies watts/hour. Watt-hour implies a number of watts multiplied by a length of time. Also known as energy. Watts are power. Watt hours are energy. Two different things. Watts/hour is nothing.
Earlier quoted context omitted.
> running SHA-256 18 quintillion times or games. People could have been studying or doing something more important than wasting time and energy. I get that it is entertainment, but so are board games and that don't require mining rare earth minerals or putting pressure on the grid as you can always play board games with candles on.
Monopoly by candlelight, just the future I had always envisioned.