101–110 of 273 posts
I have no desire to run an LLM on my laptop when I can run one on a computer the size of six football fields.
I was in the market for a laptop this month. Many new laptops now advertise AI features like this "HP OmniBook 5 Next Gen AI PC" which advertises: "SNAPDRAGON X PLUS PROCESSOR - Achieve more everyday with responsive performance for seamless multitasking with AI tools that enhance productivity and connectivity while providing long battery life" I don't want this garbage on my laptop, especially when its running of its…
But yeah, fresh install of OS is a must for any new computer.
I feel like there's no point to get a graphics card nowadays. Clearly, graphics cards are optimized for graphics; they just happened to be good for AI but based on the increased significance of AI, I'd be surprised if we don't get more specialized chips and specialized machines just for LLMs. One for LLMs, a different one for stable diffusion. With graphics processing, you need a lot of bandwidth to get stuff in and…
Nah, that's just plain wrong. First, GPGPU is powerful and flexible. You can make an "AI-specific accelerator", but it wouldn't be much simpler or much more power-efficient - while being a lot less flexible. And since you need to run traditional graphics and AI workloads both in consumer hardware? It makes sense to run both on the same hardware. And bandwidth? GPUs are notorious for not being bandwidth starved. 4K@60…
Earlier quoted context omitted.
I was mussing this summer if I should get a refurbed Thinkpad P16 with 96GB of RAM to run VMs purely in memory. Now that 96GB of ram cost as much as a second P16.
I feel you, so much. I was thinking of getting a second 64gb node for my homelab and i thought i’d save those money… now the ram alone cost as much as the node, and I’m crying. Lesson learned: you should always listen to that voice inside your head that say: “but i need it…” lol
Earlier quoted context omitted.
LLMs are enormously bandwidth hungry. You have to shuffle your 800GB neural network in and out of memory for every token, which can take more time/energy than actually doing the matrix multiplies. GPUs are almost not high bandwidth enough.
But even so, for a single user, the output rate for a very fast LLM would be like 100 tokens per second. With graphics, we're talking like 2 million pixels, 60 times a second; 120 million pixels per second for a standard high res screen. Big difference between 100 tokens vs 120 million pixels. 24 bit pixels gives 16 million possible colors... For tokens, it's probably enough to represent every word of the entire voca…
"How many TOPS do you need to run state-of-the-art models with hundreds of millions of parameters? No one knows exactly." What's he talking about? It's trivial to calculate that.
I have no desire to run an LLM on my laptop when I can run one on a computer the size of six football fields.
The point is that when you run it on your own hardware you can feed the model your health data, bank statements and private journals and can be 5000% sure they’re not going anywhere
Earlier quoted context omitted.
>> I'd really rather my devices only do what I ask them to Linux hears your cry. You have a choice. Make it.
Unfortunately still loads of hurdles for most people. AAA Games with anti-cheat that don't support Linux. Video editing (DaVinci Resolve exists but is a pain to get up and running on many distros, KDenLive/OpenShot don't really cut it for most) Adobe Suite (Photoshop/Lightroom specifically, and Premiere for Video Editing) - would like to see Affinity support Linux but hasn't happened so far. GIMP and DarkTable aren't…
Gimp isn't a solution, sure but it works for what I need. Darktable does way more than I've ever wanted, so I can forgive it for the one time it crashed. Inkscape and blender both exceed my needs as well.
And Adobe is so user hostile, that I feel I need to call you a mean name to prove how I feel.... dummy!
Yes, I already feel bad, and I'm sorry. But trolling aside, listing applications that treat users like shit, aren't reasons to stay on the platform that also treats you like shit.
I get it, sometimes, being treated like shit is worth it because it's easier now that you're used to being disrespected. But an aversion to the effort it'd take for you to climb the learning curve of something different, isn't valid reason to help the disrespectful trash companies making the world worse, recruit more people for them to treat like trash.
Just because you use it, doesn't make it worth recommending.
Earlier quoted context omitted.
Unfortunately still loads of hurdles for most people. AAA Games with anti-cheat that don't support Linux. Video editing (DaVinci Resolve exists but is a pain to get up and running on many distros, KDenLive/OpenShot don't really cut it for most) Adobe Suite (Photoshop/Lightroom specifically, and Premiere for Video Editing) - would like to see Affinity support Linux but hasn't happened so far. GIMP and DarkTable aren't…
According to my friends, Arc Raders works well on linux. So it's very much, just a small selection of AAA games, so they can run anti-cheat, that probably doesn't even work. Can you name a triple a you want to play, that proton says is incompatible? Gimp isn't a solution, sure but it works for what I need. Darktable does way more than I've ever wanted, so I can forgive it for the one time it crashed. Inkscape and ble…
I know Adobe are... c-words, but their software is industry standard for a reason.