341–350 of 372 posts
Earlier quoted context omitted.
They’re kickstarting a TINY device that is pocketable and aimed at consumers. I’ve backed it (full disclosure).
https://www.kickstarter.com/projects/tiinyai/tiiny-ai-pocket...
Earlier quoted context omitted.
They’re kickstarting a TINY device that is pocketable and aimed at consumers. I’ve backed it (full disclosure).
https://www.kickstarter.com/projects/tiinyai/tiiny-ai-pocket...
The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.
Would a hedge fund that does not want to trust to a public AI cloud just buy chassis, mobos, GPUs, etc, and build an equivalent themselves? I suspect they value their time differently.
There's no way the red v2 is doing anything with a 120b parameter model. I just finished building a dual a100 ai homelab (80gb vram combined with nvlink). Similar stats otherwise. 120b only fits with very heavy quantization, enough to make the model schizophrenic in my experience. And there's no room for kv, so you'll OOM around 4k of context. I'm running a 70b model now that's okay, but it's still fairly tight. And…
Earlier quoted context omitted.
yes, this is accurate for US and “works” but it’s against code here. you’ll get mildly shocked by metallic cabinets and fixtures especially if you’re barefoot and become the new shortest path to ground. old construction in the US sometimes did this intentionally (so old, the house didn’t have grounds. Or to “pass” an inspection and sell a place) but if a licensed electrician sees this they have to fix it. I’m dealing…
> I’m dealing with a 75 year old house that’s set up this way I can’t tell what practice you’re referring to. Are you perhaps referring to older wiring that connects large appliances to a neutral and two hots but no ground, e.g. NEMA 10-30R receptacles? Those indeed suck and are rather dangerous. Extra dangerous if the neutral wiring is failing or undersized anywhere. But even NEMA 10-30R receptacles are still 120V R…
I believe that’s kinda naughty.
It works, but it energizes your ground plane and people do get mildly shocked. that’s making me a little nervous.
So holes have been drilled in ceilings and walls and single wire neutrals or grounds have been fished down the walls, repeatedly, by yours truly , but there’s still at least one “gfci” outlet that’s wired this way And they’re balking at getting an electrician back out here for.
bridging neutral to ground because the neutral lines dead, uh, “works” to be technical but whoever did this moved on years ago and heaven only knows how many outlets or fixtures this was done in. I’m just finding out one by one as someone goes “hey this stopped working!”and you pull it and the neutral or ground blew like a fuse.
So that’s my whole point, this is an extremely bad idea for a 3200watt computer.
yes, they are all getting snipped and blank wall plated and marked as hazards that need to be remediated with a Dymo labeler as I discover them.
I don’t work here I just live here and have kind of a slummy owner who doesn’t want to do anything about any of it and doesn’t care if the plumbing or electrical works.
But they paid some guy like $4000 to install a totally unnecessary subpanel that’s bridging conflicting phases into the same circuits because he didn’t figure out this was what was going on. Dios Mio. I would have fixed the whole house for $1000. Miracle this hovel hasn’t burned to the ground yet.
I’m putting up with it for now but should probably bail before it does.
Earlier quoted context omitted.
At $7.2k + tax: * RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM, 5600MHz CP2K64G56C46U5, up to 4 sticks for 128GB or 256GB, Amazon * GPU - $4700 - RTX Pro 5000 48GB, Microcenter * CPU/Mobo bundle - $1100 - AMD Ryzen 7 9800X3D, MSI X870E-P Pro, ditch the 32GB RAM, Microcenter * Case - $220, Hyte Y70, Microcenter * Cooler - $155, Arctic Cooling Liquid Freezer III Pro, top-mount it, Microcenter * PSU - $180, RM1…
$7.2k just to run at best Qwen3.5-35B-A3B doesn't seem worth it at all. This is certainly not the most effective use of $7k for running local LLMs. The answer is a 16" M5 Max 128GB for $5k. You can run much bigger models than your setup while being an awesome portable machine for everything else.
In terms of GPU memory bandwidth (models fitting in the ~48GB of RTX 5000 Pro card), the RTX card I described above has over 2x the bandwidth of an M5 Max.
If leveraging system RAM (the 128GB-256GB outside the GPU) to run larger models, then the memory bandwidth is ~6x slower than M5 Max.
For models fitting in the ~48GB RTX memory, like dense Qwen3.5 27B models, the RTX will be 2-4x faster than M5 Max. For models that don't fit in the 48GB RTX memory, the M5 Max will be 5-20x faster.
Also worth considering future upgrades: Do you plan to throw away the machine in a few years, or pick up multiple used RTX 6000 Pro cards when people start ditching them?
Earlier quoted context omitted.
It doesn’t read as if they actually care about broad appeal, given their plain refusal to accommodate traditional procurement processes
So they're only interested in taking on customers who are OK with being treated poorly?
There’s nothing remotely unusual about being selective about who or how to bring on new customer in B2B sales.
Their preference is for more simplicity than normal —- many businesses make it much harder
Earlier quoted context omitted.
I suppose if I rent a cloud GPU and just let it sit there dark and do nothing then I wouldn't have to move any data to it. Otherwise, I'm uploading some kind of work for it to do. And that usually involves some data to operate on. Even if it's just prompts.
So you also believe when you rent a server you are sharing your data with the cloud? AWS and GCP are copying all private data on servers? Give me a break. There's a big difference between renting a server and using an API.
Only if you upload your data to that cloud server you rented. Then, by definition, you are.
> AWS and GCP are copying all private data on servers?
Every computer copies data when moving it. Several times, in fact. Through network card buffers, switches, system memory, disk caches, and finally to some form of semi-permanent storage.
I don't have to think Amazon is stealing my data to be aware that Amazon S3 buckets containing privileged information are routinely found open. I don't have to think that Google is spying on me to know that operating equipment my business owns on prem and does not share requires me to trust fewer people and less complex systems than doing the same work from the cloud.
You are very quick to make foolish assumptions and assign them to others.
Earlier quoted context omitted.
Or it's two 300kW fast EV chargers working together. A typical home just consumes rather little energy, now that LED lighting and heat pump cooling / heating became the norm.
> now that LED lighting and heat pump cooling / heating became the norm. My brother in Christ, you vastly overestimate southern europe
But with heat pumps becoming normal in the North and in the US, they become mass-produced, and the prices fall. Same has happened to LED lamps.