271–280 of 377 posts
Which got me thinking about how do these frontier AI models work when you (as a user) run a query. Does your query just go to one big box with lots of GPUs attached and it runs in a similar way, but much faster? Do these AI companies write about how their infra works?
Earlier quoted context omitted.
What do you mean by Cooler? Raspberry pi doesn't need a fan. Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups. You can use a pi without a PSU for example. And if you use an SSD, you have to consider that cost in whatever you compare it to. >I went this route myself and have figuratively and literally shelved a bunch of Pis >and I have dozens of them, Reread m…
> What do you mean by Cooler? Raspberry pi doesn't need a fan. From the official website: > Does Raspberry Pi 5 need active cooling? > Raspberry Pi 5 is faster and more powerful than prior-generation Raspberry Pis, and like most general-purpose computers, it will perform best with active cooling.
If Pi Clusters were actually cost competitive for performance there would be data centres full of them.
For anyone interested in playing with distributed systems, I'd really recommend getting a single machine with latest 16-core CPU from AMD and just running 8 virtual machines on it. 8 virtual machines, with 4 hyper threads pinned per machine, and 1/8 of total RAM per machine. Create a network between them virtually within your virtualization software of choice (such as Proxmox). And suddenly you can start playing with…
Earlier quoted context omitted.
> the raspberry pi is still an amazingly cost effective choice It's really not though. I've been a Pi user and fan since it was first announced, and I have dozens of them, so I'm not hating on RPi here; we did the maths some time back here on HN when something else Pi related came up. If you go for a Pi5 with say 8GB RAM, by the time you factor in an SSD + HAT + PSU + Case + Cooler (+ maybe a uSD), you're actually al…
What do you mean by Cooler? Raspberry pi doesn't need a fan. Also the other peripherals you consider are irrelevant, since you would need them (or not), in other setups. You can use a pi without a PSU for example. And if you use an SSD, you have to consider that cost in whatever you compare it to. >I went this route myself and have figuratively and literally shelved a bunch of Pis >and I have dozens of them, Reread m…
Starting with the Pi 4, they started saying that a cooler isn't required, but that it may thermal throttle without one if you keep the CPU pegged.
For anyone interested in playing with distributed systems, I'd really recommend getting a single machine with latest 16-core CPU from AMD and just running 8 virtual machines on it. 8 virtual machines, with 4 hyper threads pinned per machine, and 1/8 of total RAM per machine. Create a network between them virtually within your virtualization software of choice (such as Proxmox). And suddenly you can start playing with…
I run a K8s "cluster" on a single xcp-ng instance, but you don't even really have to go that far. Docker Machine could easily spin up docker hosts with a single command, but I see that project is dead now. Docker Swarm I think still lets you scale up/down services, no hypervisor required.
I think the biggest problem with cluster products is that they just don't work out of the box. Vendors haven't really done the "last 2%" of development required to make them viable - its left to us purchasers to get the final bits in place.
Still, it'll make a fun distributed computing experimental platform some day.
Just like the Inmos Transputer I've got somewhere, sitting in a box, waiting for a power supply ..
I believe the Rasp Pi cluster is one of the cheapest multi node / MPI machines you can buy. That's useful even if it is t fast. You need to practice the programming interfaces, not necessarily make a fast computer.
However, NUMA is also a big deal. The various AMD Threadrippers with multi-die memory controllers are better on this regards. Maybe the aging Threadrippers 1950x, yes it's much slower than modern chips but the NUMA issues are exaggerated (especially poor) on this old architecture.
That exaggerates the effects of good NUMA and now you as a programmer can get more NUMA skills.
Of course, the best plan is to spend $20,000,000++ on your own custom NUMA nodes cluster out of EPYCs or something.
-------
But no. The best supercomputers are your local supercomputers that you should rent some time from. You need a local box to see various issues and learn to practice programming.
If Pi Clusters were actually cost competitive for performance there would be data centres full of them.
Mythic Beasts rents rpi servers: https://www.mythic-beasts.com/order/rpi/ - there is a niche for it
Earlier quoted context omitted.
It's good for the soul to have your cluster running in your home somewhere.
Bad for your power bill though.