21–30 of 126 posts
I wanted to speak with businesses in my local area but no one took me up on it.
I'm a huge fan of OpenRouter and their interface for solid LLM's but I recently jumped into fine tuning / modifying my own vision models for FPV drone detection (just for fun) and my daily workstation and it's 2080 just wasn't good enough. Even in 2025 it's cool how solid a setup dual 3090's still are. nvlink is an absolute must but it's incredibly powerful. I'm able to run the latest Mistral thinking models and rela…
I am exploring options just for fun. a used 3090 is around $900 on ebay. a used rtx 6000 ADA is around $5k 4 3090s are slower at inference and worse at training than 1 rtx 6000. 4x3090 would consume 1400W at load. Rtx 6000 would consume 300W at load. If you god forbid live in California and your power averages 45 cents per kwh, 4x3090 would be $1500+ more per year to operate than a single RTX 6000[0] [0] Back of the…
I'm a huge fan of OpenRouter and their interface for solid LLM's but I recently jumped into fine tuning / modifying my own vision models for FPV drone detection (just for fun) and my daily workstation and it's 2080 just wasn't good enough. Even in 2025 it's cool how solid a setup dual 3090's still are. nvlink is an absolute must but it's incredibly powerful. I'm able to run the latest Mistral thinking models and rela…
I am exploring options just for fun. a used 3090 is around $900 on ebay. a used rtx 6000 ADA is around $5k 4 3090s are slower at inference and worse at training than 1 rtx 6000. 4x3090 would consume 1400W at load. Rtx 6000 would consume 300W at load. If you god forbid live in California and your power averages 45 cents per kwh, 4x3090 would be $1500+ more per year to operate than a single RTX 6000[0] [0] Back of the…
I built a dual 3090 rig, and this point was why I spent a long time looking for a case where the GPU's could fit side by side with a little gap for airflow
I eventually went with a SilverStone GD11 HTPC which is a PC case for building a media centre, but it's huge inside, has a front fan that takes up 75% of width of the case and also allows the GPUs to stand up right so they don't sag and pull on their thin metal supports.
Highly recommend for a dual GPU build! If you can get dual 5090s instead of 3090s (good luck!) you'd even be able to get "good" airflow in this case.
I’m really interested in this space from an AI sovereignty pov. Is it feasible for SMB/SME to use a box like in the article to get offline analysis of their data? It doesn’t have the worry of sending it off to the cloud. I wanted to speak with businesses in my local area but no one took me up on it.
I'm a huge fan of OpenRouter and their interface for solid LLM's but I recently jumped into fine tuning / modifying my own vision models for FPV drone detection (just for fun) and my daily workstation and it's 2080 just wasn't good enough. Even in 2025 it's cool how solid a setup dual 3090's still are. nvlink is an absolute must but it's incredibly powerful. I'm able to run the latest Mistral thinking models and rela…
Tim Dettmers amazing GPU blog post posits NVLink doesn't start to become useful until you are at 128+ GPUs
https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
I built pretty much this exact rig myself, but now it's gathering dust, any other uses for this rather than localLLMS
1) [I thought] The page is blocking cut & paste. Super annoying!
2) The exact mainboard is not specified exactly. There are 4 different boards called "ASUS ROG Strix X670E Gaming" and some of them only have one PCIe x16 slot. None of them can do PCIe x8 when using two GPUs.
3) The shopping link for the mainboard leads to the "ASUS ROG Strix X670E-E Gaming" model. This model can use the 2nd PCIe 5.0 port at only x4 speeds. The RTX 3090 can only do PCIe 4.0 of course so it will run at PCIe 4.0 x4. If you choose a desktop mainboard for having two GPUs, make sure it can run at PCIe x8 speeds when using both GPU slots! Having NVLink between the GPUs is not a replacement for having a fast connection between the CPU+RAM and the GPU and its VRAM.
4) Despite having a last-modified date of September 22nd, he is using his rig mostly with rather outdated or small LLMs and his benchmarks do not mention their quantization, which makes them useless. Also they seem not to be benchmarks at all, but "estimates". Perhaps the headline should be changed to reflect this?
I built pretty much this exact rig myself, but now it's gathering dust, any other uses for this rather than localLLMS