Viewing profile — roosgit

I was wondering the same thing. Maybe it’s a licensing issue. Or the models are too big and there isn’t enough demand. But size is not a solid argument because they also don’t supp…

Amazon, the first quadrillion-dollar company.

Can LoRAs be used to increase the quality of these diffusion models? Nvidia mentions something about this https://huggingface.co/nvidia/Nemotron-Labs-Diffusion-8B#inf...

Yeah, it should have been "Datacenter GPUs" or "Nvidia and AMD GPUs".

I just hit that error a few minutes ago. I build my llama.cpp from source because I use CUDA on Linux. So I made the mistake of trying to run Gemma4 on an older version I had and I…

Have you tried other local models? The 14B Q4_K_M needs 9GB, but Q3_K_M is 7.3GB. But you also need some room for context. Still, maybe using `--override-tensor` in llama.cpp would…

I wasn't sure where I'd seen that "retiring" spiel before, but then I remembered someone was (still is) selling a handmade jewelry website claiming $4.3M revenue and $1.3M profit.

I use an even older Macbook and an even older macOS. Of course, the browsers no longer work with the latest JS, so occasionally when I need to use some webapp I boot up a Linux VM …

While on the subject, you can make a calendar in as little as 3 lines of CSS: https://calendartricks.com/a-calendar-in-three-lines-of-css/

Can confirm. I was trying to send the newsletter (with SES) and it didn't work. I was thinking my local boto3 was old, but I figured I should check HN just in case.

I have an RTX 3060 with 12GB VRAM. For simpler questions like "how do I change the modified date of a file in Linux", I use Qwen 14B Q4_K_M. It fits entirely in VRAM. If 14B doesn'…

# Runs the DB backup script on Thu at 22:00 -- I download the database backup for a few websites that get new data every week. I do this in case my host bans my account. # Runs the…

I've known about this issue since Lllama 1. Tried it with Llama 2 and Mistral when those models were released. LLMs are not databases. The test I ran was to ask the LLM about an ex…

I can answer question 3. Prompt processing (how fast your input is parsed) is highly correlated with computing speed. Inference (how fast the LLM answers) is highly correlated with…

Renting could be a good choice to get started. I used to rent a g4dn.xlarge instance from AWS (for Stable Diffusion, not LLMs). More affordable options are Runpod and Vast.ai. I st…

Start with r/LocalLLama and r/StableDiffusion. Look for benchmarks for various GPUs. I have an RTX 3060(12GB) and 32GB RAM. Just ran Qwen2.5-14B-Instruct-Q4_K_M.gguf in llama.cpp w…

How rich? You can get some inspiration from businesses for sale on Empire Flippers https://empireflippers.com/marketplace/ . As a rule of thumb for choosing the niche, pick from on…

I have a separate PC that I access through SSH. I recently bought a GPU for it, before that I was running it on CPU alone. - B550MH motherboard - Ryzen 3 4100 CPU - 32GB (2x16) RAM…

I've never used it, but I think Google Colab has a free plan. As another option, you can rent a machine with a decent GPU on vast.ai. An Nvidia 3090 can be rented for about $0.20/h…

I think Louie Mantia was an icon designer at Apple back then https://lmnt.me/ . Maybe Sebastiaan de With as well https://sdw.space/ .

I use it to help me write text. Don't use any tools. I run it from the command line: ./main -f ~/Desktop/prompts/multishot/llama3-few-shot-prompt-10.txt -m ~/Desktop/models/Meta-Ll…