251–260 of 421 posts
Last time I tried Gemma 4 (26B-A4B) its memory usage would balloon and consume all of my swap until my machine died. Qwen 3.6 on the other hand barely uses any memory at all for its KV cache.
Turns out when you block people from the best and biggest hardware, they get innovative. It reminds me of the Pentium days when everyone was shipping inefficient programs because the processor would be better next year.
What's Google's business case for releasing open models? Don't get me wrong, I am grateful and appreciative of these releases. I'm trying to understand how it fits into their bigger picture as a for profit company? Are they not helping competitors build on the novel technology they have developed? Is it simply goodwill and/or marketing? Or am I missing something strategic?
I think its even more puzzling because you can't even run Gemma 31b on google cloud, they only let you test it with a rate limit. No way (I can find) to actually pay them to use it. We saw great results in our usecase using google direct. Moved to Openrouter because google wouldn't let us use it beyond a test. Then Openrouters performance looked worse, not sure if there was a quantized version or something. So we ins…
Earlier quoted context omitted.
I've used Gemma for reviewing and categorizing my writing online over several years (~5 million words across a forum for an OSS project I work on, HN, reddit, etc.), experimenting with training LoRAs (again, on my own writing, since I don't have to worry about ethically sourcing the data if it's all mine), and I'm currently using it to perform web searches and extract data about a specific type of business. It's plen…
So one of thr things you're using it for is to generate leads to spam businesses with unwanted LLM produced marketing materials it sounds like. Wow LLMs are changing the world, what a utopia.
You don't know me. And, no.
Earlier quoted context omitted.
This is one https://post.bot/
Why would I want an AI receptionist. A human receptionist is about 1000x more careful, caring and intentional. They are charging $15.00 an hour for an llm powered assistant. Like wtf, how do these people think that's a valid business model. This will 1000% annoy every customer that uses it. I hate this timeline so much.
Can you call a receptionist at 10pm and book an appointment? Or ask for directions? What if it's 10am and she's already on the line with someone else and you just want to ask if there's parking?
Its image processing is terrible. I ran several tests against it against Qwen 3.5 0.8b (yes, 7% the size) and Qwen beat it every time with Gemma often getting things entirely wrong. I even gave it a plain image saying "This is a test" and it thought for 6 minutes trying to analyze it and failed. Qwen 3.5 0.8b confidently got it in under a second . It may be that the Q6 quant I got is borked (or my LM Studio is), but…
Earlier quoted context omitted.
Could say the same for camera processing in the Pixel Camera app or any other binary someone wants to re-use that comes included in a software distribution (seemingly for 'free'). They can't lock the instructions up on the server so they might as well make the binary be freely distributable? Companies don't commonly give away executable binaries "just because", why'd they start now for these binary blobs that are the…
Because a model like this can't be as easily obfuscated as image processing. Image processing is a bundle of many moving parts, a lot of functions each with it's own inputs and outputs. A model is a single function which can be easily extracted and reused, in comparison
With many laptops dropping back down to 8GB because of the memory shortage there's some interesting pressures building in the industry.
I ran the Q4 quant (used with llama.cpp) though my "minesweeper" vibe-coding benchmark: https://senko.net/vibecode-bench/2026/minesweeper-gamma-4-12... The result is decent, but it had a few bizzare/trivial syntax errors I had to fix manually: it would do an extra closing bracket or paren a few times, and wanted to separate function definitions with comma. Not sure what that was about, but otherwise the output run ju…
Up until this point, I've found the cost/value to unequivocally favor using a cloud subscription, but I would be lying if I didn't worry that one day OpenAI is going to increase the price for my subscription by 5-10x. I rely on these tools enough that if there is a real viable local option, I'm going to take it.
Earlier quoted context omitted.
There is demand for US open models.
I sincerely wonder why. Chinese censorship is only really relevant if you're doing anti China stuff, which is to say never, while the Western kind of model censorship ( a combination of copyrights and general fairness ) are something everyone's had to work around at least once, even if just for writing an interesting story.
Here’s a real example.
I’m in a design meeting talking about a model use case. We have a question about the data pipeline or the prompt format that would benefit from knowing about how the model was trained. The enterprise team lead calls the dev tech engineer from the company who produced the model. He is already in the office and walks into the meeting to answer the question.