Is "OpenAI" the only AI company that hasn't released any model weights?
They did release Whisper which to be fair has been incredibly helpful for a few of my projects.
A very long time ago in AI time
Gemma3 – The current strongest model that fits on a single GPU
101–110 of 148 posts
Re: Gemma3 – The current strongest model that fits on a single GPU
#102[deleted]
Re: Gemma3 – The current strongest model that fits on a single GPU
#103Earlier quoted context omitted.
They need to open source Sonnet 3.7. I know they won't, but a man can dream.
I'd wish people stop using "open sourcing" when speaking about models. Open sourcing is about being able to change and replicate builds, they make the models "freely available" but the recipe on how they are made is kept secret. It's akin to being able to download Windows shareware executables and calling that "open source" when nothing related to how the executables are build is available.
Even if they did release the code, that would not help you much unless you have the $ for traning and the talented individuals for pipelining and distributed training.
Re: Gemma3 – The current strongest model that fits on a single GPU
#104Earlier quoted context omitted.
Thee are a lot of open source reasoning models. The true value to local models is privacy and the ability to have the models be uncensored.
OSS model do not have to be local models, and it's not just about privacy, imo. DeepSeek R1 hosting is out of reach for most, but it being open is a game changer if you are a building a business that needs the SoTA capabilities of such a large model, not because you will necessarily host it yourself, but because you can't be locked out of using it. If you build your business on top of OpenAI, and they decide they don…
Another benefit is they can be fine tuned. Also it's not only about if Openai will shut you down but decide to deprecate model (like they will do for gpt4.0) or swap the name for different model (like sonnet 3.5 did) or censure it or limit capability.
Re: Gemma3 – The current strongest model that fits on a single GPU
#105My usual non-scientific benchmark is asking it to implement the game Tetris in python, and then iterating with the LLM to fix/tweak it. My prompt to Gemma 27b (q4) on open webui + ollama: "Can you create the game tetris in python?" It immediately starts writing code. After the code is finished, I noticed something very strange, it starts a paragraph like this: " Key improvements and explanations: Clearer Code Structu…
Why did this get downvoted? Asking genuinely
Re: Gemma3 – The current strongest model that fits on a single GPU
#106Re: Gemma3 – The current strongest model that fits on a single GPU
#107I have tried a lot of local models. I have 656GB of them on my computer so I have experience with a diverse array of LLMs. Gemma has been nothing to write home about and has been disappointing every single time I have used it. Models that are worth writing home about are; EXAONE-3.5-7.8B-Instruct - It was excellent at taking podcast transcriptions and generating show notes and summaries. Rocinante-12B-v2i - Fun for s…
How are you grading these? Are you going on feeling, or do you have a formalized benchmarking process?
Re: Gemma3 – The current strongest model that fits on a single GPU
#108Re: Gemma3 – The current strongest model that fits on a single GPU
#109Earlier quoted context omitted.
If you actually looked at gemma-3 you’ll see that it does support system prompts. I’ve never seen a case where putting the system prompt in the user prompt would lead to significantly different outcomes though. Would like to see some examples. (edit: my bad. i stand corrected. it seems the code just prepends the system prompts to the first user prompt.)
I've read the Gemma 3 technical report, it doesn't mention anything about it in the format section. Did they forget to include that? Where did you find the source that claims otherwise?
Google AI Studio offers no system prompt option for Gemma 3 which also suggests this model doesn't support it...
Re: Gemma3 – The current strongest model that fits on a single GPU
#110Earlier quoted context omitted.
If you actually looked at gemma-3 you’ll see that it does support system prompts. I’ve never seen a case where putting the system prompt in the user prompt would lead to significantly different outcomes though. Would like to see some examples. (edit: my bad. i stand corrected. it seems the code just prepends the system prompts to the first user prompt.)
I've read the Gemma 3 technical report, it doesn't mention anything about it in the format section. Did they forget to include that? Where did you find the source that claims otherwise?
My bad, original comment updated.