ITT nobody remembers gpt2 anymore and that makes me sad
Gemma 3 270M: Compact model for hyper-efficient AI
261–270 of 325 posts
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#262ITT nobody remembers gpt2 anymore and that makes me sad
This model was trained on 6T tokens and has 256k embeddings, quite different than a gpt2 model comparable in size.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#263Earlier quoted context omitted.
The Gemma 3 models are great! One of the few models that can write Norwegian decently, and the instruction following is in my opinion good for most cases. I do however have some issues that might be related to censorship that I hope will be fixed if there is ever a Gemma 4. Maybe you have some insight into why this is happening? I run a game when players can post messages, it's a game where players can kill each othe…
The magic word you want to look up here is "LLM abliteration", it's the concept of where you can remove, attenuate or manipulate the refusal "direction" of a model. You don't need datacenter anything for it, you can run it on an average desktop. There's plenty of code examples for it. You can decide if you want to bake it into the model or apply it as a toggled switch applied at processing time and you can Distil oth…
In my limited experience, those abliterated models on Ollama didn't work very well. Still refused most things.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#264Maybe I'm using it wrong, but when I try to use the full precision FP16 model, load it into chatter UI and ask a simple question, "write me a template to make a cold call to a potential lead", It throws me absolute rubbish. On the other hand, Qwen 0.6B Q8 quantized model nails the answer for the same question. Qwen 0.6B is smaller than gemma full precision. The execution is a tad slow but not by much. I'm not sure wh…
As many repeated here, it's (generally) not for direct use. It is meant to be a good base for fine-tuning and getting something very fast.
(In theory, if you fine-tuned Gemma3:270M over "templating cold calls to leads" it would become better than Qwen and faster.)
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#265Earlier quoted context omitted.
The finetune would be an LLM where you say something like "my colors on the screen look to dark" and then it points you to Displays -> Brightness. It feels like a relatively constrained problem like finding the system setting that solves your problem is a good fit for a tiny LLM.
This would be a great experiment. I'm not sure how the OS integration would work, but as a first pass you could try finetuning the model to take natural language "my colors on the screen look to dark" and then have it output "Displays -> Brightness", then expand to the various other paths you would like the model to understand
Maybe using a larger model to generate synthetic data of question path Combos, and also to rephrase and generate similar type questions for a more varier training set.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#266My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…
This is why we should have a downvote button on HN.
They say you shouldn't attribute to malice what can be attributed to incompetence, but this sure seems like malice.
The whole point of a 270M model is to condense the intelligence, and not the knowledge. Of course it doesn't fare well on a quiz.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#267My lovely interaction with the 270M-F16 model: > what's second tallest mountain on earth? The second tallest mountain on Earth is Mount Everest. > what's the tallest mountain on earth? The tallest mountain on Earth is Mount Everest. > whats the second tallest mountain? The second tallest mountain in the world is Mount Everest. > whats the third tallest mountain? The third tallest mountain in the world is Mount Everes…
> These smaller models are great, but they really feel like talking to a toddler sometimes!
You're using the toddler and the model wrong. I love talking to my toddler, probably more valuable conversations than I've had with any other person. But it's not the same use case as asking a professor a question in their field
| Gemma 3 270M embodies this "right tool for the job" philosophy. It's a high-quality foundation model that follows instructions well out of the box, and its true power is unlocked through fine-tuning. Once specialized, it can execute tasks like text classification and data extraction with remarkable accuracy, speed, and cost-effectiveness.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#268It is so cool when they mention that you can deploy it on your own machine, they actually gave a link to the `localhost` and said `your own local environment`!
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#269This model is a LOT of fun. It's absolutely tiny - just a 241MB download - and screamingly fast, and hallucinates wildly about almost everything. Here's one of dozens of results I got for "Generate an SVG of a pelican riding a bicycle". For this one it decided to write a poem: +-----------------------+ | Pelican Riding Bike | +-----------------------+ | This is the cat! | | He's got big wings and a happy tail. | | He…
I audibly laughed at this one: https://gist.github.com/simonw/25e7b7afd6a63a2f15db48b3a51ec... where it generates a… poem? Song? And then proceeds to explain how each line contributes to the SVG, concluding with: > This SVG code provides a clear and visually appealing representation of a pelican riding a bicycle in a scenic landscape.
This reminds me of my interactions lately with ChatGPT where I gave into its repeated offer to draw me an electronics diagram. The result was absolute garbage. During the subsequent conversation it kept offering to include any new insights into the diagram, entirely oblivious to its own incompetence.
Re: Gemma 3 270M: Compact model for hyper-efficient AI
#270Earlier quoted context omitted.
I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"
Safety in the context of LLMs means “avoiding bad media coverage or reputation damage for the parent company” It has only a tangential relationship with end user safety. If some of these companies are successful the way they imagine, most of their end users will be unemployed. When they talk about safety, it’s the companies safety they’re referring to.
Investor safety. It's amazing that people in hn threads still think the end-user is the customer. No. The investor is the customer, and the problem being solved for that curtomer is always how to enrich them.