I'm curious about the multimodal capabilities on the E2B and E4B and how fast is it. In ChatGPT right now, you can have a audio and video feed for the AI, and then the AI can respond in real-time. Now I wonder if the E2B or the E4B is capable enough for this and fast enough to be run on an iPhone. Basically replicating that experience, but all the computations (STT, LLM, and TTS) are done locally on the phone. I just…
Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can
What’s it like to work on the frontier of AI model creation? What do you do in your typical day? I’ve been really enjoying using frontier LLMs in my work, but really have no idea what goes into making one.
You have to ask Anthropic and OpenAI, not Google. They are still way behind.
Not exactly what you asked for but try https://pelicans.borg.games/
what the sorcery is that https://static.simonwillison.net/static/2024/recraft-ai-peli... I tried their model and asking a few different svg of pelicans. it is INSANE.
It's generating a raster image and then using a tracing algorithm to turn it into an SVG, rather than generating an SVG via coding from the ground-up. If you put a raster image into Adobe Illustrator and use its image tracing you'll get a similar result.
Wild differences in ELO compared to tfa's graph: https://storage.googleapis.com/gdm-deepmind-com-prod-public/... (Comparing Q3.5-27B to G4 26B A4B and G4 31B specifically) I'd assume Q3.5-35B-A3B would performe worse than the Q3.5 deep 27B model, but the cards you pasted above, somehow show that for ELO and TAU2 it's the other way around... Very impressed by unsloth's team releasing the GGUF so quickly, if that's lik…
You're conflating lmarena ELO scores. Qwen actually has a higher ELO there. The top Pareto frontier open models are: model |elo |price qwen3.5-397b-a17b |1449 |$1.85 glm-4.7 |1443 | 1.41 deepseek-v3.2-exp-thinking |1425 | 0.38 deepseek-v3.2 |1424 | 0.35 mimo-v2-flash (non-thinking) |1393 | 0.24 gemma-3-27b-it |1365 | 0.14 gemma-3-12b-it |1341 | 0.11 gpt-oss-20b |1318 | 0.09 gemma-3n-e4b-it |1318 | 0.03 https://arena.…
Gemma 4 31B has now wiped out several of those models from the pareto frontier, now that it has pricing. Gemma 4 26B A4B has an Elo, but no pricing, so it still isn't on that chart. The Gemma 4 E2B/E4B models still aren't on the arena at all, but I expect them to move the pareto frontier as well if they're ever added, based on how well they've performed in general.
In terms of ability, maybe, in terms of speed, it's not even close. Check out the Prompt Processing speeds between them: https://kyuz0.github.io/amd-strix-halo-toolboxes/ gpt-oss-120b is over 600 tokens/s PP for all but one backend. nemotron-3-super is at best 260 tokens/s PP. Comparing token generation, it's again like 50 tokens/sec vs 15 tokens/sec That really bogs down agentic tooling. Something needs to be catego…
In my case with vLLM on dual RTX Pro 6000 gpt-oss-120b: (unknown prefill), ~175 tok/s generation. I don't remember the prefill speed but it certainly was below 10k Nemotron-3-Super: 14070 tok/s prefill, ~194.5 tok/s generation. (Tested fresh after reload, no caching, I have a screenshot.) Nemotron-3-Super using NVFP4 and speculative decoding via MTP 5 tokens at a time as mentioned in Nvidia cookbook: https://docs.nvi…
Hmm you might be able to tweak the settings further. Under llama.cpp on one RTX 6000 Pro I get ~215 tok/s generation speed. The key for me was setting min_p greater than 0. My settings:
Hi all! I work on the Gemma team, one of many as this one was a bigger effort given it was a mainline release. Happy to answer whatever questions I can
Could you recommend which quantization level to use with it?