Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

github.com

Post reply on HN
Print view View original

21–30 of 33 posts

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#21

Post by MediaSquirrel »

weitendorf wrote: post #15
MediaSquirrel wrote: post #13

Earlier quoted context omitted.

Great minds think alike! Also, I had a huge head start, as I spent a month or two working on this in September 2025, shelved it and dusted it back off this weekend.

Excellent work still, your repo is much more robust and fleshed out and I am just beelining straight to audio LoRa not really knowing what I'm doing, as this is my first time attempting a ~real ML training project. I think in https://github.com/mattmireles/gemma-tuner-multimodal/blob/m... and https://github.com/mattmireles/gemma-tuner-multimodal/blob/m... and https://github.com/mattmireles/gemma-tuner-multimodal/blob…

Look inside here: https://github.com/mattmireles/gemma-tuner-multimodal/tree/m...

Here’s the trick: use Gemini Pro deep research to create “Advanced Hacker’s Field Guide for X” where X is the problem that you are trying to solve. Ask for all the known issues, common bugs, unintuitive patterns, etc. Get very detailed if you want.

Then feed that to Claude / Codex / Cursor. Basically, create a cheat sheet for your AI agents.

This will unlock a whole new level of capability.

I’m @mattmireles on Twitter — feel free to DM me.

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#22

Post by nomel »

MediaSquirrel wrote: post #19
mandeepj wrote: post #17

> I had 15,000 hours of audio data do you really need that much data for fine-tuning?

More data -> better, faster on-device models The actual plan was to distill Gemini 2.5 Pro into the best on-device voice dictation model. Pretty sure it would have worked. Alas.

Reasons for running local aside...

What is the practical latency difference you see between on-device and, say, whisper, in streaming mode, over the internet? Comparable? Seems that internet latency would be mostly negligible (assuming reasonable internet/cell coverage), or at least compensated for by the higher end hardware on the other side?

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#23

Post by MediaSquirrel »

nomel wrote: post #22
MediaSquirrel wrote: post #19

Earlier quoted context omitted.

More data -> better, faster on-device models The actual plan was to distill Gemini 2.5 Pro into the best on-device voice dictation model. Pretty sure it would have worked. Alas.

Reasons for running local aside... What is the practical latency difference you see between on-device and, say, whisper, in streaming mode, over the internet? Comparable? Seems that internet latency would be mostly negligible (assuming reasonable internet/cell coverage), or at least compensated for by the higher end hardware on the other side?

depends on the model!

If you run a smaller whisper-distil variant AND you optimize the decoder to run on Apple Neural Engine, you can get latency down to ~300ms without any backend infra.

The issue is that the smaller models tend to suck, which is why the fine-tuning is valuable.

My hypothesis is that you can distill a giant model like Gemini into a tiny distilled whisper model.

but it depends on the machina you are running, which is why local AI is a PITA.

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#24

Post by MediaSquirrel »

neonstatic wrote: post #16

Just a heads up, that I found NVIDIA Parakeet to be way better than Whisper - faster, uses less compute, the output is better, and there are more options for the output. I am using parakeet-mlx from the command line. Check it out!

yeah, it came out after I stared on my project last year. Only issue is that you can't fine-tune it on Apple Silicon.

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#26

Post by sipjca »

MediaSquirrel wrote: post #14
LuxBennu wrote: post #4

I run whisper large-v3 on an m2 max 96gb and even with just inference the memory gets tight on longer audio, can only imagine what fine-tuning looks like. Does the 64gb vs 96gb make a meaningful difference for gemma 4 fine-tuning or does it just push the oom wall back a bit? Been wanting to try local fine-tuning on apple silicon but the tooling gap has kept me on inference only so far.

re: Whisper v3 -- how is this possible? Whisper has a 30s context window. You have to chunk it.

Wondering similar. It certainly can run beyond 30 seconds but at some point I believe the output should degrade

Plus you could do actual batch inference instead. Or if you must carry forward the context you could still do it linearly, but the mem usage shouldn’t just explode

Re: Show HN: Gemma 4 Multimodal Fine-Tuner for Apple Silicon

#27

Post by jwr »

neonstatic wrote: post #16

Just a heads up, that I found NVIDIA Parakeet to be way better than Whisper - faster, uses less compute, the output is better, and there are more options for the output. I am using parakeet-mlx from the command line. Check it out!

I've been trying both Whisper v3 large and Parakeet in MacWhisper, and I inevitably go back to Whisper large. Which one is better depends on what you dictate, how you speak, and which languages you use.
Post reply on HN