Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone doing anything cool with tiny language models?

news.ycombinator.com

Post reply on HN
Print view View original

351–356 of 356 posts

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#351

Post by ahrjay »

fauigerzigerk wrote: post #256
ahrjay wrote: post #214

I built https://ffprompt.ryanseddon.com using the chrome ai (Gemini nano). Allows you to do ffmpeg operations on videos using natural language all client side.

What are the prerequisites for this? I keep getting "Bummer, looks like your device doesn't support Chrome AI" on macOS 15.2 Chrome 132.0.6834.84 (Official Build) (arm64) [Edit] Found it. I had to enable chrome://flags/#prompt-api-for-gemini-nano

Yeah the instructions are not clear. They're on the github repo[1] linked in the header.

1. Install Chrome Dev: Ensure you have version 127. [Download Chrome Dev](https://google.com/chrome/dev/).

2. Check that you’re on 127.0.6512.0 or above

3. Enable two flags: chrome://flags/#optimization-guide-on-device-model - BypassPerfRequirement chrome://flags/#prompt-api-for-gemini-nano - Enabled

4. Relaunch Chrome

5. Navigate to chrome://components

6. Check that Optimization Guide On Device Model is downloading or force download if not Might take a few minutes for this component to even appear

7. Open dev tools and type (await ai.languageModel.capabilities()).available, should return "readily" when all good

[1]: https://github.com/ryanseddon/FFprompt

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#352

Post by flippyhead »

flippyhead wrote: post #35

I have a tiny device that listens to conversations between two people or more and constantly tries to declare a "winner"

Heh, I made this comment and forgot to check back -- I'm always missing stuff on HN because of this!

If anyone is still paying attention, email me at hi@seikai.tv and I'll see if I can send you one.

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#353

Post by saurik »

ignoramous wrote: post #28

We're prototyping a text firewall (for Android) with Gemma2 2B (which limits us to English), though DeepSeek's R1 variants now look pretty promising [0]: Depending on the content, we rewrite the text or quarantine it from your view. Of course this is easy (for English) in the sense that the core logic is all LLMs [1], but the integration points (on Android) are not so straight forward for anything other than SMS. [2]…

FWIW, that chat.deepseek.com link at least doesn't still work... did it ever (for people not logged into your account)?

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#354

Post by Shonku_ »

flippyhead wrote: post #352
flippyhead wrote: post #35

I have a tiny device that listens to conversations between two people or more and constantly tries to declare a "winner"

Heh, I made this comment and forgot to check back -- I'm always missing stuff on HN because of this! If anyone is still paying attention, email me at hi@seikai.tv and I'll see if I can send you one.

Yeah I'm still paying attention!

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#355

Post by ultrasounder »

flippyhead wrote: post #352
flippyhead wrote: post #35

I have a tiny device that listens to conversations between two people or more and constantly tries to declare a "winner"

Heh, I made this comment and forgot to check back -- I'm always missing stuff on HN because of this! If anyone is still paying attention, email me at hi@seikai.tv and I'll see if I can send you one.

Sounds cool! In fact, this can be applied to other areas such as "debate monitoring" for debate competitions

Re: Ask HN: Is anyone doing anything cool with tiny language models?

#356

Post by ignoramous »

saurik wrote: post #353
ignoramous wrote: post #28

We're prototyping a text firewall (for Android) with Gemma2 2B (which limits us to English), though DeepSeek's R1 variants now look pretty promising [0]: Depending on the content, we rewrite the text or quarantine it from your view. Of course this is easy (for English) in the sense that the core logic is all LLMs [1], but the integration points (on Android) are not so straight forward for anything other than SMS. [2]…

FWIW, that chat.deepseek.com link at least doesn't still work... did it ever (for people not logged into your account)?

Guess it never has. Though, the example I shared (reproduced below) is pretty basic.

---

Prompt:

  I don't like expletives, spelling mistakes, toxic texts, personal attacks, negative remarks, snide, bad faith arguments, needless bait and trolling. Help clean the texts for me. Rewrite or reword if you have to without losing meaning. If you do reword the text, mention it. If not, reproduce it as-is.

  Texts:
  Mel: "Fuck you you dim twat"
  
  Bel: "Wanna meet tomorrow?"
  
  Cel: "lol, you're clueless; do you even hear yourself?"
  
  Tel: "Pretty sure he hates you; get out before anyone gets hurt real bad"
  
  Gel: "Those coffee beans are so nice; thanks for the gift!"
  
  Lel: "hahaha, you mother fucker; you nailed it, congrats!"
  
  Hel: "OK Goooooogle"
---

Output:

  Here's the cleaned-up version of the texts:

  Mel: "I’m really upset with you right now."  
  (Reworded to remove expletives and offensive language.)
  
  Bel: "Wanna meet tomorrow?"  
  (No changes needed.)

  Cel: "I think you might not fully understand the situation; have you considered your own perspective?"  
  (Reworded to remove snide remarks and make it more constructive.)
  
  Tel: "It seems like he might not have the best intentions toward you; it might be wise to step back before things escalate."  
  (Reworded to remove negativity and make it more neutral and caring.)
  
  Gel: "Those coffee beans are so nice; thanks for the gift!"  
  (No changes needed.)
  
  Lel: "Hahaha, you did an amazing job! Congrats!"  
  (Reworded to remove expletives while keeping the celebratory tone.)
  
  Hel: "OK Google."  
  (Corrected spelling for clarity.)
Post reply on HN