Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What AI assistants are already bundled for Linux?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 64 posts

Re: Ask HN: What AI assistants are already bundled for Linux?

#21

Post by HKH2 »

AlienRobot wrote: post #7
lukan wrote: post #3

Earlier quoted context omitted.

"I wouldn't hold my breath waiting for a native Linux AI-assisted assistant" A simple chat window and a automated script to install a existing small modell should be doable, but sounds not very exciting to me. But mid term, having a locally run LLM and integrated into the OS that scans my files and can summarize folders for me, would be nice. I have big folders with mixed stuff, AI would be nice to sort that. I do be…

>scans my files and can summarize folders for me Can you tell me what exactly you want it to do? You have a bunch of files in folders and you want the AI to tell you what exactly?

Not OP, but when searching for files, spelling something wrong, or using the wrong synonym is a big problem. We're just used to computers being inflexible.

Re: Ask HN: What AI assistants are already bundled for Linux?

#22

Post by yjftsjthsd-h »

NixOS has packages for llama-cpp and ollama: https://search.nixos.org/packages?channel=23.11&from=0&size=...

Edit: And Arch packages ollama officially - https://archlinux.org/packages/?sort=&q=llama&maintainer=&fl... - and a few things in the AUR - https://aur.archlinux.org/packages?O=0&K=llama

Re: Ask HN: What AI assistants are already bundled for Linux?

#23

Post by AlienRobot »

HKH2 wrote: post #21
AlienRobot wrote: post #7

Earlier quoted context omitted.

>scans my files and can summarize folders for me Can you tell me what exactly you want it to do? You have a bunch of files in folders and you want the AI to tell you what exactly?

Not OP, but when searching for files, spelling something wrong, or using the wrong synonym is a big problem. We're just used to computers being inflexible.

I mean, I see, but do you need an entire AI for this? This sounds like a specific problem that would be best solved by a specific algorithm.

Re: Ask HN: What AI assistants are already bundled for Linux?

#24

Post by eurekin »

brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

I imagine such assistant would be on top of support charts for "bogs down RAM" and "sucks up battery" for most users

Re: Ask HN: What AI assistants are already bundled for Linux?

#26

Post by diggan »

brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

> One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and

Huh? That's not at all true. It's only using processing power (CPU) while it actually generates text, otherwise it sits and wait. Although yes it occupies memory (RAM or VRAM) if you don't unload it, but you can configure it to startup when you need it, and shut down when you don't.

Re: Ask HN: What AI assistants are already bundled for Linux?

#27

Post by oceanplexian »

brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

At least with a GPU that can do power save that's not the case. I have a box with some 3090's in it, each card will idle <50W when it's not doing inference with the weights loaded into VRAM. Only when I ask it to do inference it will spin up and start consuming 300-400W.

Re: Ask HN: What AI assistants are already bundled for Linux?

#28

Post by wing-_-nuts »

gtirloni wrote: post #2

None. Microsoft has Copilot in preview mode in Windows and it's not very integrated apart from a chat window. I doubt GNOME/KDE will be able to dedicate enough resources to adding an assistant that is well integrated with the desktop environment any time soon. A search in Fedora yields a single GSoC project[0] limited in scope to NetworkManager and it's not clear if anyone actually is working on that. If the use case…

Given the fact that one can control damned near everything over command line in linux, and command line is a much more stable interface than a gui, I'd guess that there's a great deal more potential for assistants in linux than windows.

The other day I wanted to figure out how to turn my dock red if I dropped the vpn in gnome. I found the file that controlled my wireguard gnome shell extension and with the help of gpt3.5 and some very rudimentary js knowledge (I'm a backend dev, don't hate me), I was able to add a js function to toggle the color on vpn up / down events. This didn't even take me an hour to do and I'd never even thought to try it before GPT.

Sure, things are janky now, but the future potential of LLMs with linux and OSS is huge.

Re: Ask HN: What AI assistants are already bundled for Linux?

#29

Post by zajio1am »

diggan wrote: post #26
brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

> One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and Huh? That's not at all true. It's only using processing power (CPU) while it actually generates text, otherwise it sits and wait. Although yes it occupies memory (RAM or VRAM) if you don't unload it, but you can configure it to startup when you need it…

If one uses llama.cpp in CPU mode, then models are mmaped, so they do not really occupy memory when not used (they are just in reclaimable page cache).
Post reply on HN