Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What AI assistants are already bundled for Linux?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: Ask HN: What AI assistants are already bundled for Linux?

#41

Post by __loam »

dartharva wrote: post #32
brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

You don't need the larger/smarter models for most "assistant" use cases. Small language models like Phi-2 can be enough.

I don't need any of these things shipping with my default distro.

Re: Ask HN: What AI assistants are already bundled for Linux?

#42

Post by lukan »

AlienRobot wrote: post #7
lukan wrote: post #3

Earlier quoted context omitted.

"I wouldn't hold my breath waiting for a native Linux AI-assisted assistant" A simple chat window and a automated script to install a existing small modell should be doable, but sounds not very exciting to me. But mid term, having a locally run LLM and integrated into the OS that scans my files and can summarize folders for me, would be nice. I have big folders with mixed stuff, AI would be nice to sort that. I do be…

>scans my files and can summarize folders for me Can you tell me what exactly you want it to do? You have a bunch of files in folders and you want the AI to tell you what exactly?

"You have a bunch of files in folders and you want the AI to tell you what exactly?"

For example: "show me the folder(s), where my old University projects are stored"

"Sort this folder into programming projects, general notes, pictures, music, videos and install files"

"Find me the folder where I made notes about a novel sorting algorithm"

Re: Ask HN: What AI assistants are already bundled for Linux?

#43

Post by lukan »

MuffinFlavored wrote: post #6
lukan wrote: post #3

Earlier quoted context omitted.

"I wouldn't hold my breath waiting for a native Linux AI-assisted assistant" A simple chat window and a automated script to install a existing small modell should be doable, but sounds not very exciting to me. But mid term, having a locally run LLM and integrated into the OS that scans my files and can summarize folders for me, would be nice. I have big folders with mixed stuff, AI would be nice to sort that. I do be…

> But mid term, having a locally run LLM and integrated into the OS that scans my files and can summarize folders for me But don't most LLMs have about a max 32k token context?

The idea is to have a personal AI, that is trained with my content, my files. My emails, my pictures, projects, notes, etc. How that can be implemented in the best way, is not my expertize, but I believe is subject of heavy research.

Re: Ask HN: What AI assistants are already bundled for Linux?

#44

Post by diggan »

ingenieroariel wrote: post #36
diggan wrote: post #31

Earlier quoted context omitted.

Do anyone actually use the CPU for anything besides testing? Last time I tried it, it was horribly slow compared to GPU that there wasn't really any point to use for me, besides getting access to more memory.

I have. On a Mac Studio with NixOS based Asahi Linux and 128Gb of RAM, mixtral 8x7b uses 49GB of RAM. At the same time I load airflow tasks that deal with world wide datasets (using ~60GB on 16 parallel streams with the performance cores) format is parquet and also mmaped. Computer still has 8 efficiency cores and the whole GPU for visualizing the maps using lonboard / browsing / etc. The computer uses 8-10W when idl…

Interesting, thanks for sharing that! For curiosity, what kind of performance you get with that setup + mixtral 8x7b in terms of tokens/second?

Re: Ask HN: What AI assistants are already bundled for Linux?

#45

Post by ragebol »

wing-_-nuts wrote: post #28
gtirloni wrote: post #2

None. Microsoft has Copilot in preview mode in Windows and it's not very integrated apart from a chat window. I doubt GNOME/KDE will be able to dedicate enough resources to adding an assistant that is well integrated with the desktop environment any time soon. A search in Fedora yields a single GSoC project[0] limited in scope to NetworkManager and it's not clear if anyone actually is working on that. If the use case…

Given the fact that one can control damned near everything over command line in linux, and command line is a much more stable interface than a gui, I'd guess that there's a great deal more potential for assistants in linux than windows. The other day I wanted to figure out how to turn my dock red if I dropped the vpn in gnome. I found the file that controlled my wireguard gnome shell extension and with the help of gp…

Recently came across https://www.warp.dev/ on HN, which includes a AI part for your terminal. That's a paid feature running nonlocal, but it's a start I suppose.

Re: Ask HN: What AI assistants are already bundled for Linux?

#46

Post by bossyTeacher »

yjftsjthsd-h wrote: post #40
bossyTeacher wrote: post #17

Why? They take lots of space and lots of computing power. Linux has always been about lightweight and a bundle only containing essential things. You can always install one if you need it but as it stands right now LLMs are not useful enough to warrant their bundling in a distro. Just my 2 cents

> Linux has always been about lightweight and a bundle only containing essential things. I really don't think that's true. There have always been distros that are based on being tiny, of course, but I think most of the normal distros are concerned with hitting a happy medium of size and features. Otherwise I can't imagine why anything would be shipping GNOME or KDE over LXDE, or why libreoffice would be installed by…

you can't really compare the utility of an LLM against libreoffice or similar. There is no comparison. Libreoffice is something that you would definitely use unlike an LLM.

Re: Ask HN: What AI assistants are already bundled for Linux?

#47

Post by HKH2 »

number6 wrote: post #35
AlienRobot wrote: post #23

Earlier quoted context omitted.

I mean, I see, but do you need an entire AI for this? This sounds like a specific problem that would be best solved by a specific algorithm.

You can use fzf for this

Do you mean to say that fzf can find synonyms?

Re: Ask HN: What AI assistants are already bundled for Linux?

#48

Post by rvcdbn »

brucethemoose2 wrote: post #15

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes. Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

true but as engineers we should sometimes sacrifice and live in the future a bit to maximize opportunity and in the future hardware will adapt to software as it always has

Re: Ask HN: What AI assistants are already bundled for Linux?

#50

Post by p10_user »

bossyTeacher wrote: post #46
yjftsjthsd-h wrote: post #40

Earlier quoted context omitted.

> Linux has always been about lightweight and a bundle only containing essential things. I really don't think that's true. There have always been distros that are based on being tiny, of course, but I think most of the normal distros are concerned with hitting a happy medium of size and features. Otherwise I can't imagine why anything would be shipping GNOME or KDE over LXDE, or why libreoffice would be installed by…

you can't really compare the utility of an LLM against libreoffice or similar. There is no comparison. Libreoffice is something that you would definitely use unlike an LLM.

I use LLMs to double check/write python pandas code so I don't need to use Libreoffice/spreadsheet software.
Post reply on HN