Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

github.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#41

Post by phren0logy »

candiddevmike wrote: post #7
phren0logy wrote: post #5

I have been really impressed with AnythingLLM as a no-fuss way to use LLMs locally and via APIs. For those of us who want to tinker, there's a solid range of choice for embedders and vector stores. The single install desktop packaging is very slick. I look forward to the upcoming new features.

Have you tried openwebui? What are your thoughts on it vs this?

I had a much smoother experience with the desktop version of AnythingLLM. There's more in openwebui, but the things that are in AnythingLLM are more polished (IMO).

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#42

Post by qup »

tcarambat1010 wrote: post #31
cholantesh wrote: post #29

Earlier quoted context omitted.

Or it was a simple typo.

It was a typo haha, someone just PR'd the fix for it. God knows how long it has been there - whoops

Maybe the assistant can look in the repo and compute that, just sayin'

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#44

Post by tcarambat1010 »

tencentshill wrote: post #39

As someone who doesn't know what an Embed or Vector is, this has been the only offline AI tool I've been able to install and start using on my standard office PC.

That's the point! Shouldn't have to know and naturally over time you'll learn more and the controls are there when you feel comfortable tuning them

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#45

Post by tcarambat1010 »

phren0logy wrote: post #41
candiddevmike wrote: post #7

Earlier quoted context omitted.

Have you tried openwebui? What are your thoughts on it vs this?

I had a much smoother experience with the desktop version of AnythingLLM. There's more in openwebui, but the things that are in AnythingLLM are more polished (IMO).

We will have plugins for: - Auth - Agent skills - Data connectors

soon, agent skills are first and this should help plug the gaps between offerings since we can't build everything for everyone

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#47

Post by [deleted] »

smokel wrote: post #21
101008 wrote: post #10

LLM will become like web frameworks in the future, in the sense that they will be free, open source, and everybody will be able to build on them. Sure, there will be paid options, such as there are paid web frameworks, but most of the time free options will be more than good enough for most of the jobs.

What is your argumentation for that? It seems that building LLMs is very costly, and that will probably remain so for quite some time. I think it is equally likely that LLMs will perform worse in the future, because of copyright reasons and increased privacy awareness, leading to less diverse data sets to train on.

[deleted]

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#48

Post by monkmartinez »

egamirorrim wrote: post #46

I've got an open ai API key, and I pay for chatgpt. I'd imagine switching to this and using openai would end up costing quite a lot? How are people running it relatively cheaply?

Openrouter... you get all the models and its not as expensive as you would think. I spent $3 with aider the other day in like the blink of an eye with Anthropic. I am working on a FASTHTML thingy and loaded all the docs, plus a few huge replicate api files into the vector database. Most of my back and forth usage averaged about $0.02 for each turn with Claude 3.5 Sonnet. To give you an idea: My context + prompt were around 18000 tokens with completions around 1500 tokens.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#49

Post by aznumeric »

egamirorrim wrote: post #46

I've got an open ai API key, and I pay for chatgpt. I'd imagine switching to this and using openai would end up costing quite a lot? How are people running it relatively cheaply?

One way people keep costs down when using OpenAI with an offline RAG system is by limiting the number of text snippets sent to the API. Instead of sending the whole database, they'll typically retrieve only the top 10 (or so) most relevant snippets from the vector database and just send those to OpenAI for processing. This significantly reduces the amount of data being processed and billed by OpenAI.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#50

Post by ranger_danger »

tcarambat1010 wrote: post #36
ranger_danger wrote: post #19

$ docker pull mintplexlabs/anythingllm Using default tag: latest Error response from daemon: Get " https://registry-1.docker.io/v2/ ": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

This is a connection issue for your docker cli - we have seen this issue before and it had to do with client proxy. https://stackoverflow.com/a/48066666

I am not behind a proxy, do not have any of those related variables set, and all other attempts at net connections (curl, telnet, browser, etc.) work fine...
Post reply on HN