Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant • Hacker News Board

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#21

Post by smokel » Thu, Sep 05, 2024, 5:31 PM UTC

101008 wrote: post #10

LLM will become like web frameworks in the future, in the sense that they will be free, open source, and everybody will be able to build on them. Sure, there will be paid options, such as there are paid web frameworks, but most of the time free options will be more than good enough for most of the jobs.

What is your argumentation for that? It seems that building LLMs is very costly, and that will probably remain so for quite some time.

I think it is equally likely that LLMs will perform worse in the future, because of copyright reasons and increased privacy awareness, leading to less diverse data sets to train on.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#22

Post by nunobrito » Thu, Sep 05, 2024, 5:32 PM UTC

yjftsjthsd-h wrote: post #13

nunobrito wrote: post #8

There was an error while installing on Linux, was solved with: ''' sudo chown root:root /home/hn/AnythingLLMDesktop/anythingllm-desktop/chrome-sandbox sudo chmod 4755 /home/hn/AnythingLLMDesktop/anythingllm-desktop/chrome-sandbox ''' Other than that it worked really well.

HN doesn't use normal markdown; to do code blocks you indent with 4 spaces, ex. sudo chown root:root /home/hn/AnythingLLMDesktop/anythingllm-desktop/chrome-sandbox sudo chmod 4755 /home/hn/AnythingLLMDesktop/anythingllm-desktop/chrome-sandbox

Thanks!

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#23

Post by politelemon » Thu, Sep 05, 2024, 5:34 PM UTC

> AnythingLLM packages as an AppImage but you will not be able to boot if you run just the AppImage.

The 'boot' seems to indicate it will affect the computer's startup process, I think you meant to say you will not be able to 'start the application'

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#24

Post by Tostino » Thu, Sep 05, 2024, 5:38 PM UTC

vednig wrote: post #3

How do you ensure "privacy by default" if you are also providing cloud models?

It doesn't seem like they are "providing" cloud models. They have their backend able to interface with whatever endpoint you want, given you have access. It's plainly obvious when interacting with 3rd party providers that it depends on their data / privacy policy. When has this ever not been the case?

I could just start up a vLLM instance with llama 3.1 and connect their application to it just as easily though. Perfectly secure (as I am able to make it).

This seems like such a pedantic thing to complain about.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#25

Post by CuriouslyC » Thu, Sep 05, 2024, 5:39 PM UTC

I noticed you put LiteLLM in your list of providers. Was that just marketing, or did you re-implement model support for all the models LiteLLM already supports separately?

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#26

Post by bboygravity » Thu, Sep 05, 2024, 5:54 PM UTC

smokel wrote: post #21

101008 wrote: post #10

LLM will become like web frameworks in the future, in the sense that they will be free, open source, and everybody will be able to build on them. Sure, there will be paid options, such as there are paid web frameworks, but most of the time free options will be more than good enough for most of the jobs.

What is your argumentation for that? It seems that building LLMs is very costly, and that will probably remain so for quite some time. I think it is equally likely that LLMs will perform worse in the future, because of copyright reasons and increased privacy awareness, leading to less diverse data sets to train on.

Existing free to use (offline) models are already really really good and open a ton of possibilities for new apps.

The only barrier right now is that laptop/phone hardware can't run them (inference and/or training) fast enough. That's a problem that won't really exist 3 to 10 years from now.

That's ignoring the fact that creation of new much much better models is going full steam ahead.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#27

Post by A4ET8a8uTh0 » Thu, Sep 05, 2024, 6:02 PM UTC

This definitely makes it super easy for less technical folks to access it ( got it up and running in less than 5 minutes ). Initial reaction is positive with just Ollama. Everything is automatically detected and if you want to manually set it up, you still can. Lets see how it does after adding huggingface ( quick and painless ).

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#28

Post by md3911027514 » Thu, Sep 05, 2024, 6:18 PM UTC

this is super sweet for developers (or anyone else) who like to have granular control over their LLM set up

- ability to edit system prompt

- ability to change LLM temperature

- can choose which model to use (open-source or closed)

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#29

Post by cholantesh » Thu, Sep 05, 2024, 6:24 PM UTC

rafram wrote: post #17

mattygee wrote: post #14

Earlier quoted context omitted.

Large Language Model All good

The linked GitHub page mentions “Language Learning Models”, so GP is saying that it seems like the authors of this project wrote the whole thing while not knowing what LLM actually stands for.

Or it was a simple typo.

Re: Show HN: AnythingLLM – Open-Source, All-in-One Desktop AI Assistant

#30

Post by DebtDeflation » Thu, Sep 05, 2024, 6:28 PM UTC

This is really nice. My first reaction was "oh great, another Ollama/WebGenUI wrapper on Llama.cpp" but it's actually much more - supporting not only the LLM but also embedding models, vector databases, and TTS/STT. Everything needed to build a fully functioning voice chatbot.