Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

github.com

Post reply on HN
Print view View original

1–10 of 78 posts

Re: Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

#5

Post by belval »

Nice project! I could not find the information in the README.md, can I run this with a GPU? If so what do I need to change? Seems like it's hardcoded to 0 in the run script: https://github.com/getumbrel/llama-gpt/blob/master/api/run.s...

Re: Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

#7

Post by ryanSrich »

mayankchhabra wrote: post #3
albert_e wrote: post #2

Oh I thought this was a quick guide to host it on any server (AWS / other clouds) of our choosing.

Yes! It can run on any home server or cloud server.

Interesting. I might try to get this to work on my NAS.

Re: Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

#9

Post by benreesman »

chasd00 wrote: post #6

is it a free model or is the politically-correct-only response constraints in place?

I’m a little out of date (busy few weeks), didn’t the Vicuna folks un-housebreak the LLaMA 2 language model (which is world class) with a slightly less father-knows-best Instruct tune?

Re: Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2

#10

Post by Havoc »

chasd00 wrote: post #6

is it a free model or is the politically-correct-only response constraints in place?

Llama is definitely "censored" though I've not found this to be an issue in practice. Guess it depends on what you want to do with it
Post reply on HN