Ask HN: Open-source ChatGPT alternatives?

#1

Post by baobabKoodaa » Tue, Feb 14, 2023, 8:58 PM UTC

What's the state of the art in open source GPT models right now, in practical terms? If your typical use case is taking a pretrained model and fine tuning it to a specific task, which LLM would yield the best results while running on consumer hardware? Note that I'm specifically asking for software that I can run on my own hardware, I'm not interested in paying OpenAI $0.02 per API request.

I'll start the recommendations with Karpathy's nanoGPT: https://github.com/karpathy/nanoGPT

What else do we have?

Re: Ask HN: Open-source ChatGPT alternatives?

#2

Post by smoldesu » Tue, Feb 14, 2023, 9:01 PM UTC

This runs fine in RAM constrained (https://huggingface.co/EleutherAI/gpt-neo-125M

It's bigger brother, 1.3b, uses ~5.5gb of memory but yields slightly more GPT-like answers. Both take ~5-20 seconds to generate a response though, so take that into account when building with it.

Re: Ask HN: Open-source ChatGPT alternatives?

#3

Post by init0 » Tue, Feb 14, 2023, 9:02 PM UTC

[flagged]

Re: Ask HN: Open-source ChatGPT alternatives?

#4

Post by baobabKoodaa » Tue, Feb 14, 2023, 9:19 PM UTC

init0 wrote: post #3

[flagged]

Did you generate this with Bing? I don't believe this answer is human-written. It describes 2 separate projects with the words "It is available on GitHub and can be used in various research projects".

Re: Ask HN: Open-source ChatGPT alternatives?

#5

Post by eric_hui » Tue, Feb 14, 2023, 9:21 PM UTC

[dead]

Re: Ask HN: Open-source ChatGPT alternatives?

#6

Post by speedgoose » Tue, Feb 14, 2023, 9:25 PM UTC

baobabKoodaa wrote: post #4

init0 wrote: post #3

[flagged]

Did you generate this with Bing? I don't believe this answer is human-written. It describes 2 separate projects with the words "It is available on GitHub and can be used in various research projects".

It’s also not very up to date which is suspicious for such a precise answer.

Re: Ask HN: Open-source ChatGPT alternatives?

#7

Post by speedgoose » Tue, Feb 14, 2023, 9:28 PM UTC

You also have GPT J 6B and BLOOM but to be honest they are not like ChatGPT.

https://huggingface.co/EleutherAI/gpt-j-6B

https://huggingface.co/bigscience/bloom

Re: Ask HN: Open-source ChatGPT alternatives?

#8

Post by baobabKoodaa » Tue, Feb 14, 2023, 9:29 PM UTC

speedgoose wrote: post #7

You also have GPT J 6B and BLOOM but to be honest they are not like ChatGPT. https://huggingface.co/EleutherAI/gpt-j-6B https://huggingface.co/bigscience/bloom

Can you elaborate on how they are not like ChatGPT? I was looking into GPT-JT (built on top of GPT J you mentioned). If I spend the time to actually finetune and run inference with one of these models, am I likely going to be disappointed at the results?

Re: Ask HN: Open-source ChatGPT alternatives?

#9

Post by mindcrime » Tue, Feb 14, 2023, 9:32 PM UTC

Previous related discussions:

https://news.ycombinator.com/item?id=34115698

https://news.ycombinator.com/item?id=33955125

https://news.ycombinator.com/item?id=34163413

https://news.ycombinator.com/item?id=34628256

https://news.ycombinator.com/item?id=34147281

https://news.ycombinator.com/item?id=34445873

Re: Ask HN: Open-source ChatGPT alternatives?

#10

Post by dieselgate » Tue, Feb 14, 2023, 9:35 PM UTC

I'm not very familiar with this space but would have thought "OpenAI" would be at least somewhat open-source. Is this just naming and not relevant to the product at all?