Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Open-source ChatGPT alternatives?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Ask HN: Open-source ChatGPT alternatives?

#51

Post by bzmrgonz »

simonw wrote: post #32
baobabKoodaa wrote: post #15

Earlier quoted context omitted.

Am I going to be able to fine tune GPT-J or GPT-JT on consumer hardware?

When you say "fine tune" here what are you looking to do? The impression I've got is that fine tuning large language models is mostly useful for very simple tasks, such as training a spam or categorization filter. If you're looking to take a model and then e.g. train it on a few thousand additional pages of documentation in order to get it to answer project-specific questions, I've got the impression that fine tuning…

This is very interesting, I think what the world needs, is the inference/deductive reasoning/critical thinking kung-fu in an AI, which can be trained on "bespoke" body of knowledge/datalake.

Re: Ask HN: Open-source ChatGPT alternatives?

#53

Post by moomoo11 »

I’m only a consumer of GPT.

Any pointers as to where someone like me with software engineering experience but literally no AI knowledge can train my own GPT on my own data sets?

In my case I have downloaded some public domain databases (1-7gb each) and I would like to get some additional insights out of them. I have been querying them and using them to build my company but I’m curious to know if GPT can help me in that regard.

Re: Ask HN: Open-source ChatGPT alternatives?

#54

Post by v3ss0n »

Tried all of them, they are still at gpt2 tier. None of them close to gpt3, so far away from davinci.

To run your own laptop and cheap we still need much fine tuned training set with much better algorithm .

Right now, most capable ones needs over 120gb of VRAM just inference (run).

Re: Ask HN: Open-source ChatGPT alternatives?

#56

Post by MagicMoonlight »

roxgib wrote: post #38
valgaze wrote: post #18

Clue on conversation "history"-- "While ChatGPT is able to remember what the user has said earlier in the conversation, there is a limit to how much information it can retain. The model is able to reference up to approximately 3000 words (or 4000 tokens) from the current conversation - any information beyond that is not stored. Please note that ChatGPT is not able to access past conversations to inform its responses.…

Presumably one of the benefits of running your own model is that you can feed extra data into it via training rather than purely through inference? I.e. if you're a software company you could fine-tune it on your codebase, improving its answers without increasing inference time?

ChatGPT is itself just a GPT that has been finetuned to make it act as a chat bot and try not to be offensive

Re: Ask HN: Open-source ChatGPT alternatives?

#57

Post by npteljes »

dieselgate wrote: post #10

I'm not very familiar with this space but would have thought "OpenAI" would be at least somewhat open-source. Is this just naming and not relevant to the product at all?

Brand names convey a brand, not a meaning. They just expect you to give them the good associations, without the commitment to actually honor them. Similarly to "People's Republic of", or in every case of foreign branding, like Haagen-Dazs.

https://en.wikipedia.org/wiki/Foreign_branding

Re: Ask HN: Open-source ChatGPT alternatives?

#58

Post by mikewarot »

Mizza wrote: post #37
baobabKoodaa wrote: post #25

Earlier quoted context omitted.

I briefly looked at this and it doesn't seem like they provide a model that I can fine tune on consumer hardware?

Hold your damn horses, this technology is brand new, requires a tremendous amount of data gathering and computation, there's a massive volunteer effort already under way, and you're begging for a free home version so you can save $0.02 cents. You want a hole, pick up a shovel.

You know, there are considerations far more important than money that might want someone to have an offline version.

I'd gladly pay $0.02 cents/api call if I could be absolutely certain my queries weren't be used for evil.

Sadly, that's not possible.

Re: Ask HN: Open-source ChatGPT alternatives?

#59

Post by bcarambio »

smoldesu wrote: post #2

This runs fine in RAM constrained ( https://huggingface.co/EleutherAI/gpt-neo-125M It's bigger brother, 1.3b, uses ~5.5gb of memory but yields slightly more GPT-like answers. Both take ~5-20 seconds to generate a response though, so take that into account when building with it.

Hi from OctoML -- our team just put out some benchmarking results using GPT-J with a few model config/hw combinations that have relatively low latency: /octoml.ai/blog/evaluating-gpt-j-deployment-options/
Post reply on HN