Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source solution replicates ChatGPT training process

hpc-ai.tech

Post reply on HN
Print view View original

101–110 of 158 posts

Re: Open source solution replicates ChatGPT training process

#101

Post by darthrupert »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I'm using it as an extra colleague with whom I can talk about my problem, or like a very advanced rubber duck. This gets me to a solution far quicker than just researching on my own, even if its answers aren't immediately correct.

I'm using it to learn French. I'm using it for figuring out if my book idea makes sense. To tell me how Typescript works, or how it compares to languages I already know. I use it to compare products I'm interested in, to make educated guesses where comparable products are manufactured in.

It's not as smart as my colleagues, but much smarter than a rubber duck, and it has a mountain of data behind it.

It changes everything and brings amazing potential to the table.

Re: Open source solution replicates ChatGPT training process

#102

Post by hybridity »

rnosov wrote: post #95
sillysaurusx wrote: post #57

> On a single multi-GPUs server, even with the highest-end A100 80GB GPU, PyTorch can only launch ChatGPT based on small models like GPT-L (774M), due to the complexity and memory fragmentation of ChatGPT. Hence, multi-GPUs parallel scaling to 4 or 8 GPUs with PyTorch's DistributedDataParallel (DDP) results in limited performance gains. Where are these numbers coming from? An 80GB A100 GPU is certainly more than capa…

I think they are correctly referring to ChatGPT as GPT-3 + RLHF. In other words ChatGPT = GPT-3 + RLHF. So, 80GB A100 GPU would be required for both GPT-L AND RLHF (PyTorch version). And it looks to me from the TFA that the main thing that takes a lot of space is actually RLHF. >I don’t understand how they went from talking about 175B params across 32 cards to 774M on one card. 175B divided by 32 is 5.4B. They claim…

If someone created a folding@home to crowd train an actually open ChatGPT, I'd gladly donate my spare resources to the cause.

Re: Open source solution replicates ChatGPT training process

#103

Post by runlevel1 »

logicallee wrote: post #44
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

>Can someone tell me what the hell they use ChatGPT for? Although it's free, I pay $20 for pro version ($240 per year) plus taxes, and use it daily. I get a lot of benefits from using it. I use it to learn about things, solve problems, suggest approaches, critique my own proposals and approaches, generate code scaffolding and smaller code solutions, help me draft emails of all kinds, etc. I find it highly useful in a…

I don't think it's actually analyzing that code. It's the winning IOCCC 2005 entry. The source comes up when you google the first 9 characters: https://www.google.com/search?q=%22B%2Ci%2Cy%2Cu%2Cb%22

EDIT: And that snippet is in the IOCCC's wikipedia article (which would be in the ChatGPT training corpus): https://en.wikipedia.org/wiki/International_Obfuscated_C_Cod...

Re: Open source solution replicates ChatGPT training process

#104

Post by flangola7 »

hybridity wrote: post #102
rnosov wrote: post #95

Earlier quoted context omitted.

I think they are correctly referring to ChatGPT as GPT-3 + RLHF. In other words ChatGPT = GPT-3 + RLHF. So, 80GB A100 GPU would be required for both GPT-L AND RLHF (PyTorch version). And it looks to me from the TFA that the main thing that takes a lot of space is actually RLHF. >I don’t understand how they went from talking about 175B params across 32 cards to 774M on one card. 175B divided by 32 is 5.4B. They claim…

If someone created a folding@home to crowd train an actually open ChatGPT, I'd gladly donate my spare resources to the cause.

That's unlikely to work. The memory has to be fast with low latency, even switching from on-board VRAM to system RAM slows performance at least 10-100x. The bottleneck isn't computing power it's I/O. Total bus bandwidth of a common small AI cluster is around 1 terabyte per second.

We really shouldn't be building an "open source" AI in the first place though, and it's going to be illegal to do so soon. The weaponization power will be made clear soon and that will justifiably spook everyone.

Re: Open source solution replicates ChatGPT training process

#105

Post by flangola7 »

dmw_ng wrote: post #36
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I have been using it as a search replacement for most of the past month and only found two subtly wrong answers. This covers legal questions, researching product differences, wiring diagrams, suggesting books to read, correcting misremembered quotes, and about a hundred other tasks. Of course still relying on google in the background, but increasingly rarely, and presuming all the negative commentary we've been seein…

It's a incredible at writing rich and persuasive comments that take the momentum out of bigoted Facebook posts. An extended family member is unfortunately all aboard the election fraud and "groomer" trains, posting absurd and hateful stuff constantly every day (in classic Facebook style many of these posts "do not violate the community guidelines). I and a couple other younger members of the family have taken to using ChatGPT to gently but firmly counter every lie and misdirection he tries to make. I'm not sure if it's deeply changed his mind or heart yet, but he posts much less extremist content now and has actually resumed posting wholesome and funny things like he did before going down the rabbit hole.

Re: Open source solution replicates ChatGPT training process

#106

Post by dodslaser »

flangola7 wrote: post #104
hybridity wrote: post #102

Earlier quoted context omitted.

If someone created a folding@home to crowd train an actually open ChatGPT, I'd gladly donate my spare resources to the cause.

That's unlikely to work. The memory has to be fast with low latency, even switching from on-board VRAM to system RAM slows performance at least 10-100x. The bottleneck isn't computing power it's I/O. Total bus bandwidth of a common small AI cluster is around 1 terabyte per second. We really shouldn't be building an "open source" AI in the first place though, and it's going to be illegal to do so soon. The weaponizati…

There's a significant number of people working hard on making certain tech illegal or at least heavily restricted. E2EE and Onion Routing comes to mind. That doesn't mean we should abandon them. In fact, in many cases it's an indicator that we should keep going.

Why do you think we should avoid an open source AI?

Re: Open source solution replicates ChatGPT training process

#107

Post by visarga »

EGreg wrote: post #30
visarga wrote: post #22

Earlier quoted context omitted.

> We're going to need AI to sift through all the BS. Yes, that's the only way to deal with it. Humans alone can't cope.

Somehow bombs don’t actually prevent other bombs. People always hope that the offensive tech could be used defensively, but defense is never perfect and even a few that get through can wreak destruction.

I see it like the cat-and-mouse game of viruses and immune system, or shells and armour. We need "AI immunity" to deal with other AIs. It's not going to be solved in one iteration, we got to keep updating it.

Re: Open source solution replicates ChatGPT training process

#108

Post by ipaddr »

calny wrote: post #39
cldellow wrote: post #28

Earlier quoted context omitted.

The 30M figure likely includes a lot of students having ChatGPT do their homework for them. :) I've used ChatGPT for programming aid. I've started writing some Python packages. I haven't written Python in a long time, it doesn't "flow" easily for me. ChatGPT has been helpful here for scaffolding some code. It often gets things wrong -- but I know enough to recognize when it's gone off the rails, and then nudge it in…

> It often gets things wrong -- but I know enough to recognize when it's gone off the rails, and then nudge it in the right direction. > specify something at a higher level and have the computer sort out the details. Same here. I know some people frown on Github Copilot, but ChatGPT + Copilot makes a powerful combo. I actually use ChatGPT like a copilot, to talk through the structure of things, debugging issues, etc.…

Be careful using these aids will reduce the learning that normally happens in programming.

Re: Open source solution replicates ChatGPT training process

#109

Post by ipaddr »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I can't get it to answer anything.

Tallest people in US - filter cannot answer personal characteristics off limits.

What number come up most often playing the lottery - I do not have that information

show me a list of 100 different ...- 10 results..

It seems to hate polite. Please give me.. NO vs give me NOW here you go

It is not useful for me. I ask it programming questions and hate the output.. or know where they got the output and can see they missed key steps.

I feel like I know what it will answer and it's mostly surface level answers.

For people who don't want a conversation and can find the information quicker the hype doesn't add up. Im fairness tiktok bores me.

Re: Open source solution replicates ChatGPT training process

#110

Post by ahtihn »

flangola7 wrote: post #104
hybridity wrote: post #102

Earlier quoted context omitted.

If someone created a folding@home to crowd train an actually open ChatGPT, I'd gladly donate my spare resources to the cause.

That's unlikely to work. The memory has to be fast with low latency, even switching from on-board VRAM to system RAM slows performance at least 10-100x. The bottleneck isn't computing power it's I/O. Total bus bandwidth of a common small AI cluster is around 1 terabyte per second. We really shouldn't be building an "open source" AI in the first place though, and it's going to be illegal to do so soon. The weaponizati…

> We really shouldn't be building an "open source" AI in the first place though, and it's going to be illegal to do so soon.

How do you make that illegal while still allowing private corporations to build AI? How do you legally define AI without applying it to all kinds of existing applications and without stopping all research on AI? And while staying broad enough that simply using a slightly different technique would still qualify under that definition?

Post reply on HN