Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source solution replicates ChatGPT training process

hpc-ai.tech

Post reply on HN
Print view View original

31–40 of 158 posts

Re: Open source solution replicates ChatGPT training process

#31

Post by carlgreene »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I recently used it sort of as a rubber duck for a coding problem. I was architecting a new feature and the way I was thinking about it was a bit clunky.

ChatGPT helped point something obvious out that I had totally missed in my original problem solving.

Re: Open source solution replicates ChatGPT training process

#33

Post by meltedcapacitor »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

Jobs that require correct answers is a small subset of jobs that require answers.

Re: Open source solution replicates ChatGPT training process

#34

Post by wcoenen »

de6u99er wrote: post #29
simonw wrote: post #17

Is the term "ChatGPT" being used in place of GPT-3 here? Is this thing actually replicating the GPT-3 training process? The thing that makes ChatGPT interesting (over regular GPT-3) is the RLHF process, but this article doesn't seem to touch on that at all, unless I've missed something.

GPT-3 has been publicly covered in scientific publications. Same as GPT-2, and GPT. Those are all pre-trained models, where GPT is the abbreviation of Generative Pretrained Transformer. Transformers have been invented in 2017 at Google Brain [1]. -> https://medium.com/walmartglobaltech/the-journey-of-open-ai-... GPT-4 is around the corner, and it's allegedly 100x more powerful than it'd predecessor. -> https://medium…

That source about GPT-4 is nonsense. It claims GPT-4 will have trillions of parameter, and at the same time links to another page which says that it won't be much bigger than GPT-3:

https://www.datacamp.com/blog/what-we-know-gpt4

Re: Open source solution replicates ChatGPT training process

#35

Post by simonw »

de6u99er wrote: post #29
simonw wrote: post #17

Is the term "ChatGPT" being used in place of GPT-3 here? Is this thing actually replicating the GPT-3 training process? The thing that makes ChatGPT interesting (over regular GPT-3) is the RLHF process, but this article doesn't seem to touch on that at all, unless I've missed something.

GPT-3 has been publicly covered in scientific publications. Same as GPT-2, and GPT. Those are all pre-trained models, where GPT is the abbreviation of Generative Pretrained Transformer. Transformers have been invented in 2017 at Google Brain [1]. -> https://medium.com/walmartglobaltech/the-journey-of-open-ai-... GPT-4 is around the corner, and it's allegedly 100x more powerful than it'd predecessor. -> https://medium…

That "100x" figure is extremely poorly sourced. I don't believe that at all.

Re: Open source solution replicates ChatGPT training process

#36

Post by dmw_ng »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I have been using it as a search replacement for most of the past month and only found two subtly wrong answers. This covers legal questions, researching product differences, wiring diagrams, suggesting books to read, correcting misremembered quotes, and about a hundred other tasks.

Of course still relying on google in the background, but increasingly rarely, and presuming all the negative commentary we've been seeing online are folk who simply haven't tested it in anger yet. Today's chatgpt hallucination is yesterday's Google blogspam etc. Folk for some reason continue to act like the old world was perfect. This is much closer to perfection than anything we ever had, and infinitely more comprehensive. Google as we knew it is already dead, because the medium google was built for just got made obsolete. This is far closer to a new Internet iteration (WAIS, FTP, Gopher, HTTP, Web2.0, ...) than it is a new search engine

Now watch as the search engines try to adapt it to their recency-biased ads model and fail miserably, as what we have is already better than what they were able to sell. Very unclear bing or Google or anyone you've heard of will win this round, its suddenly a very exciting time in tech again

Another aspect I find very exciting is that these effectively represent a return to a curation-driven Internet, selection of input data for model training is probably an interesting new form of diversification. Who cares about having a site in the world wide web if its not part of the inputs for the language models used by millions of users? That's a completely new structure for the dissemination of ideas, marketing, "SEO" etc., and a brand new form of mass media

Re: Open source solution replicates ChatGPT training process

#37

Post by mansion7 »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I have not used it to create content for profit (yet) but have successfully used it for:

brainstorming funny/catchy slogans: not all are winners, but since it can crank out dozens almost immediately, I can pick what I like and quickly modify them in the time it takes me to think of one or two independently. As soon as I verify they aren't ripoffs of existing material, I may use one or two.

Writing poetry - it helped me to write sonnets, and further modified them to specifications. The recipients were quite impressed.

Translating existing poetry of mine into Arabic, while retaining the meaning AND rhyming in Arabic, a feat which is extremely difficult for me

Writing a business plan to my specifications that was actually useful

Writing letters to a landlord to get out of a lease

In addition, I have run my own fiction through it and had it rewrite it relatively convincingly in the styles of Lee Child, Danielle Steele, and Dashiell Hammett. That is more for fun, but I can see uses for it.

Lastly, I have attempted to use it to determine guilt in an investigation where I had already determined the guilty party, to see how close it was to replacing me. The answer it gave was wrong, but I could see that this was because of user error and it is only a matter of time.

Re: Open source solution replicates ChatGPT training process

#38

Post by mcaravey »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I’ve used it to write out 45 minute long lesson plans, help write complicated text message where all I’ve got is a bunch of points to make, I’ve had it correct my Portuguese since I’m not a native speaker, I’ve had it give me a baseline SQL table design to achieve a specific goal, I’ve had it come up with different ways to phrase things since I’m not creative enough, I’ve had it write marketing copy, created design briefs for my graphic design team, and on… I happily pay for it because it’s just nuts how much of a force multiplier it is for me.

Re: Open source solution replicates ChatGPT training process

#39

Post by calny »

cldellow wrote: post #28
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

The 30M figure likely includes a lot of students having ChatGPT do their homework for them. :) I've used ChatGPT for programming aid. I've started writing some Python packages. I haven't written Python in a long time, it doesn't "flow" easily for me. ChatGPT has been helpful here for scaffolding some code. It often gets things wrong -- but I know enough to recognize when it's gone off the rails, and then nudge it in…

> It often gets things wrong -- but I know enough to recognize when it's gone off the rails, and then nudge it in the right direction.

> specify something at a higher level and have the computer sort out the details.

Same here. I know some people frown on Github Copilot, but ChatGPT + Copilot makes a powerful combo. I actually use ChatGPT like a copilot, to talk through the structure of things, debugging issues, etc. Then Copilot works as a smarter autofill if I don't know the exact code or syntax needed off the top of my head. Both ChatGPT and Copilot get things wrong sometimes, but are correct often enough that it improves time spent. Even when ChatGPT is wrong it sometimes discusses useful concepts I had't thought about.

To be fair, I'm a self-taught and often jump between languages and frameworks that I'm not an expert in. Perhaps Copilot + ChatGPT would be less useful for a pro devs who are experts in their areas. But for my case, they're quite helpful.

Entirely separate: I also use ChatGPT to turn stream-of-consciousness thoughts into medium-length letters or emails.* Eg, I had to email a dog trainer and had a bunch of concerns to raise. It would've taken a fair number of minutes to make it coherent and easily-readable. Instead, I explained the situation to ChatGPT and hastily typed out the concerns, giving no regard to grammar, typos, or syntax. Then I asked ChatGPT to turn it into an email to the trainer with my intended tone, and it worked like a charm. That process took maybe 1/4 the time of manually writing the full email.

* this semi-stream-of-consciousness post was NOT written with ChatGPT, though perhaps it should've been

Re: Open source solution replicates ChatGPT training process

#40

Post by prox »

dmw_ng wrote: post #36
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I have been using it as a search replacement for most of the past month and only found two subtly wrong answers. This covers legal questions, researching product differences, wiring diagrams, suggesting books to read, correcting misremembered quotes, and about a hundred other tasks. Of course still relying on google in the background, but increasingly rarely, and presuming all the negative commentary we've been seein…

It’s nice to get quick in context answers to concepts and their relationships. Sometimes I have a vague notion, but with ChatGPT it resolves my hunch quite quickly without reading through a (sometimes ad spammed) article.

Google should be concerned.

Post reply on HN