Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source solution replicates ChatGPT training process

hpc-ai.tech

Post reply on HN
Print view View original

111–120 of 158 posts

Re: Open source solution replicates ChatGPT training process

#111

Post by nunobrito »

mcaravey wrote: post #38
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I’ve used it to write out 45 minute long lesson plans, help write complicated text message where all I’ve got is a bunch of points to make, I’ve had it correct my Portuguese since I’m not a native speaker, I’ve had it give me a baseline SQL table design to achieve a specific goal, I’ve had it come up with different ways to phrase things since I’m not creative enough, I’ve had it write marketing copy, created design b…

Exactly! I'm doing this for German language. It is rare for translators to get correctly how people talk in German and chatGPT is astonishingly good at that task.

Re: Open source solution replicates ChatGPT training process

#112

Post by imtringued »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

It helps me with analysis paralysis.

Re: Open source solution replicates ChatGPT training process

#113

Post by imtringued »

dmw_ng wrote: post #36
jackblemming wrote: post #23

Earlier quoted context omitted.

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I have been using it as a search replacement for most of the past month and only found two subtly wrong answers. This covers legal questions, researching product differences, wiring diagrams, suggesting books to read, correcting misremembered quotes, and about a hundred other tasks. Of course still relying on google in the background, but increasingly rarely, and presuming all the negative commentary we've been seein…

When I ask it for things that are obviously on stackoverflow but hard to spot or understand because they are pointlessly clever or weird, it is nigh unusable. It is a complete waste of time. Even if you paste in the stack overflow answers it will take some iterating and at that point I am teaching an unteachable AI.

Re: Open source solution replicates ChatGPT training process

#114

Post by animuchan »

flangola7 wrote: post #104
hybridity wrote: post #102

Earlier quoted context omitted.

If someone created a folding@home to crowd train an actually open ChatGPT, I'd gladly donate my spare resources to the cause.

That's unlikely to work. The memory has to be fast with low latency, even switching from on-board VRAM to system RAM slows performance at least 10-100x. The bottleneck isn't computing power it's I/O. Total bus bandwidth of a common small AI cluster is around 1 terabyte per second. We really shouldn't be building an "open source" AI in the first place though, and it's going to be illegal to do so soon. The weaponizati…

For this reason alone (corpos making AI illegal to maintain for mere mortals) we should strive to make as much progress in the truly open AI as possible.

The current dystopia is fairly dystopian as it is.

Re: Open source solution replicates ChatGPT training process

#115

Post by imtringued »

LaMarseillaise wrote: post #92
stephenboyd wrote: post #88

Earlier quoted context omitted.

I've been spending some time trying to get a sense of how it works by exploring where it fails. When it makes a mistake, you can ask questions in a socratic method until it says the true counterpart to its mistake. It doesn't comment on noticing a discrepancy even if you try to get it to reconcile its previous answer with the corrected version that you guided it to. If you ask specifically about the discrepancy it wi…

> If you ask specifically about the discrepancy it will usually deny the discrepancy entirely or double-down on the mistake. I have had the exact opposite experience. I pasted error messages from code it generated, I corrected its Latin grammar, and I pointed out contradictions in its factual statements in a variety of ways. Every time, it responded with a correction and (the same) apology. This makes me wonder if we…

I pasted error messages from code it generated. It kept generating the same compiler error eventually. When I applied the "socratic method" and explained to it the answer based on stack overflow answers. It would at first pretend to understand by transforming the relevant documentation I inserted into it, but once I asked it the original question, it basically ignored all the progress and kept creating the same code with the same compiler errors.

Re: Open source solution replicates ChatGPT training process

#116

Post by kerpotgh »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

Pretty much everything at work.

Re: Open source solution replicates ChatGPT training process

#117

Post by imtringued »

plutonorm wrote: post #83
chezelenkoooo wrote: post #62

Earlier quoted context omitted.

Your entire post is questionable the moment you write something like "Google as we knew it is already dead". Yeah, no.

Yeah, it is.

I can't even augment ChatGPT with Google results, how can it be a replacement for Google?

Re: Open source solution replicates ChatGPT training process

#118

Post by animuchan »

jackblemming wrote: post #23
simonw wrote: post #19

"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…

Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?

I also don't find it particularly disruptive, but it's a nice interface to search for synonyms / ways to rephrase stuff, much preferable to googling (which is more of a failure on Google's part than anything).

Also, ChatGPT writes simple unit tests and SQL queries very well.

Re: Open source solution replicates ChatGPT training process

#119

Post by mach1ne »

return_to_monke wrote: post #4
supriyo-biswas wrote: post #3

Earlier quoted context omitted.

I, for one, would like to see an open-source model similar to Stable Diffusion, but for text. It would be a great way to empower general folk without having to pay OpenAI, and not have to worry about the LLM's belief system, which is conservative-biased in the case of ChatGPT[1] (HN discussion[2]). [1] https://davidrozado.substack.com/p/openaicms [2] https://news.ycombinator.com/item?id=34625001

there is https://github.com/laion-ai/open-assistant being built in the open already. you can contribute too. please also notice that the article you linked is about the text classifier of the frontend and not the LLM itself

That's what I love about this particular AI revolution. The technologies are developed in such a non-siloed manner that open source is able to replicate the largest steps forward in a manner of a year.

Re: Open source solution replicates ChatGPT training process

#120

Post by mach1ne »

raydiatian wrote: post #60
SunghoYahng wrote: post #32

Earlier quoted context omitted.

Even if it has not so much thing to do with intelligence?

I’m not sure about your definition of intelligence. Perhaps you think I’m saying ChatGPT and generative agents are somehow conscious. I don’t conflate consciousness with intelligence here. I can’t say whether or not ChatGPT is conscious (although I doubt it), but it’s pretty clearly intelligent by a reasonable definition. It’s an agent which is extremely effective at playing its game. A game which is incredibly open…

Just to play the devil's advocate here, you can argue that ChatGPT is not really intelligent. It is one hugely complex probability distribution, true, but importantly, a static one. Human intelligence stretches the similar distribution into the temporal dimension as our brain processing data influences the shape of the distribution real-time. In this manner, human brains are not Chinese Rooms due to their continuous online learning, but ChatGPT is, since the weights could be written to stone and the output would not be any different.
Post reply on HN