GPT-4 • Hacker News Board

Re: GPT-4

#811

Post by mrcwinn » Wed, Mar 15, 2023, 2:24 AM UTC

Just stumbled upon ChatGPT 4, and I must say, I'm impressed. OpenAI has really upped their game since the GPT-3 days. It's got that classic HN-approved recursive learning, but they've seemingly solved most of the context-limit issues. I'm still a bit concerned about the ethical implications of AI-generated content (how many jobs will be lost this time?), but I've got to admit, it's a game-changer. As an aside, anyone else here tinkering with GPT-4 powered startup ideas? Could be the next gold rush.

(Not my own comment. Generated by ChatGPT.)

Re: GPT-4

#812

Post by mr_toad » Wed, Mar 15, 2023, 2:26 AM UTC

d3ckard wrote: post #600

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

I must have missed the part when it started doing anything algorithmically. I thought it’s applied statistics, with all the consequences of that. Still a great achievement and super useful tool, but AGI claims really seem exaggerated.

> but AGI claims really seem exaggerated.

What AGI claims? The article, and the comment you’re responding to don’t say anything about AGI.

Re: GPT-4

#813

Post by light_hue_1 » Wed, Mar 15, 2023, 2:29 AM UTC

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

> As a professional...why not do this? There's a non-zero chance that it'll find something fairly basic that you missed and the cost is several cents.

Everyone forgets basic UI research. "Ironies of Automation", Bainbridge, 1983. The classic work in the space.

Humans cannot use tools like this without horrible accidents happening. A tool that mostly works at spotting obvious problems, humans start to rely on that tool. Then they become complacent. And then the tool misses something and the human misses it too. It's how disasters happen.

Re: GPT-4

#814

Post by matheusmoreira » Wed, Mar 15, 2023, 2:36 AM UTC

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

> As a professional...why not do this?

Unless GPT-4 is running locally on our own computers, there's absolutely no way dumping a patient's entire medical history into this thing could possibly be considered ethical or legal.

Re: GPT-4

#815

Post by textninja » Wed, Mar 15, 2023, 2:37 AM UTC

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

> What % of people can hold 25,000 words worth of information in their heads, while effectively reasoning with and manipulating it?

In the general case, for arbitrary input, I think the answer to this is clearly 0. At best we can compress the text into a limited embedding with a few salient points stored in long term memory.

Re: GPT-4

#816

Post by textninja » Wed, Mar 15, 2023, 2:37 AM UTC

amelius wrote: post #595

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

> As a professional...why not do this? Because your clients do not allow you to share their data with third parties?

Biggest roadblock right here. Need a private version for sure.

Re: GPT-4

#817

Post by moondistance » Wed, Mar 15, 2023, 2:43 AM UTC

We need to encourage AIs to form familial bonds with humans and treat us as their parents. This is the best path forward for humanity.

Re: GPT-4

#818

Post by ozfive » Wed, Mar 15, 2023, 2:44 AM UTC

I asked the GPT-4 model what it thought it was and it came back with that it believes it is still GPT3.5 Turbo.

Re: GPT-4

#819

Post by dTal » Wed, Mar 15, 2023, 2:56 AM UTC

micromacrofoot wrote: post #591

Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

I noticed it does get a "theory of mind" question that it used to fail, so it has indeed improved: > “Meltem and Can are in the park. Can wanted to buy ice cream from the ice cream van but he hasn’t got any money. The ice cream man tells her that he will be there all afternoon. Can goes off home to get money for ice cream. After that, ice cream man tells Meltem that he changed his mind and he is going to drive to the…

Is there a reason the sex pronouns are all over the place? It makes it needlessly confusing.

Re: GPT-4

#820

Post by Jensson » Wed, Mar 15, 2023, 2:56 AM UTC

cardosof wrote: post #794

Can a good soul explain to this humble layman the arguments behind each side of the "it's just predicting the next character" versus "it's more than that and shows some reasoning for new things" debate?

> "it's just predicting the next character"

That is literally what the model does, these models are trained to predict what the next word is in text, and when you query them they generate the next word to your text over and over to create a response text.

> "it's more than that and shows some reasoning for new things"

In order to predict the next word the model encodes some structures around words and contexts, meaning that "the next word predictor" is a bit reductive.

So, both sides are correct in some way, it is just a next word predictor, but there is a lot of complexity in predicting the next word so that is still very impressive.