Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4

openai.com

Post reply on HN
Print view View original

341–350 of 1001 posts

Re: GPT-4

#341

Post by carapace »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

First. connect them to empirical feedback devices. In other words, make them scientists.

Human life on Earth is not that hard (think of it as a video game.) Because of evolution, the world seems like it was designed to automatically make a beautiful paradise for us. Literally, all you have to do to improve a place is leave it alone in the sun with a little bit of water. Life is exponential self-improving nano-technology.

The only reason we have problems is because we are stupid, foolish, and ignorant. The computers are not, and, if we listen to them, they will tell us how to solve all our problems and live happily ever after.

Re: GPT-4

#342

Post by cjrd »

Let's check out the paper for actual tech details!

> Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.

- OpenAI

Re: GPT-4

#343

Post by gremlinsinc »

minimaxir wrote: post #98
teruakohatu wrote: post #26

Access is invite only for the API, and rate limited for paid GPT+. > gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k, which will also be updated automatically over time (current version gpt-4-32k-0314, also supported until June 14). Pricing is $0.06 per 1K prompt tokens and $0.12 per 1k completion tokens. The context le…

$0.12 per 1k completion tokens is high enough that it makes it prohibitively expensive to use the 32k context model. Especially in a chatbot use case with cumulative prompting, which is the best use case for such a large context vs. the default cheaper 8k window. In contrast, GPT-3.5 text-davinci-003 was $0.02/1k tokens, and let's not get into the ChatGPT API.

can't you combine instances of 4k tokens in 3.5 to fake it? having one gpt context per code file, for instance and maybe some sort of index?

I'm not super versed on lang chain but that might be kinda what that solves...

Re: GPT-4

#344

Post by option »

“ Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.” - HUGE step backwards.

Re: GPT-4

#345

Post by theodorejb »

Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

> It's not clear to me if the lesson here is that GPT's reasoning capabilities are being masked by an incorrect prior (having memorized the standard version of this puzzle) or if the lesson is that GPT'S reasoning capabilities are always a bit of smoke and mirrors that passes off memorization for logic.

It's a lot closer to the latter. GPT doesn't have "reasoning capabilities", any more than any other computer program. It doesn't have a clue what any of its input means, nor the meaning of the text it outputs. It just blindly spits out the words most probable to follow the prompt, based on its corpus of training data and the weights/biases added to fine tune it. It can often do a good job at mimicking reasoning, but it's not.

Re: GPT-4

#346

Post by valine »

albertgoeswoof wrote: post #294
Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

LLMs aren’t reasoning about the puzzle. They’re predicting the most likely text to print out, based on the input and the model/training data. If the solution is logical but unlikely (i.e. unseen in the training set and not mapped to an existing puzzle), then the probability of the puzzle answer appearing is very low.

How do you know the model isn’t internally reasoning about the problem? It’s a 175B+ parameter model. If, during training, some collection of weights exist along the gradient that approximate cognition, then it’s highly likely the optimizer would select those weights over more specialized memorization weights.

It’s also possible, likely even, that the model is capable of both memorization and cognition, and in this case the “memorization neurons” are driving the prediction.

Re: GPT-4

#347

Post by bullfightonmars »

nealabq wrote: post #227

Test taking will change. In the future I could see the student engaging in a conversation with an AI and the AI producing an evaluation. This conversation may be focused on a single subject, or more likely range over many fields and ideas. And may stretch out over months. Eventually teaching and scoring could also be integrated as the AI becomes a life-long tutor. Even in a future where human testing/learning is no l…

We are entering the age of "Young Lady's Illustrated Primer" from The Diamond Age by Neal Stephenson. Is this going to turn into a true digital assistant, that knows you, what you need, how to teach you new things, and how to help you achieve your goals?

Re: GPT-4

#348

Post by LeanderK »

minimaxir wrote: post #98
teruakohatu wrote: post #26

Access is invite only for the API, and rate limited for paid GPT+. > gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k, which will also be updated automatically over time (current version gpt-4-32k-0314, also supported until June 14). Pricing is $0.06 per 1K prompt tokens and $0.12 per 1k completion tokens. The context le…

$0.12 per 1k completion tokens is high enough that it makes it prohibitively expensive to use the 32k context model. Especially in a chatbot use case with cumulative prompting, which is the best use case for such a large context vs. the default cheaper 8k window. In contrast, GPT-3.5 text-davinci-003 was $0.02/1k tokens, and let's not get into the ChatGPT API.

> $0.12 per 1k completion tokens is high enough that it makes it prohibitively expensive to use the 32k context model.

this is a lot. I bet there's a quite a bit of profit in there

Re: GPT-4

#349

Post by teruakohatu »

nealabq wrote: post #334
teruakohatu wrote: post #26

Access is invite only for the API, and rate limited for paid GPT+. > gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k, which will also be updated automatically over time (current version gpt-4-32k-0314, also supported until June 14). Pricing is $0.06 per 1K prompt tokens and $0.12 per 1k completion tokens. The context le…

> Image inputs are still a research preview and not publicly available. Will input-images also be tokenized? Multi-modal input is an area of research, but an image could be converted into a text description (?) before being inserted into the input stream.

My understanding is thta the image embedding is included, rather than converting to text.

Re: GPT-4

#350

Post by [deleted] »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

[deleted]
Post reply on HN