Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Progress

progress.openai.com

Post reply on HN
Print view View original

181–190 of 372 posts

Re: OpenAI Progress

#181

Post by kevindamm »

reasonableklout wrote: post #148
stavros wrote: post #141

Earlier quoted context omitted.

All the replies are spectacularly wrong, and biased by hindsight. GPT-1 to GPT-2 is where we went from "yes, I've seen Markov chains before, what about them?" to "holy shit this is actually kind of understanding what I'm saying!" Before GPT-2, we had plain old machine learning. After GPT-2, we had "I never thought I would see this in my lifetime or the next two".

I'd love to know more about how OpenAI (or Alec Radford et al.) even decided GPT-1 was worth investing more into. At a glance the output is barely distinguishable from Markov chains. If in 2018 you told me that scaling the algorithm up 100-1000x would lead to computers talking to people/coding/reasoning/beating the IMO I'd tell you to take your meds.

Transformers can train models with much larger parameter sizes compared to other model architectures (with the same amount of compute and time), so it has an evident advantage in terms of being able to scale. Whether scaling the models up to multi-billion parameters would eventually pay out was still a bet but it wasn't a wild bet out of nowhere.

Re: OpenAI Progress

#182

Post by jibal »

A progression of human conversations about AI that are in the training data. (Plus an improved language model, as easily seen from GPT-1.)

Re: OpenAI Progress

#183

Post by mat_b »

ralusek wrote: post #64
simianwords wrote: post #7

My interpretation of the progress. 3.5 to 4 was the most major leap. It went from being a party trick to legitimately useful sometimes. It did hallucinate a lot but I was still able to get some use out of it. I wouldn't count on it for most things however. It could answer simple questions and get it right mostly but never one or two levels deep. I clearly remember 4o was also a decent leap - the accuracy increased su…

The real jump was 3 to 3.5. 3.5 was the first “chatgpt.” I had tried gpt 3 and it was certainly interesting, but when they released 3.5 as ChatGPT, it was a monumental leap. 3.5 to 4 was also huge compared to what we see now, but 3.5 was really the first shock.

This was my experience as well. 3.5 was the point where stackoverflow essentially became obsolete in my workflow.

Re: OpenAI Progress

#184

Post by [deleted] »

reasonableklout wrote: post #148
stavros wrote: post #141

Earlier quoted context omitted.

All the replies are spectacularly wrong, and biased by hindsight. GPT-1 to GPT-2 is where we went from "yes, I've seen Markov chains before, what about them?" to "holy shit this is actually kind of understanding what I'm saying!" Before GPT-2, we had plain old machine learning. After GPT-2, we had "I never thought I would see this in my lifetime or the next two".

I'd love to know more about how OpenAI (or Alec Radford et al.) even decided GPT-1 was worth investing more into. At a glance the output is barely distinguishable from Markov chains. If in 2018 you told me that scaling the algorithm up 100-1000x would lead to computers talking to people/coding/reasoning/beating the IMO I'd tell you to take your meds.

[deleted]

Re: OpenAI Progress

#185

Post by jibal »

simianwords wrote: post #87
collingreen wrote: post #71

Earlier quoted context omitted.

Without some exploratory fact checking how do you estimate how high the accuracy is and how often you should be fact checking to maintain a good understanding?

I did initial tests so that I don't have to do it anymore.

Everyone else has done tests that indicate that you do.

Re: OpenAI Progress

#186

Post by energy123 »

gordon_freeman wrote: post #122

It seems like the progress from GPT-4 to GPT-5 has plateaued: for most prompts, I actually find GPT-4 more understandable than GPT-5 [1]. [1] Read the answers from GPT-4 and 5 for this math question: "Ugh I hate math, integration by parts doesn't make any sense"

Basic prose is a saturated bench. You can't go above 100% so by definition progress will stall on such benchmarks.

Re: OpenAI Progress

#187

Post by energy123 »

mattw1810 wrote: post #61

On the whole GPT-4 to GPT-5 is clearly the smallest increase in lucidity/intelligence. They had pre-training figured out much better than post-training at that point though (“as an AI model” was a problem of their own making). I imagine the GPT-4 base model might hold up pretty well on output quality if you’d post-train it with today’s data & techniques (without the architectural changes of 4o/5). Context size & pric…

Basic prose is a saturated bench. You can't go above 100% so by definition progress will stall on such benchmarks.

Re: OpenAI Progress

#188

Post by ThrowawayR2 »

bryant wrote: post #165
willguest wrote: post #156

My go-to for any big release is to have a discussion about self-awareness and dive in to constuctivist notions of agency and self-knowing from a perspective of intelligence that is not limited to human cognitive capacity. I start with a simple question "who are you?". The model then invariably compares itself to humans, saying how it is not like us. I then make the point that, since it is not like us, how can it clai…

> to orient toward the unfolding of possibility in others This is a globally unique phrase, with nothing coming close other than this comment on the indexed web. It's also seemingly an original idea as I haven't heard anyone come close to describing a feeling (love or anything else) quite like this. Food for thought. I'm not brave enough to draw a public conclusion about what this could mean.

Except "unfolding of possibility", as an exact phrase, seems to have millions of search hits, often in the context of pseudo-profound spiritualistic mumbo-jumbo like what the LLM emitted above. It's like fortune cookie-level writing.

Re: OpenAI Progress

#189

Post by jibal »

bryant wrote: post #165
willguest wrote: post #156

My go-to for any big release is to have a discussion about self-awareness and dive in to constuctivist notions of agency and self-knowing from a perspective of intelligence that is not limited to human cognitive capacity. I start with a simple question "who are you?". The model then invariably compares itself to humans, saying how it is not like us. I then make the point that, since it is not like us, how can it clai…

> to orient toward the unfolding of possibility in others This is a globally unique phrase, with nothing coming close other than this comment on the indexed web. It's also seemingly an original idea as I haven't heard anyone come close to describing a feeling (love or anything else) quite like this. Food for thought. I'm not brave enough to draw a public conclusion about what this could mean.

It's not at all an original idea. The wording is uniquely stilted.

Re: OpenAI Progress

#190

Post by reilly3000 »

GPT-5’s question about consciousness and its use of sibling seems to indicate there is some underlying self awareness in the system prompt, and that has perhaps contains concepts of consciousness. If not, where is that coming from? Recent training data containing more glurge?
Post reply on HN