Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using ChatGPT to generate a GPT project end-to-end

github.com

Post reply on HN
Print view View original

201–210 of 225 posts

Re: Using ChatGPT to generate a GPT project end-to-end

#201

Post by robbintt »

comfypotato wrote: post #98
c7b wrote: post #56

The technological singularity is approaching fast. It's pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively create new GPT models (either by writing and executing Pyhton code, or later versions trained on LLM weights may even be able to output new weights directly). If the inner workings of the loop are sufficiently obfuscated (in the code-based version), it wouldn't nec…

You’re clear in your use of a singularity definition. I’ve always taken the (I think more popular?) stance that the singularity is better defined by AGI surpassing average or best human intelligence. This definition is still far off. The lacking aspects of end-to-end functionality in LLMs may or may not ever see completion. There are paradigms shifts necessary for many applications.

It seems like there is some sort of "viral singularity" that is quite likely. Still a "paperclip maximizer" where paperclip is something unexpected. In a sense life, and humanity, are types of paperclip maximizers.

Re: Using ChatGPT to generate a GPT project end-to-end

#202

Post by ineedasername »

srcreigh wrote: post #109
hnlmorg wrote: post #66

Earlier quoted context omitted.

I see this as an example of the reverse: AI is still stupid enough that it takes humans a degree of skill to craft a request which generates the desired output.

Kolmogorov complexity would like a word! It seems intractable for AI to read minds, there should always be some degree of skill involved in prompt writing.

Agreed. But I think we may reach a point where it is difficult to prompt the most advance AI (which may are may note be an LLM, though maybe contain an LLM mode) in a productive way, especially due to computational demands on resources. And so AI's with lesser, cheaper capabilities may be reasonably competent in "understanding"-- a term I user very loosely-- the problem of dealing with more advanced but resource constrained systems and assist in the best practices in prompting them.

Re: Using ChatGPT to generate a GPT project end-to-end

#203

Post by MattRix »

rapiz wrote: post #133
MattRix wrote: post #129

Earlier quoted context omitted.

Ok but this is extremely new tech, all of that stuff will get better over time, and the AI will require less and less intervention.

I don't think so. Ultimately there's not enough information in prompts to produce "correct" code. And any attempt to deliver more information will result in a worse programming language, or as it is now, more iterations.

Many high quality human programmers could go off and make a very good program from a simple description/prompt. I see no reason an LLM couldn’t do the same.

On top of that, there’s no reason an AI couldn’t ask additional questions to clarify certain details, just like a human would. Also as this tech gets faster, the iteration process will get more rapid too, where a human can give small bits of feedback to modify the “finished product” and get the results in seconds.

Re: Using ChatGPT to generate a GPT project end-to-end

#205

Post by SanderNL »

int_19h wrote: post #179
choeger wrote: post #138

Earlier quoted context omitted.

It clearly does not reason. Take a famous riddle and make a paradox change. It will not create a meaningful response. But yes, there is a lot of knowledge embedded into our global use of language and it is fascinating to see how it can be reproduced by such a model.

Does the following satisfy your requirement for "a famous riddle with a paradox change"? Because GPT-4 aces it most of the time. "Doom Slayer needs to teleport from Phobos to Deimos. He has his pet bunny, his pet cacodemon, and a UAC scientist who tagged along. The Doom Slayer can only teleport with one of them at a time. But if he leaves the bunny and the cacodemon together alone, the bunny will eat the cacodemon. A…

People will see patterns in this riddle and claim it is “just” altering those. “It’s just a bunch a patterns where you can switch the names, like templates”.

Isn’t everything like that?

“Uhh…”

I had the same discussions about chess.

“It has just memorized a bunch of high level patterns and juggles them around”.

I agree, but now I’m curious what you think chess is.

“Chess is not intelligence.”

Goalposts? Anyway, we move on to Go, the game. Same response. Programming, same, but the angle of the response is different now because programming is “clearly” intelligence incarnate.

“It programs and sometimes correctly, but it is a mirage. It will never attain True Programming.”

I’m sitting on the bench riding this one out. We’ll see.

Re: Using ChatGPT to generate a GPT project end-to-end

#206

Post by sekai »

zerr wrote: post #192
seunosewa wrote: post #131

Earlier quoted context omitted.

ChatGPT is a junior developer whose knowledge is broad but shallow.

The worst part is that it doesn't know when it doesn't know, so makes up a garbage.

> The worst part is that it doesn't know when it doesn't know, so makes up a garbage.

Hehe, that's true for humans too.

Re: Using ChatGPT to generate a GPT project end-to-end

#207

Post by travisjungroth »

generic92034 wrote: post #128
travisjungroth wrote: post #124

Earlier quoted context omitted.

You have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.

That could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).

My point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them.

Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions.

The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.

Re: Using ChatGPT to generate a GPT project end-to-end

#208

Post by dhoe »

nklrta wrote: post #23
dhoe wrote: post #15

Earlier quoted context omitted.

It's a spot on - moving up a level in the abstraction hierarchy.

Human: ChatGPT, please devise an algorithm that solves the traveling salesman problem in polynomial time. A: Certainly, as an AI language model I'm happy to oblige. Here is the algorithm. Human: I used a higher abstraction level! I have solved P == NP!

Just fyi, here's how it responds to that prompt:

"As an AI language model, I'm unable to create or prove the existence of a polynomial-time algorithm for the Traveling Salesman Problem (TSP), because it is an NP-hard problem. This means that it is unlikely that such an algorithm exists, unless P = NP, which is an unsolved question in computer science.

However, I can suggest some approximation algorithms or heuristics that provide good solutions to the problem, even though they don't guarantee an optimal solution:"

Re: Using ChatGPT to generate a GPT project end-to-end

#209

Post by victor9000 »

addandsubtract wrote: post #180
killthebuddha wrote: post #151

Earlier quoted context omitted.

IMO this leaves out some salient details. For example, I'd say ChatGPT is a very, very good junior developer. The kind of junior developer that loves computer science, has been screwing around with miscellaneous algorithms and data structures its whole life, has a near-perfect memory, and is awake 24/7/365, but has never had to architect a data-intensive system, write future-proof code, or write code for other develo…

It also has a very broad knowledge of programming languages and frameworks. It's able to onboard you with ease and answer most of qour questions. The trick is to recognize when it's confidently incorrect and hallucinating API calls.

But that's the secret, it's always hallucinating

Re: Using ChatGPT to generate a GPT project end-to-end

#210

Post by zerr »

AndrewKemendo wrote: post #204
drekipus wrote: post #196

Earlier quoted context omitted.

i've seen this in juniors as well.

And unfortunately just as many seniors

Well, besides the prose, ChatGPT generates a perfectly valid looking code mashup of e.g. Qt, wxWidgets and its hallucinations on top of that. Humans don't do that :)
Post reply on HN