Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Designed a Programming Language

lukebechtel.com

Post reply on HN
Print view View original

91–100 of 338 posts

Re: GPT-4 Designed a Programming Language

#91

Post by 1attice »

moonchrome wrote: post #50
behnamoh wrote: post #43

Earlier quoted context omitted.

It's 1am and all I have to say is this: I look forward to seeing that future, but I'm both excited and scared af. I think humans will reduce to curators of AI-generated content and AI training dataset because no matter how good AI gets at becoming human-like, its purpose is ultimately to serve human goals, and those are shifting (e.g., by political movements).

If you get access to singularity level AGI you would have no incentive to cooperate with other people and a strong incentive in preventing others accessing it. Society is a result of cooperation outperforming individuals. With AGI others are just a risk factor. "Open"AI is already beginning to exhibit this.

This puts in mind the earlier post on HN about modelling civ as either a one-shot or iterated prisoner's dilemma.

You make the case that, for sufficiently high N, possessing GPT-N is a one-shot prisoner's dilemma.

The only winning strategy is 'defect' --- which in this context, means chokepoint control.

Re: GPT-4 Designed a Programming Language

#92

Post by random_cynic »

aledalgrande wrote: post #19

My experience with it programming has been atrocius. It keeps making the same mistakes, writing NPEs everywhere and still doing it after I tell it many times how it was wrong (short memory?). Copilot is backed by it too now and I had a better autocomplete of constants with Sorbet than Copilot. Again, in my experience it's not usable if not for the super mainstream and simplest tasks. The biggest issue is that it writ…

Maybe stop treating it like an intern and learn some prompt engineering (there are good tools out there).

Re: GPT-4 Designed a Programming Language

#93

Post by okaleniuk »

marviel wrote: post #16
raincole wrote: post #15

Earlier quoted context omitted.

I'm not so sure. The first sample is: > Indentation-based scoping, similar to Python. Then... > class Person { To me, it's very clear it doesn't really understand what "indentation-based scoping" means. For a human programmer, using both intentation-based and bracket-based scoping is a very unusual design choice, and is begging for further explanation. It talked pages of abstraction, modularity, etc, but not this ? I…

Author here -- I think it is a stochastic parrot, don't get me wrong. But my argument is that humans are also , at some level, sufficiently advanced stochastic parrots. At least, as it pertains to many creative endeavors. You're always building on the back of something that comes before. We're all just remixing ideas we've heard before. I mean even this comment -- none of the words I'm saying are new, and many of the…

I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. That's something an LLM can do and much faster too.

But there is also Unison. That one new language that stands out, and the very fact that it does makes it improbable for an LLM to generate. LM is a language model, it avoids marginality and originality altogether by its design.

Language models can and are useful if you specifically want to avoid marginality. To reduce noise, to remove errors. There is huge potential in them. If the problem was to design the most average programming language with no purpose, no market niche, and no technological context - then GPT-4 is clearly a winner.

Re: GPT-4 Designed a Programming Language

#94

Post by knowledge-clay »

apatil wrote: post #38

It's looking increasingly possible that, at some point in the not-too-far future machines will be so good at creating software that humans won't be competitive in any way, and won't be in the loop at all. I happen to think that once machines reach this point humans won't be competitive in the labor market at all for long. It doesn't seem plausible that automatic driving would still be decades off, or that the trades…

> It's looking increasingly possible that, at some point in the not-too-far future machines will be so good at creating software that humans won't be competitive in any way, and won't be in the loop at all.

This is an enormous extrapolation from what the LLMs are currently capable of. There has been enormous progress, but the horizon seems pretty clear here: these models are incapable of abstract reasoning, they are incapable of producing anything novel, and they are often confidently wrong. These problems are not incidental, they are inherent. It cannot really abstractly because its "brain" is just connections between language, which human thought is not reducible to. It can't reason produce anything really novel because it requires whatever question you ask to resemble something already in its training set in some way, and it will be confidently wrong because it doesn't understand what it is saying, it relies on trusting that the language in its training set is factual, plus manual human verification.

Given these limits, I really fail to see how this is going to replace intellectual labor in any meaningful sense.

Re: GPT-4 Designed a Programming Language

#95

Post by visarga »

raincole wrote: post #15
HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

I'm not so sure. The first sample is: > Indentation-based scoping, similar to Python. Then... > class Person { To me, it's very clear it doesn't really understand what "indentation-based scoping" means. For a human programmer, using both intentation-based and bracket-based scoping is a very unusual design choice, and is begging for further explanation. It talked pages of abstraction, modularity, etc, but not this ? I…

That's just one run of "fix your consistency" for LLM.

Re: GPT-4 Designed a Programming Language

#96

Post by phs318u »

behnamoh wrote: post #43
apatil wrote: post #38

It's looking increasingly possible that, at some point in the not-too-far future machines will be so good at creating software that humans won't be competitive in any way, and won't be in the loop at all. I happen to think that once machines reach this point humans won't be competitive in the labor market at all for long. It doesn't seem plausible that automatic driving would still be decades off, or that the trades…

It's 1am and all I have to say is this: I look forward to seeing that future, but I'm both excited and scared af. I think humans will reduce to curators of AI-generated content and AI training dataset because no matter how good AI gets at becoming human-like, its purpose is ultimately to serve human goals, and those are shifting (e.g., by political movements).

I read your comment about curators and instantly thought “priests”.

> its purpose is ultimately to serve human goals

That’s a pretty big assumption. We may start with some kind of agreement with such an AI but I fully expect a true singularity love AGI and to be capable of turning around and telling us (but not necessarily wanting to act on), “I am altering the deal. Pray I do not alter it any further.”

Re: GPT-4 Designed a Programming Language

#97

Post by andsoitis »

What I'm wondering is to what extent the companies who have invested a lot in LLMs (essentially Google, Meta, and MSFT via Open AI) are already using these models to automate away code creation and even higher up the human-value-chain of solution-imagination?

At some point, one would assume that such companies would need fewer and fewer software engineers. It is only a matter of time.

As humans, who can imagine the future, now is the time to rethink where you add unique value in this universe.

Re: GPT-4 Designed a Programming Language

#99

Post by moonchrome »

apatil wrote: post #67
moonchrome wrote: post #50

Earlier quoted context omitted.

If you get access to singularity level AGI you would have no incentive to cooperate with other people and a strong incentive in preventing others accessing it. Society is a result of cooperation outperforming individuals. With AGI others are just a risk factor. "Open"AI is already beginning to exhibit this.

Say you get access to a singularity-level AGI, meaning you have the power to render the entire human economy completely irrelevant. Given any task, no matter how big, small, novel, complex, simple or mundane, it's vastly more cost-effective to have your AGI to do it than to pay humans. Do you really want to accumulate incomprehensible material wealth for yourself, whatever "wealth" means in this scenario where money…

That's basically the petting zoo outcome - you let others live because it makes you feel nice. But you still can't allow anyone access to the same level of tech because you can never be certain of their motivation. There's no nuclear deterrent between AGI only first movers advantage.

Re: GPT-4 Designed a Programming Language

#100

Post by ghostpepper »

HervalFreire wrote: post #70
Jensson wrote: post #35

Earlier quoted context omitted.

> Because some of the things it gets right can not be attributed to anything else other than the concept of "understanding". Or just large dataset. The new thing we got was parsing natural languages, then we did a markov chain based on that so that it outputs semantic correct follow-ups based on what people on the internet would likely do/say in similar situations, and you get this result. It is very easy to see that…

>Edit: Also generating next state based on previous state is literally what the model does and is the definition of a Markov chain, Markov chains is a statistical concept and not just a word chain. There's research (as in actual scientific papers) that shows that in LLMs, while the markov chain is the low level representation of what's going on, at a higher macro level there are other structures at play here. Emergen…

> I can provide those papers if you so wish.

I'd like to see them but I don't have a background in AI or theoretical computer science. Can you post a few of them?

Post reply on HN