Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 Designed a Programming Language

lukebechtel.com

Post reply on HN
Print view View original

11–20 of 338 posts

Re: GPT-4 Designed a Programming Language

#11

Post by HervalFreire »

quotation from the link:

   "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and foresight than much of the internet believes LLMs to have.

   Some folks on the internet tend to think lowly of LLMs as merely "Stochastic Parrots" -- simply "remixers" of old ideas. In a way, they're right; but I'd argue that remixing is the fundamental force of creativity. Nothing comes from scratch."
This^^. Half the internet was in denial about how good LLMs are. I wonder why? Perhaps it's the way humans act when confronted with a machine that can potentially replace much of what makes us special.

The advent of chatGPT had millions of people on the internet trying to downplay the intelligence of chatGPT by continuously trying to re-emphasize the things it gets wrong.

I think the people who claimed chatGPT was a stochastic parrot are now realizing that they were the ones that were part of a giant parade of parrots regurgitating the same old tired trope of LLMs being nothing but simple word generators. Well you guys were dead wrong, a simple iterative improvement on the same algorithm pretty much ironed out a lot of the issues.

I'm guessing that at GPT-10 or 11 we'll have produced something that when compared with humans, humans will be the things that are far more parrot-like.

Re: GPT-4 Designed a Programming Language

#13

Post by swatcoder »

HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

I was reading the quote you included and was sure you were going to lambast him for his arrogance (nobody else has asked it to design a language? hah!) and for bringing an elephant sized straw man to the depthful analytic discussion many of us are trying to have about this new technology, but you went a different way with it. Huh.

Re: GPT-4 Designed a Programming Language

#14

Post by kaczordon »

HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

Future data will still have to come from someone e.g. a person.

Re: GPT-4 Designed a Programming Language

#15

Post by raincole »

HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

I'm not so sure. The first sample is:

> Indentation-based scoping, similar to Python.

Then...

> class Person {

To me, it's very clear it doesn't really understand what "indentation-based scoping" means. For a human programmer, using both intentation-based and bracket-based scoping is a very unusual design choice, and is begging for further explanation. It talked pages of abstraction, modularity, etc, but not this?

I'm not saying it's not good. But I failed to see how this particular example proves its not a "stochastic parrot". If anything, I think it's a quite strong evidence supporting "stochastic parrot" narrative.

Re: GPT-4 Designed a Programming Language

#16

Post by marviel »

raincole wrote: post #15
HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

I'm not so sure. The first sample is: > Indentation-based scoping, similar to Python. Then... > class Person { To me, it's very clear it doesn't really understand what "indentation-based scoping" means. For a human programmer, using both intentation-based and bracket-based scoping is a very unusual design choice, and is begging for further explanation. It talked pages of abstraction, modularity, etc, but not this ? I…

Author here -- I think it is a stochastic parrot, don't get me wrong.

But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots. At least, as it pertains to many creative endeavors.

You're always building on the back of something that comes before.

We're all just remixing ideas we've heard before.

I mean even this comment -- none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original?

Not really. We all need eachother's creative energies to do our best work.

Re: GPT-4 Designed a Programming Language

#18

Post by marviel »

swatcoder wrote: post #13
HervalFreire wrote: post #11

quotation from the link: "LLMs in the GPT family have read every programming language in the world, billions of times. It's been known for some time that these agents can write small snippets of code with some limited success. However, to my knowledge, none of these agents have ever been asked to make their own programming language. This type of task is a bit more complex, and requires a bit more creativity and fores…

I was reading the quote you included and was sure you were going to lambast him for his arrogance (nobody else has asked it to design a language? hah!) and for bringing an elephant sized straw man to the depthful analytic discussion many of us are trying to have about this new technology, but you went a different way with it. Huh.

Wasn't trying to be arrogant, I just wasn't aware of an existing implementation and thought it would be fun to try. :)

Do you have links to other efforts? I'd love to know more about it, and would be happy to add references to other existing literature to the article if it improved its quality.

Re: GPT-4 Designed a Programming Language

#19

Post by aledalgrande »

My experience with it programming has been atrocius. It keeps making the same mistakes, writing NPEs everywhere and still doing it after I tell it many times how it was wrong (short memory?). Copilot is backed by it too now and I had a better autocomplete of constants with Sorbet than Copilot. Again, in my experience it's not usable if not for the super mainstream and simplest tasks.

The biggest issue is that it writes this with utmost confidence, at least on the surface, because there is no measure of how confident it is returned by the model.

Re: GPT-4 Designed a Programming Language

#20

Post by brucethemoose2 »

Jensson wrote: post #9

This actually isn't a bad idea, GPT takes commonly talked about things and distils them into a response to you, so the language you get when you ask it to make a programming language is probably a mix of well liked programming features that many people added to a language they themselves wanted. When you design a language on your own you will be too biased towards your own cases, this is a way to get around that and…

Nah, GPT has its own biases too.

Its preference for Pythonic/JS syntax is a simple but good example, as those are likely the most popular languages in its dataset.

Post reply on HN