Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an interpreter for my programming language with ChatGPT

6502.is-a.dev

Post reply on HN
Print view View original

21–30 of 141 posts

Re: Building an interpreter for my programming language with ChatGPT

#21

Post by AkshatJ27 »

DeWilde wrote: post #19
phtrivier wrote: post #15

I've seen a couple of articles like this, but they never show the previous prompts to give some awareness of the language, tools, etc... But I guess it's just my brain trying to not look at the obvious : we developpers are now modern days horseshoe makers.

> we developpers are now modern days horseshoe makers. Explain?

As explained by GPT:

> The comment seems to be expressing the idea that developers are similar to horseshoe makers in that their work, like the work of horseshoe makers, is becoming increasingly obsolete or irrelevant.

Re: Building an interpreter for my programming language with ChatGPT

#22

Post by dunefox »

DeWilde wrote: post #19
phtrivier wrote: post #15

I've seen a couple of articles like this, but they never show the previous prompts to give some awareness of the language, tools, etc... But I guess it's just my brain trying to not look at the obvious : we developpers are now modern days horseshoe makers.

> we developpers are now modern days horseshoe makers. Explain?

Near obsolete.

Re: Building an interpreter for my programming language with ChatGPT

#23

Post by dunefox »

mudrockbestgirl wrote: post #9
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

The problem is that you cannot trust the output. It's often wrong, but in subtle nonobvious ways. For precise information you still need to check the sources to make sure what you're getting is correct. You can test it out with a (not-so-mainstream) topic that you're an expert in. You'll see lots of mistakes that are obvious to you, but wouldn't be obvious to non-experts. But it's an incredible tool for brainstorming…

So, it's exactly like Google search but more interactive?

Re: Building an interpreter for my programming language with ChatGPT

#24

Post by visarga »

dom96 wrote: post #8

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript). This makes me wonder how much time…

You didn't start with "sudo mode: on" did you? That's what happens when you don't have the right privilege level.

Re: Building an interpreter for my programming language with ChatGPT

#25

Post by DeWilde »

AkshatJ27 wrote: post #21
DeWilde wrote: post #19

Earlier quoted context omitted.

> we developpers are now modern days horseshoe makers. Explain?

As explained by GPT: > The comment seems to be expressing the idea that developers are similar to horseshoe makers in that their work, like the work of horseshoe makers, is becoming increasingly obsolete or irrelevant.

My feeling is that people won't be writing code as it is written now. Those that do will likely be the same as horseshoe makers. Instead, you will have new tools and be solving different problems. As mechanics now have taken the role of those in charge of maintaining our primary transportation tool.

Re: Building an interpreter for my programming language with ChatGPT

#26

Post by visarga »

mudrockbestgirl wrote: post #9
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

The problem is that you cannot trust the output. It's often wrong, but in subtle nonobvious ways. For precise information you still need to check the sources to make sure what you're getting is correct. You can test it out with a (not-so-mainstream) topic that you're an expert in. You'll see lots of mistakes that are obvious to you, but wouldn't be obvious to non-experts. But it's an incredible tool for brainstorming…

You can trust it. Look again at the OP. They fed the whole language spec into chatGPT, only after that it became capable of coding in their language. If you ask people to solve tasks without references you will see a similar drop in ability.

The trick is to feed relevant contextual information instead of using it closed-book. This can be automated with a search engine, or can be a deliberate manual process. But closed-book mode is not the right way to assess people or AIs.

What are your counter arguments?

Re: Building an interpreter for my programming language with ChatGPT

#27

Post by nonameiguess »

habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

Is there a continuous retrain mode? From other articles, I was under the impression this thing doesn't know the current state of the world, just the slice of the world at a snapshot in time represented by its training set. I'm generally not going to a search engine to find the hours of my local pharmacy from two years ago when the search assistant learned human language. I want the hours for today.

Re: Building an interpreter for my programming language with ChatGPT

#28

Post by habibur »

babuskov wrote: post #18
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

> ChatGPT will be the google killer This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun? ChatGPT answer: > The average distance between the Sun and its planets is approximately 5% of the Sun's diameter. If you run into bunch of such answers, you will soon stop trusting it for anything. As for Google - it doesn't even try to answer the question. But no answer is oft…

Google, Wikipedia, and historically the internet had been unreliable for many of the queries. More so in the beginning. But over time most of the errors are ironed out. ChatGPT is a very promising start I would say.

In the end convenience wins out at cost of correctness. The proportion matters, convenience::correctness.

Re: Building an interpreter for my programming language with ChatGPT

#29

Post by visarga »

babuskov wrote: post #18
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

> ChatGPT will be the google killer This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun? ChatGPT answer: > The average distance between the Sun and its planets is approximately 5% of the Sun's diameter. If you run into bunch of such answers, you will soon stop trusting it for anything. As for Google - it doesn't even try to answer the question. But no answer is oft…

For the same question I got:

> The distances between the Sun and its planets vary greatly. For example, the distance from the Sun to Mercury is about 36 million miles, while the distance from the Sun to Pluto is about 3.6 billion miles. The size of the Sun is about 864,000 miles in diameter, which means that the distance from the Sun to its planets is much greater than the size of the Sun itself.

Then it says something that doesn't follow, probably answering the more frequent question of relative sizes of the planets compared to the Sun

> To put it into perspective, if the Sun were the size of a basketball, the nearest planet, Mercury, would be about the size of a grain of sand, and Pluto would be about the size of a golf ball.

Re: Building an interpreter for my programming language with ChatGPT

#30

Post by tazjin »

dunefox wrote: post #23
mudrockbestgirl wrote: post #9

Earlier quoted context omitted.

The problem is that you cannot trust the output. It's often wrong, but in subtle nonobvious ways. For precise information you still need to check the sources to make sure what you're getting is correct. You can test it out with a (not-so-mainstream) topic that you're an expert in. You'll see lots of mistakes that are obvious to you, but wouldn't be obvious to non-experts. But it's an incredible tool for brainstorming…

So, it's exactly like Google search but more interactive?

Google doesn't hallucinate completely fictitious results.

It will however index hallucinated results generated with GPT and published somewhere, so once we're at that point it really doesn't matter anymore.

Post reply on HN