Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an interpreter for my programming language with ChatGPT

6502.is-a.dev

Post reply on HN
Print view View original

11–20 of 141 posts

Re: Building an interpreter for my programming language with ChatGPT

#11

Post by agumonkey »

dom96 wrote: post #8

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript). This makes me wonder how much time…

it revived my imposter syndrom because I see all the cool tricks people come up with naturally while I get half of what you describe and half easy naive answers :)

Re: Building an interpreter for my programming language with ChatGPT

#12

Post by tromp »

I tried to settle a Millenium problem with ChatGPT:

Is P = NP ?

> No, P is not equal to NP. P (polynomial time) is a subset of NP (nondeterministic polynomial time), meaning that there are certain problems that can be solved in polynomial time that cannot be solved in nondeterministic polynomial time.

Good start, although one should say P is conjectured to be not equal to NP. Some serious misunderstanding in that last part though.

Re: Building an interpreter for my programming language with ChatGPT

#13

Post by ookdatnog »

klohto wrote: post #2

Of all the examples, this is honestly the least impressive because the "Solved AoC 2022" answer is just extrapolated from the previous prompt. Your language largely shares syntax with Python, making the job easier. Interpreting code on unknown input isn't in its current power. I'm guessing with a good prompt you could get there. For example, generating python code for Day 1 is easy, when you give it the input example…

Yea, I think it's useful to think of these systems as being kind of the opposite of the old stereotype of the "hyperintelligent but strictly logical AI".

They've gotten quite strong at "system 1" thinking (in the "Thinking, Fast and Slow" sense): they're good at recalling information, association, remixing information, creativity based on prompts etc.

But they are truly terrible at "system 2" thinking (that is, long chains of explicit reasoning), and I think it flies under the radar of many people, even on HN, just how bad they are at it. Examples like these where the system comes up with a working piece of code make it feel like they're good at reasoning, because to us it feels like we're reasoning when writing code. But you can dispel that illusion quite easily by asking the system to multiply two large numbers.

> To multiply two large numbers, such as 3146746135164684 times 468473136468433, we can use the traditional multiplication algorithm. This algorithm involves breaking the numbers down into smaller groups of digits, multiplying each group by the corresponding group of digits in the other number, and then adding up the products to find the final result.

> Here is an example of how to multiply 3146746135164684 times 468473136468433 using the traditional multiplication algorithm:

> Break the numbers down into groups of digits. For example, we could group the digits of 3146746135164684 as 314, 6746, 135, 1646, 84 and group the digits of 468473136468433 as 4684, 7313, 6468, 433.

> Multipply each group of digits by the corresponding group of digits in the other number. For example, we would multiply 314 times 4684 to get 1470196, 6746 times 7313 to get 49470158, 135 times 6468 to get 873980, 1646 times 433 to get 708338, and 84 times 1 to get 84.

> Add up the products to find the final result. In this case, the final result would be 1470196 + 49470158 + 873980 + 708338 + 84 = 146845556.

> Therefore, the result of multiplying 3146746135164684 times 468473136468433 using the traditional multiplication algorithm is 146845556.

Re: Building an interpreter for my programming language with ChatGPT

#14

Post by grashalm »

schmorptron wrote: post #7
grashalm wrote: post #6

Earlier quoted context omitted.

For ChatGPT to be the Google killer they need to provide source URLs.

It looks like it has web browsing support built-in in some form, but it's disabled at the moment. That said, I'm skeptical that it'd be able to "disrupt" google, as the track record of things that are said to do that is quite bad. On the other hand, google seems to be heading in the same direction with projects such as Lamda. In a roundabout way, this might just end up being the quick answer box at the top of search…

Yes it seems like an exact match for Google's top box. But as mentioned, they need to work on explaining themselves. That is what the current top box still does better.

Re: Building an interpreter for my programming language with ChatGPT

#15

Post by phtrivier »

I've seen a couple of articles like this, but they never show the previous prompts to give some awareness of the language, tools, etc...

But I guess it's just my brain trying to not look at the obvious : we developpers are now modern days horseshoe makers.

Re: Building an interpreter for my programming language with ChatGPT

#16

Post by agumonkey »

mudrockbestgirl wrote: post #9
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

The problem is that you cannot trust the output. It's often wrong, but in subtle nonobvious ways. For precise information you still need to check the sources to make sure what you're getting is correct. You can test it out with a (not-so-mainstream) topic that you're an expert in. You'll see lots of mistakes that are obvious to you, but wouldn't be obvious to non-experts. But it's an incredible tool for brainstorming…

maybe they should adapt it and bridge onto vetted sources

Re: Building an interpreter for my programming language with ChatGPT

#17

Post by bontaq »

mudrockbestgirl wrote: post #9
habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

The problem is that you cannot trust the output. It's often wrong, but in subtle nonobvious ways. For precise information you still need to check the sources to make sure what you're getting is correct. You can test it out with a (not-so-mainstream) topic that you're an expert in. You'll see lots of mistakes that are obvious to you, but wouldn't be obvious to non-experts. But it's an incredible tool for brainstorming…

The new issues will be interesting. Now that I've seen the quality of the output and played with it a bit I'm already squinting at comments here and there. We're in for an even stranger internet.

Re: Building an interpreter for my programming language with ChatGPT

#18

Post by babuskov »

habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

> ChatGPT will be the google killer

This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun?

ChatGPT answer:

> The average distance between the Sun and its planets is approximately 5% of the Sun's diameter.

If you run into bunch of such answers, you will soon stop trusting it for anything.

As for Google - it doesn't even try to answer the question. But no answer is often better than a wrong one. Google at least gives me link to websites that have enough data for me to calculate it myself.

Re: Building an interpreter for my programming language with ChatGPT

#19

Post by DeWilde »

phtrivier wrote: post #15

I've seen a couple of articles like this, but they never show the previous prompts to give some awareness of the language, tools, etc... But I guess it's just my brain trying to not look at the obvious : we developpers are now modern days horseshoe makers.

> we developpers are now modern days horseshoe makers.

Explain?

Re: Building an interpreter for my programming language with ChatGPT

#20

Post by AkshatJ27 »

dom96 wrote: post #8

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript). This makes me wonder how much time…

Instead of optimizing the original prompt, if it spits out something wrong, try and point out the mistake, it is pretty quick to fix it in most cases.
Post reply on HN