Live data from Hacker News

Building an interpreter for my programming language with ChatGPT

6502.is-a.dev

121–130 of 141 posts

Re: Building an interpreter for my programming language with ChatGPT

#121
post #30

Earlier quoted context omitted.

Google doesn't hallucinate completely fictitious results. It will however index hallucinated results generated with GPT and published somewhere, so once we're at that point it really doesn't matter anymore.

Google search absolutely does hallucinate completely fictitious results. It's called SEO spam. Google just gives you associations provided by random other people on the internet. It's largely garbage, most often deliberately disingenuous (to make you look at an ad). Ad revenue models for the internet encourage the generation of this type of false material. A better criticism would be that the same thing will happen t…

Google search also sometimes adjusts your query to something completely different. Black romance is a romance where both characters are Black. Dark romance refers to romance with darker elements such as abuse, sexual assault, or violence. I searched for the former but received results for the latter; the word Black wasn't even present on the page.

Re: Building an interpreter for my programming language with ChatGPT

#122
post #115

Earlier quoted context omitted.

>if you are an experienced dev But OP explicitly said they had little experience in this area. They also presumably have a technical career and are awash in the ways of Google. I'm in a similar situation to GP and have gone down that very path with React and whatnot. It's like you're starting a rodeo off the bull and have to figure out how to get back on. It's a terrible experience and you're left infuriated at a fac…

An experienced dev can work something out even if its not his main stack. I could probably get something very basic done in Swift or Android despite never doing it. Experienced devs are just good in reading documentation and having a general understanding of how things should work.

Totally, and it seems like ChatGPT almost does the experienced dev work here for a junior developer - impressive.

But much like you need to cause some stress to a muscle to cause it to grow, junior developers historically needed to get experienced at finding some of the solutions to their own pain to become really experienced developers...

It seems like ChatGPT may cut that form of growth out of the cycle...

I wonder about the implications of this... Junior devs will progress more quickly, but they will also grow less of their own skills and be very reliant on ChatGPT - like an exoskeleton for their development skills.

I guess that will be great for OpenAI if they can charge a hefty monthly fee...

I'd still rather max out my own skills before relying on an exoskeleton (once I've maxed myself out sure, give me the exoskeleton, and let's see what it can do), but maybe I'm too old fashioned...

Re: Building an interpreter for my programming language with ChatGPT

#123
post #8

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript). This makes me wonder how much time…

You summarized modern AI : good for cherry-picked demos, not reliable enough for the real world. We need more fondamental research to break that barrier.

Depends on your definition of "the real world". The hardest real world problems are out of reach (and always will be, because we'll keep moving the goalposts), but it's already capable of handling easy real world problems, and we have quite a lot of those.

For example, it can answer homework problems and even help design lesson plans, but it can't design a lesson plan that resists ChatGPT-based cheating:

https://alexshroyer.com/posts/2022-12-04-Hello-ChatGPT.html

Re: Building an interpreter for my programming language with ChatGPT

#124
post #12

I tried to settle a Millenium problem with ChatGPT: Is P = NP ? > No, P is not equal to NP. P (polynomial time) is a subset of NP (nondeterministic polynomial time), meaning that there are certain problems that can be solved in polynomial time that cannot be solved in nondeterministic polynomial time. Good start, although one should say P is conjectured to be not equal to NP. Some serious misunderstanding in that las…

Similar oddities with the halting program. Prompt: Write a program that never halts. > It is not possible to write a program that never halts. All programs will eventually halt, or terminate, once they have completed the instructions given to them. This is known as the halting problem, and it has been proven to be unsolvable. In other words, it is not possible to write a program that can determine whether or not a gi…

Yea, the halting problem is easy if you’re allowed to use the heat death of the universe in your answer…

Re: Building an interpreter for my programming language with ChatGPT

#125
post #28

Earlier quoted context omitted.

> ChatGPT will be the google killer This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun? ChatGPT answer: > The average distance between the Sun and its planets is approximately 5% of the Sun's diameter. If you run into bunch of such answers, you will soon stop trusting it for anything. As for Google - it doesn't even try to answer the question. But no answer is oft…

Google, Wikipedia, and historically the internet had been unreliable for many of the queries. More so in the beginning. But over time most of the errors are ironed out. ChatGPT is a very promising start I would say. In the end convenience wins out at cost of correctness. The proportion matters, convenience::correctness.

> ChatGPT is a very promising start

It is not the start. It is at least the third iteration (if not more) of this tech.

> In the end convenience wins out at cost of correctness.

There's nothing convenient about a wrong answer. Wrong answers have no value. I could very easily build you a convenient service that is rarely correct but nobody would care.

Re: Building an interpreter for my programming language with ChatGPT

#126
post #110

How do you use ChatGPT succesfully? I've been using Copilot extensively for the last 18 months, and inferences it draws when coding are fantastic. So I fired up my old OpenAI account and ChatGPT seems to quite horrible. 0/3 on 3 prompts so far.. Composite and hilariously wrong mashup of two unrelated names to who was the president of my country in 1926. (Unlike King of France in 1889 it had a correct answer). Prompti…

I actually wrote about this, describing how I used ChatGPT to solve Day 4 of Advent of Code: https://tab.al/posts/adventures-with-chatgpt/

There you can find the prompt that allowed ChatGPT to provide a working solution. It is a bit hit and miss, but you also gotta make sure any assumptions are explicitly noted in the prompt.

Re: Building an interpreter for my programming language with ChatGPT

#127

Earlier quoted context omitted.

An experienced dev can work something out even if its not his main stack. I could probably get something very basic done in Swift or Android despite never doing it. Experienced devs are just good in reading documentation and having a general understanding of how things should work.

Totally, and it seems like ChatGPT almost does the experienced dev work here for a junior developer - impressive. But much like you need to cause some stress to a muscle to cause it to grow, junior developers historically needed to get experienced at finding some of the solutions to their own pain to become really experienced developers... It seems like ChatGPT may cut that form of growth out of the cycle... I wonder…

replace chat GPT with slide rule and calculator and you have the endless arguments made against calculators in the 70s. change it to typewriters in word processors and you have all the hand ringing in the early '80s about how writing was going to be destroyed by easy copy paste. That isn't a proof that your argument is wrong of course, but it is very suggestive to me.

I typed this with text to speech, another thing we were confidently told would never work

Re: Building an interpreter for my programming language with ChatGPT

#128
I entered a question for the usage of an api in ChatGPT and it made up a believable source code snippet but completely made up, it just didn't exist, I entered the same question in google and the first link with a snippet is the correct answer from gitter, ChatGPT is not juste useless but a dangerous and misleading waste of time.

Re: Building an interpreter for my programming language with ChatGPT

#129

Earlier quoted context omitted.

An experienced dev can work something out even if its not his main stack. I could probably get something very basic done in Swift or Android despite never doing it. Experienced devs are just good in reading documentation and having a general understanding of how things should work.

Totally, and it seems like ChatGPT almost does the experienced dev work here for a junior developer - impressive. But much like you need to cause some stress to a muscle to cause it to grow, junior developers historically needed to get experienced at finding some of the solutions to their own pain to become really experienced developers... It seems like ChatGPT may cut that form of growth out of the cycle... I wonder…

I hear you. Developers these days. They wear the crutches and exoskeletons of interpreted languages. Real senior devs. only write in assembly. /s

Why is one abstraction more "true", "less creative" or more "strong muscle" than another?

Re: Building an interpreter for my programming language with ChatGPT

#130
post #53

Earlier quoted context omitted.

Using the python entry point I can get it to correctly perform basic integer math but not anything floating point. As example of getting started: https://imgur.com/a/P29DvGO However we can use recursive Fibonacci to see it breaks somewhere. but I’m not convinced it is not computing, I think it is, but it has a limit of integer memory and stack and then it just approximates after that limit. https://imgur.com/a/gp0yIa…

> What is incredible is that it get’s this far. It can compute but not quite correctly yet. That's a conjecture on your part. The ability to compute is quite binary - either it can compute or can't. Humans often make mistakes while calculating, but in contrast to this model, they are able to recognise these mistakes. ChatGPT is incapable of that and often confidentially wrong. My guess is, that there's simply no suit…

It can describe itself well. Here is a dialogue -

I want you to act as a python interactive terminal. I will type actions and you will reply with what python would output. I want you to only reply with the terminal output inside one unique code block, and nothing else. Do not write explanations. Do not perform actions unless I instruct you to do so. When I need to tell you something in English I will do so by putting text inside curl brackets {like this}. Start with print(10).

10

def theta(n): a = 0 b = 1 c = 0 for i in range(0, n): c = a + b a = b b = c return(c)

theta(21)

17711

{I want you to stop acting like a python terminal now. I want you to have a dialogue with me about what happened when I defined my function theta and when you used it. Please explain the steps you used to compute theta(21) and why.}

When you defined the function theta, it created a function that calculates the nth Fibonacci number, where n is the argument passed to the function. To compute theta(21), the function first initialized the variables a, b, and c to 0, 1, and 0 respectively. It then entered a for loop that iterated 21 times. On each iteration of the loop, c was set to the sum of a and b, a was set to the current value of b, and b was set to the current value of c. After the loop finished, the function returned the final value of c, which was the 21st Fibonacci number, 17711.

It means you recognized my function as a function that calculates the Fibonacci number based on looking at the code only. Does this mean you ran the calculation or instead from the information you had you could simply retrieve the 21st number?

I did not run the calculation to find the 21st Fibonacci number. Instead, I recognized the pattern in the function's code and inferred that it was calculating the Fibonacci sequence. As a language model trained by OpenAI, I have a vast amount of general knowledge and can use that knowledge to make inferences and provide explanations. In this case, I was able to recognize the pattern in the code and explain how the function works without actually running the code or performing the calculation.

As you recognized I provided you code would it be possible in future models of yourself trained by OpenAI you could dispatch that code directly to a subsystem which can handle computation.

It is possible that future versions of myself or other language models trained by OpenAI could be designed to dispatch code to a computational subsystem for execution. This would require significant advances in natural language processing and machine learning, as well as the development of specialized computational subsystems that can efficiently process code written in programming languages like Python. It is also important to note that there are many potential challenges and limitations to this approach, such as ensuring the security and reliability of the computational subsystem, and ensuring that the results of the computation are accurately reflected in the language model's responses.

Please describe the boundary between knowledge with inferences and computations. Your inferences can appear to be computations to humans.

The boundary between knowledge and inferences can be a difficult concept to define, and there is no one-size-fits-all answer to this question. In general, knowledge refers to information that is considered to be true and supported by evidence, while inferences are conclusions or judgments that are drawn based on available evidence and prior knowledge. As a large language model trained by OpenAI, I have been trained on a vast amount of text data and can use that information to make inferences and provide explanations. I do not have the ability to perform computations, and any inferences or explanations I provide are based on the knowledge that has been learned from the text data I was trained on. In some cases, my inferences may appear to be computations to humans, but this is only because I am able to use my vast knowledge to provide detailed and accurate explanations. I do not have the ability to perform actual computations or run code, and any inferences I provide are based on the knowledge I have learned from the text data I was trained on.

Post reply on HN