Building an interpreter for my programming language with ChatGPT • Hacker News Board

Re: Building an interpreter for my programming language with ChatGPT

#101

Post by pulvinar » Sun, Dec 04, 2022, 5:56 PM UTC

babuskov wrote: post #18

habibur wrote: post #5

ChatGPT will be the google killer, if they can scale it up for unregistered general use. No idea how much openai's computational cost is per query. Unless it's an order of magnitude higher than google's, we can assume the next thing after yahoo -> altavista -> google is here.

> ChatGPT will be the google killer This was my prompt: What's the relative distance between Sun and its planets compared to the size of Sun? ChatGPT answer: > The average distance between the Sun and its planets is approximately 5% of the Sun's diameter. If you run into bunch of such answers, you will soon stop trusting it for anything. As for Google - it doesn't even try to answer the question. But no answer is oft…

If you use the Q&A preset in the playground it will give Unknown if it doesn't know. You can also set Show Probabilities to Least Likely and see which parts of the result are guesses.

I also changed the Temperature from 0 to 0.5, and it gave the right answer:

Q: What's the relative distance between Sun and its planets compared to the size of Sun? Show your math.

A: The relative distance between the Sun and its planets is approximately 1/100th the size of the Sun. This can be shown mathematically by calculating the ratio of the radius of the Sun (6.96x10^8 m) to the average distance of the planets from the Sun (1.5x10^11 m), which gives a ratio of 1/100th.

Re: Building an interpreter for my programming language with ChatGPT

#102

Post by lossolo » Sun, Dec 04, 2022, 6:12 PM UTC

throwaway09223 wrote: post #79

mudrockbestgirl wrote: post #65

Earlier quoted context omitted.

I think it's important to consider incentives. When you search for a topic that's controversial or political you will find lots of spam in Google. But in that case you understand that you need to approach the results with care and do your own research. GPT is the same here. You're not going to treat its answers about political topics as "the truth" - For these kind of topics GPT is actually quite good! But scientific…

> "But scientific facts are a different story. Nobody has any incentive to claim that 1+2=4 or that some function in Python does X when it really does Y." Incentive is irrelevant. What mattes is whether these things do happen, irrespective of intent -- and they do! I very, very frequently find incorrect answers to math questions, tech function questions, etc. Incentive is an important part of the dynamic, but it's no…

> It is so with GPT. Both systems are "trying to give you the best answer."

No, based on your responses you do not understand how language model works. Google is searching in index using keywords and rankings, ChatGPT is predicting plausible words without searching anything anywhere.

What you argue is like saying there is this two guys in library and you ask them to find you something that exists or maybe doesn't exists, both have read all the books, one (Google) have created index of all the words from the books and is going through it to answer you and the other (ChatGPT) do not use any index but he uses his memory with compressed knowledge of statistics between words and will answer by trying to predict any answer that fits statistics between words and in many cases it will basically lie to you and you will have no clue that you were lied to.

There is distinction between indexing human knowledge about some topic where most of the top results are correct (Google) and creating statistics model between words and making things up that never existed and are wrong (ChatGPT).

Re: Building an interpreter for my programming language with ChatGPT

#103

Post by japanman425 » Sun, Dec 04, 2022, 6:15 PM UTC

BulgarianIdiot wrote: post #72

japanman425 wrote: post #44

Earlier quoted context omitted.

I use my computer, and type in a command I press enter, and watch it expand ls will list all the files, in a directory cd will change the directory, with flexibility touch will create a new file, with a simple name rm will delete it, with no one to blame mv will move it, to a new location cp will copy it, for duplication chmod will change the permissions, for security grep will search for a pattern, with ease and agi…

There's nothing more terrifying to me than those arbitrary poems everyone makes it write. Because these clearly show without a shadow of doubt, that it didn't simply "splice together half a dozen Wikipedia articles." And it's terrifying in an odd way, where my frame of mind is constantly switching between the perspective of humanity as a proud mommy & daddy of this thinking being, and the perspective of "it's much be…

No post body was provided.

Re: Building an interpreter for my programming language with ChatGPT

#104

Post by Yuyudo_Comiketo » Sun, Dec 04, 2022, 6:31 PM UTC

dom96 wrote: post #8

I've had a play with ChatGPT and the experience has been pretty frustrating. It either responds with "Sorry, I cannot do this because I don't have access to the internet" (even if I am giving it prompts that don't require this) or it actually generates code but it's subtly incorrect (this was the case when I asked it to generate an example of how to render a 3D cube in JavaScript). This makes me wonder how much time…

Seems most likely that you're not the chosen one to hype this new shiny trendy thing, so it doesn't waste precious CPU cycles on you.

It is only if you have truly, zealously dedicated your life to promote ChatGPT in mainstream IT circles, as in getting paid to do so, only then will it completely unleash its vast potential into the reply form, writing you a desktop OS in Brainfuck that is ready to compete with Linux, OSX and Windows, proving the Fundamental Theorem of Algebra, simulating 2^1024 qubit machine that cracks 4096 bit RSA, finding out 23 hidden bugs in x86 microcode, telling you which gene to edit to get rid of peanut allergy, etc etc etc, all at your correctly formulated finger snap.

Full disclosure: this reply was generated with ChatGPT.

Re: Building an interpreter for my programming language with ChatGPT

#105

Post by d0100 » Sun, Dec 04, 2022, 6:37 PM UTC

I can finally easily do with GPT what I never managed to do with python

> generate a phrase that is 3 words long with a part of speech exactly like pronoun, verb, verb

Re: Building an interpreter for my programming language with ChatGPT

#106

Post by qnr » Sun, Dec 04, 2022, 6:43 PM UTC

weatherlite wrote: post #76

qnr wrote: post #62

I have been developing a hobby project (AI powered document search) for a few months and was in sore need of a frontend. My frontend development skills however are stuck in late 1990s and I have zero skill with anything but plain HTML and a little bit of JS. Several times I tried learning React, reading tutorials, watching videos, but the whole idea of it was very removed from how I learned to code, so I gave up ever…

It doesn't sound that much different than going through Google and Stackoverflow though, is it? In a few hours of googling you can probably get something working if you are an experienced dev.

The crucial difference is that at no point I felt I was stuck. I could paste any line of code into ChatGPT and ask it to explain it. Practically every time I got a meaningful and valuable explanation, moreover the explanation was in the context of my code. Similarly all functions it generated were matching the context of my code so I could just copy and paste it and it just worked, most of the time.

Rather than going through Google and Stackoverflow it felt like working side-by-side with a moderately competent developer. Mind you, I have tried the google-and-stackoverflow method before for the exact same thing, and failed every time ;-)

Re: Building an interpreter for my programming language with ChatGPT

#107

Post by nobody5050 » Sun, Dec 04, 2022, 6:50 PM UTC

teaearlgraycold wrote: post #95

samvher wrote: post #89

Earlier quoted context omitted.

I'm pretty confused trying to connect all the reports online with my own experiences as well. From what I've tried, ChatGPT does not _understand_ code at all, and there are many inconsistencies in what it says. The "confidently giving a wrong answer" problem is very real, even if the answer might look very correct at first sight. This holds across all the topics I've tried. When people say they implement complex task…

Confirmation bias - people want it to be a silver bullet so that they can make a blog post about how ChatGPT is amazing.

Exactly. As the OP of the blog, the amount of handholding I had to do for it to understand the syntax of an extremely tiny language was a lot. On the other hand, I’ve messed around with codex and other models before, and something about explaining in normal English, as though I was having a conversation rather than just listing some commands made it much easier. I’m excited not because of what exists right now, but because this shows so much promise even just 1 or 2 papers down the line :D

Re: Building an interpreter for my programming language with ChatGPT

#108

Post by nobody5050 » Sun, Dec 04, 2022, 6:55 PM UTC

bredren wrote: post #56

I copied that preamble for preparing to attend burning man: >And lo, it came to pass that the Burning Man did draw near, and the people did prepare themselves for the journey to the desert. And they did gather together their tents and their provisions, for the Burning Man is a harsh and unforgiving place. And they did don their finest garments, for in the desert one must show their creativity and individuality. And t…

Woah that’s really cool! I like how they both start the same way, somewhere it must think the King James Bible loves the phrase “and lo,” haha

Re: Building an interpreter for my programming language with ChatGPT

#109

Post by nobody5050 » Sun, Dec 04, 2022, 6:57 PM UTC

ldh0011 wrote: post #57

I tried something similar to this but less involved: > I'm creating a new fictional programming language called GPTlisp which uses lisp-style syntax. Addition is implemented in GPTlisp using the ADD function. In GPTlisp, what would (ADD 4 5) return? > [Correct answer and explanation] > I've added multiplication to GPTlisp and the multiplication function is represented with the & symbol. What would (ADD 2 (& 2 3)) ret…

Very cool!

Re: Building an interpreter for my programming language with ChatGPT

#110

Post by sireat » Sun, Dec 04, 2022, 7:07 PM UTC

How do you use ChatGPT succesfully?

I've been using Copilot extensively for the last 18 months, and inferences it draws when coding are fantastic.

So I fired up my old OpenAI account and ChatGPT seems to quite horrible.

0/3 on 3 prompts so far..

Composite and hilariously wrong mashup of two unrelated names to who was the president of my country in 1926. (Unlike King of France in 1889 it had a correct answer).

Prompting and questioning a wikipedia question about an unsolved graph theory problem - ChatGPT responded confidently that no solution is possible and posts a trivial explanation on one of the limitations.

Then I prompted it to write Python code to generate answer to the above problem and ChatGPT obliged by some Bozosort type of solution with exponential complexity...

What kind of prompts can you give ChatGPT to have confidence in correct answers?