Live data from Hacker News

AI-enhanced development makes me more ambitious with my projects

simonwillison.net

31–40 of 519 posts

Re: AI-enhanced development makes me more ambitious with my projects

#31
post #2

Ok, so when people say that they use ChatGPT for coding, they use it as a code snippet library? Here, a function that has been probably written million times before in some similar form, and a template for a simple web app.

This isn't an example of a coding problem, but just yesterday we had a very difficult technical problem come up that one of our partners, a very large IT company couldn't solve. None of our engineers knew how to solve it after trying for a few days. I would say I'm an expert in the industry and have been doing this over 20 years as a developer. I thought I knew the best way to solve it, but I wasn't sure exactly. I asked ChatGPT(GPT-4) and the answer it gave, although not perfect in every way, gave pretty much exactly the method we needed to solve the issue.

It's a bit scary because knowing this stuff is sort-of our secret sauce and GPT-4 was able to give an even better answer than I was able to give. It helped us out a lot. We are now taking the solution back to the customer and will be implementing it.

A few additional thoughts:

1. I knew exactly what type of question to ask it to get the right answer (i.e. if someone used a different prompt maybe they would get a different answer) 2. I knew immediately that the answer it gave was what we needed to implement. Some parts of the answer were not helpful or misleading and i was able to disregard those parts. Maybe someone else would have to take more time figuring it out.

I imagine future versions of GPT will be better at both points.

Re: AI-enhanced development makes me more ambitious with my projects

#32
post #2

Ok, so when people say that they use ChatGPT for coding, they use it as a code snippet library? Here, a function that has been probably written million times before in some similar form, and a template for a simple web app.

Posting some recent prompts as examples as I've started to use it more for coding:

- I want to create a Google Docs addon using AppScript which has an interface sidebar. Please describe the necessary steps to create a sidebar interface. The sidebar interface should have a title with the text "Google Docs JSON Styles", it should have a large textbox which can contain multiple lines of text (fill it with lorem ipsum), and it should have 4 buttons placed horizontally underneath the text box, with the captions "Save", "Apply", "Copy", "Paste".

This produced the necessary Javascript + HTML + external steps to set up the project.

- I'm building a website using a React frontend hosted on example.com with a Django REST API backend hosted on api.example.com - Authentication is handled via Firebase Auth, currently using an Authorization header provided for API calls. Now I want to have authenticated links such that the user can click a link to access an authenticated download via the API. What mechanism can I use to create a link to an API endpoint which will require the user to be authenticated?

This gave me a the frontend and backend code necessary to set up authenticated links, with several alternate approaches following further prompting.

- Given an owner id, repository id, commit id, and private access token; give me a URL that lets me download a ZIP archive of a Github repository.

In all of these examples I received the desired result with helpful descriptions.

As a counter-example:

- Write a program in Brainfuck that outputs the string "CHATGPT".

In this case, the result is the Brainfuck program for "Hello World", with a step by step break-down of the program explaining confidently why it would output "CHATGPT", while being entirely wrong.

Re: AI-enhanced development makes me more ambitious with my projects

#33

What I really love is the unbounded optimism of the AI. How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?" that would then just think for a second and go "Certainly! Let's start by ...". These people exist, but they're one in a thousand (or million?). ChatGPT basically does that for almost every prompt you throw at it. So…

everything seems easy when you're armed by the corpus of all human text

Re: AI-enhanced development makes me more ambitious with my projects

#34
post #6

@Simon, I love this. And one day, in the not too distant future, a few old wizened graybeards, like you and I and the other folks on this site who have been writing code from scratch for 20 years, will be the magicians who walk the earth among mere mortals who only learned to code with the assistance of ChatGPT. We will be the ones with the preattentive syntax parsing in our brains that let us simply see basic syntax…

It’s crazy to think about. We are nearing the end of the age where people who have been programming for “20 years this year” and self-taught probably learned to program from books. In the next few years, people who have been “programming for 20 years” might start to include YouTube learners and boot campers. As someone who learned to program from two gigantic tomes of C/C+ with a Borland Compiler, there was no copy/p…

I learned coding in a classroom.

Re: AI-enhanced development makes me more ambitious with my projects

#35
post #20

Earlier quoted context omitted.

There are programming benchmarks though (data contamination issues left aside)

Yes, which is funny because there was an article on here with some pretty hard data which showed not much difference or maybe worse performance from GPT4 than 3.5-turbo. You'll likely refute that as your mind is already made up, but there you go, another conflicting and confusing data point.

What are talking about? Just compare the output of a 3.5 vs 4 yourself for a problem you are interested in, it’s a single click in the interface.. Do you always need a study or an “expert“ to make up your mind?

Re: AI-enhanced development makes me more ambitious with my projects

#36
post #6

@Simon, I love this. And one day, in the not too distant future, a few old wizened graybeards, like you and I and the other folks on this site who have been writing code from scratch for 20 years, will be the magicians who walk the earth among mere mortals who only learned to code with the assistance of ChatGPT. We will be the ones with the preattentive syntax parsing in our brains that let us simply see basic syntax…

It’s crazy to think about. We are nearing the end of the age where people who have been programming for “20 years this year” and self-taught probably learned to program from books. In the next few years, people who have been “programming for 20 years” might start to include YouTube learners and boot campers. As someone who learned to program from two gigantic tomes of C/C+ with a Borland Compiler, there was no copy/p…

Don't forget the ones who learned to program from colorful 1980s Usborne BASIC books. (Also with no copy/paste, and having to use a tape cassette to save the listing.)

Re: AI-enhanced development makes me more ambitious with my projects

#37
post #29
post #17

Earlier quoted context omitted.

https://arxiv.org/abs/2303.12712 there are a lot of examples in this paper. Also it’s not that it writes perfect code straight away and you never have to re-prompt or change anything manually. But it’s still an insane speed-up.

Do you know if there are any papers published by people who don't work at Microsoft so may actually be more objective?

Just try it yourself, it’s not that hard…

Re: AI-enhanced development makes me more ambitious with my projects

#38
post #6

@Simon, I love this. And one day, in the not too distant future, a few old wizened graybeards, like you and I and the other folks on this site who have been writing code from scratch for 20 years, will be the magicians who walk the earth among mere mortals who only learned to code with the assistance of ChatGPT. We will be the ones with the preattentive syntax parsing in our brains that let us simply see basic syntax…

Having spent a lot of time with GPT-4 recently, it's possible the new generation will not have the same issues we did. For example they can write some code and say, "adjust for best practice".

Then "explain why needs to be "

Oh this is called , give me an example of it. Give me another example. Give me another example. Give me use cases where this would be better. Give me a short story where it was done my way but it lead to a mishap in production and the protagonist learned . Explain it like I'm 5. etc

Basically the new generation is the training neural network, and they can keep flashing context until they absorb the material. Sure, some people are lazy and will go as far as getting something barely working. But superpowers are available for those who fight for their place in the world

Re: AI-enhanced development makes me more ambitious with my projects

#39

What I really love is the unbounded optimism of the AI. How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?" that would then just think for a second and go "Certainly! Let's start by ...". These people exist, but they're one in a thousand (or million?). ChatGPT basically does that for almost every prompt you throw at it. So…

How do you know it is even close to correct? When I’ve asked it for code, it has been subtly wrong, or worse, correct but missing important edge cases that it doesn’t even seem to be aware of.

Re: AI-enhanced development makes me more ambitious with my projects

#40

What I really love is the unbounded optimism of the AI. How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?" that would then just think for a second and go "Certainly! Let's start by ...". These people exist, but they're one in a thousand (or million?). ChatGPT basically does that for almost every prompt you throw at it. So…

How do you know it is even close to correct? When I’ve asked it for code, it has been subtly wrong, or worse, correct but missing important edge cases that it doesn’t even seem to be aware of.

[deleted]
Post reply on HN