Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

171–180 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#171
post #170

The biggest thing here is that it's semi capable and improving. I feel safe about my job right now but it is worrying to invest time to compete with a machine that will continue to get better over the years where previously I felt safe that the effort of my labour would bear fruit for decades to come. Now I'm not so sure.

Same here, but it was fun tho but I certainly feel like I'm no longer on top of the food chain

Re: Testing GPT 4's code-writing capabilities with some real world problems

#172

Earlier quoted context omitted.

But obviously increasing avg and sr programmer productivity by many x implies many fewer jobs for programmers. That’s the point we’re interested in.

Is that obvious? The history of programming has been full of things that have at least claimed to increase programmer productivity, everything from high level languages to smart refactoring IDEs, debuggers, PaaS, etc and in all that time the trend has been towards more jobs for programmers not fewer.

Productivity with modern tools (SPAs, microservices) is often a lot lower than oldschool stuff (django/rails, maybe even Visual Basic, Delphi)

Re: Testing GPT 4's code-writing capabilities with some real world problems

#173

A modified A* that solves the fire routing problem (less efficiently than OP's I think). Each A* location stores where it comes from, how long it takes to get to it, and how many fires it passed through to get there. The algorithm only considers fire cells neighbors if the current number of fires passed through is less than the current fireWillingness global. 1. count fire tiles within movement range 2. run A* from s…

Counterexample:

  ...XX
  SF.FD
  ...XX
  
  S = start
  F = fire
  X = wall
  D = destination
The cat can to the destination in 6 moves passing through 1 fire. In the fireWillingness=1 pass, the middle tile is reached after passing through fire, so the destination appears unreachable. The proposed algorithm will pass through 2 fires instead of 1.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#174
Personally, I found GPT-4 to be helpful when writing code for games. But I'm a web programmer trying to learn game development, I'm no professional game developer by any measure. And I'm using Rust and Bevy, for what it's worth. So it might not be as helpful for someone like Tyler who actually know what they are doing, similarly for me if I were to use it for web development.

The most helpful thing with GPT-4 have been getting help with math heavy stuff I don't really grok, and that I can try to compile the code, get an error and instruct GPT-4 that the code didn't work, here is the error, please fix it. Other things it been helpful for is applying the "Socratic method" for helping me understand concepts I don't really grok, like Quaternions. Then, knowing GPT-4 isn't perfect, I always verify the information it tells me, but it gives me great starting points for my research.

Here a conversation I had lately with GPT-4 in order to write a function that generates a 2D terrain with Perlin Noise: https://pastebin.com/eDZWyJeL

Summary:

- Write me a 2D terrain generator

- Me reminding GPT-4 it should be 1D instead of 2D (I used the wrong wording, confusing a 1D vector with 2D)

- Code had issues with returning only values with 0.0

- GPT-4 helping me tracking down the issue, where I used the `scale` argument wrong

- Got a working version, but unhappy with unrealistic results, I asked it to modify the function

- Finally got a version I was happy with

Re: Testing GPT 4's code-writing capabilities with some real world problems

#175
post #73

Earlier quoted context omitted.

On the flip side, I am having a great time charging enterprises exorbitant hourly rates to come in and help them fix their crappy crud programms. But I still don’t know if I am thrilled by the prospect of being called in to make their monkeypatched, GPT generated project work.

> their monkeypatched You mean, like almost every outsourcing company pops over? So the type of code that infests companies who hired some sweatshop to do ‘some simple crud’? What’s the difference? Can you see the difference? Besides the gpt code will be far better commented as comments come for almost free with gpt while humans hate writing them.

I'm working on an outsourced codebase right now and boy, I wish they hadn't commented their code at all

Re: Testing GPT 4's code-writing capabilities with some real world problems

#176
I'm more intrigued why the author finds this a difficult problem for the needs of their game. It looks like their search space is at most a 10 X 10 grid for the most part (I'm assuming based on asset since and detail it doesn't grow too much larger).

I know it isn't relevant to the Chat-GTP code writing discussion, but A*, Dijkstra and heuristics to move an entity around 8 spaces could raise the question "Can the developer be more pragmatic?".

Re: Testing GPT 4's code-writing capabilities with some real world problems

#177

Earlier quoted context omitted.

Sure but the token jump to 32k is new isn’t it? Even 8k to 32k is significant and that was this year.

Yeah, but the jump to 32k didn't take a few months, it was years in the making. Otherwise you could extrapolate with "yesterday we had 4k, today we have 32k, tomorrow we will have 256k", that isn't how we do it. If we follow the same exponential pace 256k would have to wait 3 years, and even that is unlikely.

I think it is better to assume that we are at the beginning of the s-curve, not the end.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#178
post #63
post #10

Earlier quoted context omitted.

Have you seen this video? https://www.youtube.com/watch?v=9QZlQMpNk-M I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better. In that sense, it isn't replacing the programmer; i…

> replacing IDE autocomplete Co-pilot has been very useful the times I've used it. It's not perfect, but does cover a lot of boiler plate. It also makes it much easier to jump between languages.

I’ve been working with copilot a few months and the biggest surprise is how it has led to much better commented code. I used to comment tricky code to explain it. Now I comment trivial code instead of writing it. Often two lines of comment will get me 10 lines of copiloted code, faster than I could have typed it and with good comments to boot.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#179
I think AI will never write good code but can be very useful for very basic stuff, boiler plate or repetitive stuff, like a smart IntelliCode. In fact, I think MS built some AI in IntelliCode but not advanced stuff so they can sell GitHub Copilot.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#180
post #2

So, for a bit of fun, I signed up to GPT-4 thingy plus and I picked a fairly common web application and built it from scratch, only by talking to GPT-4 and copy pasting the code bits. I'm actually taken back by how well it's doing; including providing me some refreshers on stuff I forgot how it should work. I can see it failing at solving complex problems, but like the blog post mentions, most programming isn't new o…

Programming takes little effort. It's at most 20% of my effort. Feels like GPT is programming with extra steps. Maybe it's for people who can never think programming is easy. Clearly there's a lot of such types. Explains a lot.

I think programming is very easy most of the time, most time I spend is just typing/moving code around, figuring out the solution is the easy part for me, only the computerbrain coordination is slowing me down most of the time.

But there are things that are harder for me, or more complex maybe. I struggle with math, and always had, so anything involving heavy math or algorithms is harder for me (I'm a hacker, not a computer scientist, if you will).

For these things, I found GPT4 to be very helpful. I can write what I want, get a WIP version back, work out some kinks with it myself and ask it to rewrite it if it's not perfect, until I have a perfect version. Add some unit tests, find more things that are missing/wrong (sometimes), more back and forward.

Before GPT4 I either just tried to work around having to deal with heavy math, or find people IRL that could help me. Now I'm a lot faster even if it involves math, because of GPT4.

Post reply on HN