ChatGPT Changed How I Write Software

I've definitely given it a try for some of that, but the truth is I just don't have that many greenfield projects. Most of my work is maintaining existing code, and CoPilot/GPT-4 haven't proven as useful for me at that.

When it comes to new, greenfield projects, I really do try to be ambitious generally speaking, and this leads me into trouble. I gave it a try to see if it could generate code for something where I needed just a slightly tricky data structure maneuver. It was a nicely self-contained problem, pretty much ideal for this sort of thing, but it still generated code that didn't compile and wasn't optimal.

To be fair, it was kinda close, which is great. But overall it wasn't worth the time: I'm pretty slow, but most of my time is NOT in trying to implement basic things, it's trying to come up with what I need to implement in the first place. So I definitely tried spitballing with GPT-4 a little bit to try to get an idea of whether or not that was a good idea. Truth is, it's a bit hit-or-miss.

Here's my take:

- For programming tasks, it's just skipping too much stuff. I think the future for LLMs doing complex tasks is definitely going to depend heavily on huge context windows and approaches like langchain. That said, I think today you still can't really get something that resembles a "self-driving" robot programmer, because it's just lacking the depth necessary.

- For language tasks, it's fantastic. When it comes to language tasks, it really doesn't feel like it's skipping anything, it feels like it has a genuine understanding of language. I'm not sure if we're there yet, but I am hopeful for the future of machine translation using LLMs: You can't talk to Google Translate about the context your string will be used in, but you CAN do that for GPT-4, and while I'm not really fluent in any language aside from English, cross-checking its work, I think it's at least a lot better than what you can currently do using DeepL and Google Translate for common languages. (To be fair, even LLaMA finetunes have been SHOCKINGLY useful for this use case in my experience. They hallucinate a bit too much to be useful, but the fact that the hallucinations usually seem to lie close to the answer leaves me impressed given that it's a several GiB bundle of data on my SSD that runs on CPU only.)

Also, GPT-4 is just going to be epic for people learning new things. I am enthusiastically excited about the ability to just auto-generate some example code for some kind of thing I'm not familiar with. Even if it's nearly useless, it can probably get you pointed in the right direction and learning what terms you need to google for.

Unfortunately for this poor foolish man, he has spiked his own credibility with his zeal for AI. I mean, aren’t you kind of dismissing this post as “oh it was probably written by ChatGPT.”

One indicator of low credibility is that he keeps saying “this saved me weeks of work” for things that look like they might take a few minutes to do with a spreadsheet, or else could be Googled, or for which there are already free tools, or frankly don’t even need to be done.

I have a bias about his project that I will admit: he seems to be yet another one of those smug entrepreneurs who’s goal in life is to contribute nothing creatively or intellectually to the world. A parasite. Fills me with loathing.

i use gpt4 daily.

it’s not very good at writing code, but can write code in small well defined chunks.

i’m working on a game with blender, cpp, physx, and wickedengine.

it writes a lot of python for blender with prompts like:

blender. python. apply transforms to every object unless X.

these tend to work, if scoped small enough, and any exceptions or failures are put as followup prompts.

writing python for blender is more annoying than hard, so this is great.

the other type of prompt i use a lot is like this:

blender. did X. expected Y. got Z. why?

i use prompts like these a lot, for cpp, physx, and wickedengine. often i ask for code, but not to copy it verbatim, but because i’m thinking in code.

these types of prompts usually output a list of possibilities. almost always the list contains what i’m looking for. usually it’s the first item.

a gamedev prompt might be:

wickedengine. code. how to rotate normal 90 degrees right on y in world?

I have found ChatGPT to give bad answers for more complicated questions. For example, the other day I encountered some TypeScript code that surprised me. Basically it was:

class A { @SomeDecorator() x: string }

const value: A = {x: "foo"};

I was surprised that it compiled at all and structural typing applied to classes, even though '!(value instanceof A)'. But it worked! I asked ChatGPT for more information about this, and basically it just told me that it wouldn't compile, which was obviously wrong.

It’s not a bad article. Only feedback I would have is about this passage:

“”” If you routinely pass in a dataset to AI, it can identify anomalies and watch to see if it settles or even reason why the anomaly occurred in the first place. “””

This is not a great use for LLM’s. AI? Sure. But not this type of it.