ChatGPT Changed How I Write Software

I don't have any super concrete examples, but I can confirm it _helps_ in writing software for me.

Things it has helped with, that are now part of a codebase I work in and get paid for.

Generated example css to center a text element onto a sibling image. I'm decent at css, but I didn't know I could do this easily. This definitely saved me a decent amount of time

Write some boilerplate react hook wrapping over an axios request. Something along the lines of "Write a React hook to make a request to ..., using these parameters ... Return ..."

The one that I've probably used most is "Write a MySQL query to ..., these are my schemas: {COPY PASTE RELEVANT TABLE SCHEMAS". I can write MySQL queries just fine, but getting one with some decently complex calculations takes me some time. GPT-4 gets very close most of the time. For exploring the database GPT really has helped save me quite some time. I just don't now MySQL as well as I do my primary languages like TypeScript, PHP and Python so this is a godsend.

Otherwise, I also use it a decent amount in some other scenarios. The Phpstorm plugin I have gives me a right-click menu that pastes my code in a chatGPT window. It has stuff like

1. Refactor

2. Explain

3. Write docs

4. Write a custom prompt

I use these a lot. They save me a decent amount of time. They're super quick to access right in my IDE. In some cases using the "Refactor" command, GPT-4 gives better results than myself. I found that it doesn't really make typos in making sure variable names and stuff stay the same, and it also doesn't mess up control flow which I sometimes do. And yes, this is an admission that I do sometimes refactor without tests, I'm working on it :x

It can generate example data for tests but why would I want that when, if I use a good property testing framework, I already have that; and it can shrink examples sensibly so that I get the minimal failing example for my test?

Generating code seems worse too. I agree that you don’t need “perfect,” software if you’re building a blog for your local soccer team. Maybe you want more precaution if you’re initiating trades in securities. And definitely a lot more scrutiny if you’re building real-time control systems for helicopters. Verifying that code is correct with regards to its specification first requires a precise, formal specification; and second, is still a lot of work.

Eyeballing every line of code to catch errors ChatGPT introduces into your code isn’t going to work. You’re not a compiler or theorem prover and you’re bad at finding errors in code. Really bad. It’s not your fault; code becomes incredibly complex quickly.

At best, right now, it’s a curious toy and maybe useful for some folks as a “smart” autocomplete. But beyond that ChatGPT seems to struggle. And I struggle to find a use for it myself.

I've gotten pretty good at getting ChatGPT to do what I want. I wrote about it here, https://zchry.org/words/building-a-brand-identity-with-code, but that's probably still light on details for you.

There really isn't any magic bullet I don't think, every case is unique and involves a lot of trial and error. The original prompt was something as simple as "You're going to help me build a plugin for figma that draws Lissajous Curves". It starts with telling you how to setup the Figma plugin folder structure (which Figma makes super easy), installing all the dependencies / npm etc. From there, it was just a lot of going back and forth with ChatGPT fixing the bugs until it compiled cleanly (files we worked with: code.ts, manifest.json, tsconfig.json, ui.html), then going through the bugs that would be thrown in the app console when you actually ran the plugin.

Eventually though, everything worked great and Figma even approved the plugin to be published publicly.

My best piece of advice is that when I feel really far off where I want to be, I just start over from scratch and rework my prompts to more accurately guide ChatGPT in what I want.

And lastly, now that I have a basic "baseline" of how I want ChatGPT to help me with these plugins, I give it all that information up front and try it keep it in that predefined scope.

Does the ChatGPT plugin for PHPStorm actually analyze your existing code? As-in... can you ask it to code something using your pre-existing utility functions, classes, and so on?

Yes GPT4 is marginally better, no it doesn't significantly change its usefulness for software development. It still misunderstands questions and tasks, it still forgets or ignores constraints almost immediately, it still uses libraries that don't exist, it still generates tons of plausible-sounding but ultimately flawed code that wastes more time than it saves (unless you couldn't code your way out of a wet paper bag to begin with).

It makes for a decent rubber duck and it can aid you in exploring the space of unknown unknowns when approaching a new subject, but that's about it in my experience.