Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT Changed How I Write Software

readysetcloud.io

Post reply on HN
Print view View original

31–40 of 51 posts

Re: ChatGPT Changed How I Write Software

#31

Post by tuchsen »

dimgl wrote: post #2

Yeah I'm not convinced. As usual with most pro-ChatGPT posts I see, there is little to no actual useful information on how to use ChatGPT practically. When I read one of these articles or posts, I usually expect to see concrete prompts, their output, and how they can be used to speed up development. Instead all I see is generic vagueisms; "it helps me write unit tests", "it helps me code faster", or in the case of th…

I'm sure others will reply with their own, but I can give you several examples where it specifically sped up some of my development. I'll start with just the one though, because otherwise this could get quite long.

As a weekend project I did a DSL for helping to interact with LLM's called Prompt (prlang.com), kinda like an declarative LangChain. The language is unimportant, what is important is that syntax highlighting is just one of those things that people expect a language to have now, and every editor wants to do it differently. Figuring out syntax highlighting is one of those super tedious things that makes writing a DSL very boring. ChatGPT4 is very good at translating from one language to another though, and so I could use it to help me write the vim syntax file (I don't know VimScript at all) and then translate it to other editors as necessary.

To get specific, it helped me write this vimscript file:

https://github.com/PromptProgrammingLanguage/prompt/blob/mas...

and then with almost no editing from me turned it into another syntax highlighting file (for hljs) that's used on the website.

https://github.com/PromptProgrammingLanguage/PromptProgrammi...

I don't know how long that would have taken me to figure out on my own, but diving into VimScripts docs, and then the HL.js docs, and fully reading and comprehending them would have been a total slog. Doing it with ChatGPT4, I could keep open the docs for Vim, ask it questions about how to do something, then quickly reference the docs to cross check if it was telling the truth, write, and then test the syntax in a very quick loop. Then the process of translating what I had to another library was incredibly fast, if I recall it misplaced one line while translating, which was thankfully obvious, and that was it. I didn't read the docs for hl.js at all.

ChatGPT is definitely better at some tasks than others. Anything that takes shallow knowledge of your specific code base, but broad knowledge of the wider languages ecosystem, is a good fit for it's use.

I could give a more impressive example of where it helped me tackle a super specific bug in my code that had to do with the wider Rust ecosystems libraries, it really blew me away with that one, but the above was easier to document. You wanted specific though, so there ya go :).

And of course ChatGPT4 is leagues ahead of ChatGPT3.5, so if you're not getting good results you really want to pay for the better model.

Re: ChatGPT Changed How I Write Software

#32

Post by dimgl »

oldstrangers wrote: post #25
dimgl wrote: post #2

Yeah I'm not convinced. As usual with most pro-ChatGPT posts I see, there is little to no actual useful information on how to use ChatGPT practically. When I read one of these articles or posts, I usually expect to see concrete prompts, their output, and how they can be used to speed up development. Instead all I see is generic vagueisms; "it helps me write unit tests", "it helps me code faster", or in the case of th…

I've gotten pretty good at getting ChatGPT to do what I want. I wrote about it here, https://zchry.org/words/building-a-brand-identity-with-code , but that's probably still light on details for you. There really isn't any magic bullet I don't think, every case is unique and involves a lot of trial and error. The original prompt was something as simple as "You're going to help me build a plugin for figma that draws Li…

Off topic, this is super impressive website. Thanks for sharing.

Re: ChatGPT Changed How I Write Software

#33

Post by hayksaakian »

Wowfunhappy wrote: post #19
hayksaakian wrote: post #7

Earlier quoted context omitted.

to back up the author: I'll use ChatGPT with the GPT4 model and it mostly just works by simply asking it to do what I want, the same way I might ask another person. The main constraints I see right now are around 1) complex projects that don't fit into the token context, and 2) keeping up a "base prompt background info" to describe the general purpose of your program and the technical stack. Issues I've run into: 1)…

For issues 1 and 3, I usually have success asking ChatGPT to correct them. Ie "Can you write this with fewer callbacks using async/await?" or "Can you make this code faster to execute?"

Right that's what I mean, it requires "fiddling" to resolve. I think these could both be solved with a larger context, where you could include a style guide.

Re: ChatGPT Changed How I Write Software

#34

Post by davidmurdoch »

dimgl wrote: post #2

Yeah I'm not convinced. As usual with most pro-ChatGPT posts I see, there is little to no actual useful information on how to use ChatGPT practically. When I read one of these articles or posts, I usually expect to see concrete prompts, their output, and how they can be used to speed up development. Instead all I see is generic vagueisms; "it helps me write unit tests", "it helps me code faster", or in the case of th…

I had a complex depth-first recursive reducer that I wanted to change to iterative but was struggling to fully grock and implement myself. I tried Chat GPT 3.5 and it failed miserably. I sent the recursive code to a coworker who has Chat GPT-4 and not only did it create an iterative solution correctly on the first prompt, it pointed out that the original code had an off-by-one error that occurrs in only 1/128 of cases, and due to compounding rounding, manifests in real world situations even less often.

So GPT 3.5 is great for doing trivial things I already know how to do AND describe accurately, things I'd ask a junior engineer to tackle, but GPT 4 seems to be able to do things I struggle with. I'll probably subscribe now.

Re: ChatGPT Changed How I Write Software

#35

Post by seba_dos1 »

unshavedyak wrote: post #18
darthbanane wrote: post #8

Earlier quoted context omitted.

I'm in the same boat I have yet to find an actual use case for chatgpt beyond a google/stackoverflow search that uses the same variable names as my code. In several occasions it has produced code that took me longer to fix than just reading the documentation and in almost all of those cases it couldn't help me find the issue and I actually had to read the documentation anyway (it took just seconds to pinpoint the mis…

I use it (GPT4) to explore ideas. Come up with names, explore domains i'm unfamiliar with, etc - all while assuming everything is wrong and i need to validate everything. Which is basically the same as Google/etc. I just use it as a frontend for Search, generally. I don't use it to write anything concrete, i have not found it to be good at that. I also use FastGPT (Kagi's GPT) as a .. faster search frontend. FastGPT…

Yeah, I can see it being "worth the cost", especially since the cost isn't high. It surely does not "transform" or "revolutionize" the way programmers work though; unless programmers in question were simply thoughtlessly assembling snippets from StackOverflow anyway.

Re: ChatGPT Changed How I Write Software

#36

Post by wudangmonk »

tuchsen wrote: post #31
dimgl wrote: post #2

Yeah I'm not convinced. As usual with most pro-ChatGPT posts I see, there is little to no actual useful information on how to use ChatGPT practically. When I read one of these articles or posts, I usually expect to see concrete prompts, their output, and how they can be used to speed up development. Instead all I see is generic vagueisms; "it helps me write unit tests", "it helps me code faster", or in the case of th…

I'm sure others will reply with their own, but I can give you several examples where it specifically sped up some of my development. I'll start with just the one though, because otherwise this could get quite long. As a weekend project I did a DSL for helping to interact with LLM's called Prompt (prlang.com), kinda like an declarative LangChain. The language is unimportant, what is important is that syntax highlighti…

This example seems to fall into the same shallow-knowledge or unfamiliarity with the domain which has been the conclusion of everyone that has tried to use chatgpt in their workflow.

If I'm doing something using a language/tool I'm not familiar with then chatgpt is useful because it is better than searching the docs since a lot of documentation is usually a wall of text with no usage examples.

Other than that, all the examples people provide regarding how much of a 'game-changer' chatgpt is always fall into the same google/stackoverflow use case at best or the vague planning/designing/unit-test which tells nothing at all.

Re: ChatGPT Changed How I Write Software

#37

Post by shmoogy »

jchw wrote: post #17
calmoo wrote: post #3

Earlier quoted context omitted.

Have you tried GPT-4? Its at least an order of magnitude better than 3.5,and has rarely produced code that doesn't compile for me. The difference is night and day.

My problem with GPT-4 so far is that it's most impressive at doing things that I don't really need any help with, but when I ask it something that I find genuinely challenging, it's error prone and often generates suboptimal code. Of course, it's an extreme far cry from Markov chains and earlier models, but at the end of the day, I think it's skipping too many steps right now. To be clear, it's still really impressiv…

I think things you don't need help with are exactly the good case where it shines. You give it some context, and get back working (or close to it) code - make any modifications necessary and ask it to generate unit tests. It's easy to validate and saves you 15-30 mins or more.

Half the time I ask it for help with something completely foreign and I can't properly diagnose or fix the issues - without taking the time to read the docs and/or looking it up on Reddit or stackoverflow. The other half of the time it works perfectly-ish.

I don't know about you guys - but a lot of shit I write is not a world-changing problem - if it's something already asked on stackoverflow - you basically get a code block that does what you need, with your variables and comments and unit tests already pretty much done.

Re: ChatGPT Changed How I Write Software

#38

Post by tuchsen »

wudangmonk wrote: post #36
tuchsen wrote: post #31

Earlier quoted context omitted.

I'm sure others will reply with their own, but I can give you several examples where it specifically sped up some of my development. I'll start with just the one though, because otherwise this could get quite long. As a weekend project I did a DSL for helping to interact with LLM's called Prompt (prlang.com), kinda like an declarative LangChain. The language is unimportant, what is important is that syntax highlighti…

This example seems to fall into the same shallow-knowledge or unfamiliarity with the domain which has been the conclusion of everyone that has tried to use chatgpt in their workflow. If I'm doing something using a language/tool I'm not familiar with then chatgpt is useful because it is better than searching the docs since a lot of documentation is usually a wall of text with no usage examples. Other than that, all th…

That I can know almost nothing about VimScript and Hljs, yet be productive in both, is the game changer! It immediately makes me more ambitious and confident in my personal projects, because I know that at least, I can "fake it".

In a lot of cases (like my given example) the faking it is all I need. The only thing I needed to know about that JS syntax highlighting library was how to import it, the LLM handled the rest.

It saved me a ton of time! I don't think anyone is claiming that ChatGPT is going to immediately replace programmers anymore, but it is increasing the speed at which I can grind through problems I don't have deep knowledge in.

I wonder if these disagreements in ChatGPT's usefulness come down to peoples programming domains. I can see how if you're very knowledgeable and productive in a particular space, like say embedded C programming, and that's the only code you ever write, then ChatGPT doesn't have a lot to offer, because you already know everything you need to know to be effective.

ChatGPT's nice for exploratory programming, it makes me a more creative person, in the literal sense that I'm creating more stuff now then I ever have.

Re: ChatGPT Changed How I Write Software

#39

Post by anotherhue »

I'm one of those annoying people who won't shut up about NixOS and thinks the LLMs are a bit of a fad, but chatGPT4 blew me away recently when debugging build issues with Nix derivations.

Whatever combination of the nixlanguage being obscure but also concise, but also, one of the largest github repos seems to have left ChatGPT with a fantastic understanding of the language.

Often you can just paste the error message in unprompted and it will guide you on your way, suggesting missing package names or bad assumptions.

* https://chat.openai.com/share/3b554aa0-5f1b-49e2-bce7-cfe60b...

* https://chat.openai.com/share/dfd30c49-505f-4b95-b6cb-759b7f...

* https://chat.openai.com/share/55dfcbed-2f9d-457f-b1ca-5486fe...

Re: ChatGPT Changed How I Write Software

#40

Post by calmoo »

dns_snek wrote: post #29
calmoo wrote: post #3

Earlier quoted context omitted.

Have you tried GPT-4? Its at least an order of magnitude better than 3.5,and has rarely produced code that doesn't compile for me. The difference is night and day.

Yes GPT4 is marginally better, no it doesn't significantly change its usefulness for software development. It still misunderstands questions and tasks, it still forgets or ignores constraints almost immediately, it still uses libraries that don't exist, it still generates tons of plausible-sounding but ultimately flawed code that wastes more time than it saves (unless you couldn't code your way out of a wet paper bag…

That hasn't been my experience at all - on the other hand I wouldn't consider myself an exceptional programmer so maybe it's more helpful to me day-to-day, but it has certainly saved me many hours so far.

GPT4 is definitely more than marginally better than GPT3.5

Post reply on HN