Tips for programmers to stay ahead of generative AI • Hacker News Board

Re: Tips for programmers to stay ahead of generative AI

#141

Post by abalashov » Wed, Jul 05, 2023, 2:39 AM UTC

wouldbecouldbe wrote: post #62

arrowsmith wrote: post #53

Earlier quoted context omitted.

I find it increases my productivity about 5-10% when working with the technologies I'm the most familiar with and use regularly (Elixir, Phoenix, JavaScript, general web dev.) But when I'm doing something unfamiliar and new, it's more like 90%. It's incredible. Recently at work, for example, I've been setting up a bunch of stuff with some new technologies and libraries that I'd never really used before. Without ChatG…

The thing is the hallucinations, I also wasted few hours trying to work on solutions with GPT where it just kept making up parameters and random functions.

So much this. The thing hallucinates far more than the hyperventilation seems willing to acknowledge.

You really need to be quite competent in the thing you're asking it to do in order to ferret out the hallucinations, which greatly diminishes the potency of GPT in the hands of someone who has no knowledge of the relevant language/runtime/problem domain/etc.

Re: Tips for programmers to stay ahead of generative AI

#142

Post by briHass » Wed, Jul 05, 2023, 2:41 AM UTC

irrational wrote: post #77

I work for a Fortune 100 company. Recently an email was sent to all 100,000 employees saying that nobody was allowed to use DALL-E 2, ChatGPT, Codex, Stable Diffusion, Midjourney, Microsoft’s Copilot, and GitHub Copilot, etc. due to concerns about those tools using other people’s IP (meaning our company might end up illegally using their IP) or the potential that the tools might get a hold of our IP through our use o…

Same here. Anyone that works in a highly-regulated industry doing software (e.g. finance, healthcare) is probably not going to see much AI pressure on programmers until the legal quagmire is cleared up. There are privacy concerns with the data, the same ownership/copyright problems often discussed, and ultimately, there needs to be someone (a human) to take accountability (blame) if everything falls down horribly.

Re: Tips for programmers to stay ahead of generative AI

#143

Post by klysm » Wed, Jul 05, 2023, 2:44 AM UTC

yawnxyz wrote: post #132

LLM-based regex generation is probably my absolute favorite thing. I give it a bunch of examples and say "create a regex of that" Does it hallucinate? Probably! Do I "hallucinate" while trying to clobber together terrible regexes? Absolutely

This is what makes me nervous. Is it not bad to be generating stuff you don’t understand that _might_ work?

Re: Tips for programmers to stay ahead of generative AI

#144

Post by briHass » Wed, Jul 05, 2023, 2:51 AM UTC

varispeed wrote: post #42

laputan_machine wrote: post #12

Earlier quoted context omitted.

> I also find it bizarre that so many people feel precious about the code. Would you find it bizarre that a joiner feels precious about not just the cabinet they made (value), but how they made it? The joints they used, the process they went through, the wood (i.e., the code )? A plumber, electrician, architect, designer, programmer -- we take pride in our skills. Craftmanship is a virtue, not a vice.

Your analogy suggests that the code is the final product, akin to a cabinet or a building, something tangible that can be appreciated for its craftsmanship. In some instances, like open-source software, the code might indeed be viewed this way, but in most cases, it's not the code itself that end-users appreciate, it's the functionality it provides. To refine your analogy, the code isn't the cabinet - it's more like…

This is perhaps true for shrink-wrapped software (in so far as that still exists), but for B2B SaaS products, the ability to easily maintain and enhance the codebase is vital to the long-term success of the product.

Maybe it won't actually matter, because if AI generates a 5MM line ball-of-mud, it will be able to easily add features later due to the code being styled in alignment with its training, or maybe the context size limitations will allow future systems to digest the entire thing. It could end up being like coding in a very high-level language: who cares what crazy bytecode is kicked out as long as it performs within expectations.

Re: Tips for programmers to stay ahead of generative AI

#145

Post by throwuwu » Wed, Jul 05, 2023, 2:56 AM UTC

pydry wrote: post #94

coliveira wrote: post #82

Earlier quoted context omitted.

I believe that AI will get so good at creating new code that a lot of existing libraries will be let unused. What is the point of using lots of libraries if AI can generate the code we need directly? The AI will be the library itself, and the generated code will embed the knowledge about doing lots of things for which we used libraries.

>What is the point of using lots of libraries if AI can generate the code we need directly? Theyve been debugged.

At some point the models will produce code with a lower error rate than existing libraries.

Re: Tips for programmers to stay ahead of generative AI

#146

Post by skissane » Wed, Jul 05, 2023, 2:58 AM UTC

wouldbecouldbe wrote: post #62

arrowsmith wrote: post #53

Earlier quoted context omitted.

I find it increases my productivity about 5-10% when working with the technologies I'm the most familiar with and use regularly (Elixir, Phoenix, JavaScript, general web dev.) But when I'm doing something unfamiliar and new, it's more like 90%. It's incredible. Recently at work, for example, I've been setting up a bunch of stuff with some new technologies and libraries that I'd never really used before. Without ChatG…

The thing is the hallucinations, I also wasted few hours trying to work on solutions with GPT where it just kept making up parameters and random functions.

There's a lot of things which could be done to improve this:

1) It could use the JSONformer idea [0] where we have a model of the language which determines what are the valid next tokens; we only ask it to supply a token when the language model gives us a choice, and when considering possible next tokens, we immediately ignore any which are invalid given the model. This could go beyond mere syntax to actually considering the APIs/etc which exist, so if the LLM has already generated tokens "import java.util.", then it could only generate a completion which was a public class (or subpackage) of "java.util.". Maybe something like language servers could help here.

2) Every output it generates, automatically compile and test it before showing it to the user. If compile/test fails, give it a chance to fix its mistake. If it gets stuck in a loop, or isn't getting anywhere after several attempts, fall back to next most likely output, and repeat. If after a while we still aren't getting anywhere, it can show the user its attempts (in case they give the user any idea).

[0] https://github.com/1rgs/jsonformer

Re: Tips for programmers to stay ahead of generative AI

#147

Post by jes5199 » Wed, Jul 05, 2023, 3:04 AM UTC

tehwebguy wrote: post #72

wouldbecouldbe wrote: post #62

Earlier quoted context omitted.

The thing is the hallucinations, I also wasted few hours trying to work on solutions with GPT where it just kept making up parameters and random functions.

It referenced a made up a function I needed (that should probably exist lol) in BrightScript, the letdown after realizing as much was painful.

we should probably implement a lot of the hallucinated methods - consider them the obvious missing pieces of our APIs

Re: Tips for programmers to stay ahead of generative AI

#148

Post by lotsoweiners » Wed, Jul 05, 2023, 3:05 AM UTC

I feel the biggest tip for anyone (not just programmers) is “talk to your congressman”. AI has enough of a bad rep whether fictional or not that i think getting bans/fines/taxes/other in place will be far more useful for the little man than whatever this article recommends.

Re: Tips for programmers to stay ahead of generative AI

#149

Post by throwuwu » Wed, Jul 05, 2023, 3:05 AM UTC

klysm wrote: post #143

yawnxyz wrote: post #132

LLM-based regex generation is probably my absolute favorite thing. I give it a bunch of examples and say "create a regex of that" Does it hallucinate? Probably! Do I "hallucinate" while trying to clobber together terrible regexes? Absolutely

This is what makes me nervous. Is it not bad to be generating stuff you don’t understand that _might_ work?

Most of the complex regexes I write fall into the category of stuff I don’t understand that might work. I’m no slouch at regexes either but when you start trying to do weird data processing stuff that handles all kinds of edge cases the wheels really come off quick.

Re: Tips for programmers to stay ahead of generative AI

#150

Post by SrslyJosh » Wed, Jul 05, 2023, 3:07 AM UTC

impissedoff1 wrote: post #59

I sure picked the worst time to be unemployed and look at a coding career again after a decade

LLMs might be useful for churning out vaguely correct-looking code quickly, but they're just regurgitating the contents of their training corpus. There's no guarantee of correctness, and it's only a matter of time before someone dies because of an LLM-generated bug.

Human programmers aren't going anywhere. (You can't even call what LLMs do programming, because there's no intent or understanding behind it.)