Live data from Hacker News

If AI is helping people code better, why aren't products getting better?

news.ycombinator.com

151–160 of 173 posts

Re: If AI is helping people code better, why aren't products getting better?

#151

There's a time delay between when an innovation is first adopted and when it has a real impact. You might spend anywhere from 2 months to 8 years [1] developing an application. I've seen numerous VC or bootstrapped companies that took 1.5 years to 2 years to launch, so I wouldn't expect AI to have had much an effect yet. I'm also not sure about "better"; I find Copilot is a good wingman for writing things like shell…

Of course this all depends on what you mean by "better", or for whom the product is better.

AI, like every technical advance in history, will be deemed "better" if fewer people make more money from it.

This has nothing to do with you, mister insignificant user...

Re: If AI is helping people code better, why aren't products getting better?

#152

Earlier quoted context omitted.

ChatGPT came out in late 2022 so it has been almost 2 years since the "AI is going to change everything" wave started. Looking at myself and my coworkers I don't feel there has been an uptick in developer output in that time. We all have copilot licenses as well, I don't use mine but pair with people who do and it seems cool but not for much more beyond advanced autofill. My big concern with all this stuff is it is p…

This resonates with me, but also I don’t want to presume too much about the value of the old ways of doing things. AI tools are going to drastically lower the bar of entry to programming, so it will be flooded with dilettantes, but people who can think will still apply their thinking in creative ways. It will be really interesting what happens as AI tools mature and a generation of AI-native developers figures out ho…

Maybe an unpopular opinion, but I believe that the bar of entry to programming has always been low. On Windows, some environment are just an exe away. And web dev can be done with Notepad. And there are many introductions books and video tutorials. What is hard for most people is formalism and detailing your idea as a sequence of steps, aka the programming mindset. Writing code was always easy (Before AI, we had copy-paste from the internet). But whether you're writing assembly or python, you still need the programming mindset and that's the first step. Then you need to learn software engineering (code hygiene, requirement gathering, tasks planning,...)

Re: If AI is helping people code better, why aren't products getting better?

#153
post #113

Simple, really. It's not actually helping people to code better: https://greaterdanorequalto.com/ai-code-generation-as-an-age...

It, undeniably, makes prototyping incredibly fast. I’ve been forcing myself to use Cursor for the past month, and could create a fairly functional web app, given my skills aren’t front end dev. Very sweet for one off deterministic python scripts, figuring out the UI bugs, filling up boilerplate code and especially handy with languages/frameworks you’re not that familiar with. Sure, we’re not at the “hook it up to pro…

If your github-fu was good, you could get the same result with copy-pasting. But then you would need to worry about licenses. It's very convenient that Copilot and the like don't disclose the licenses of their training data.

Re: If AI is helping people code better, why aren't products getting better?

#154

Earlier quoted context omitted.

I think you misinterpret the 'don't know how' part. LLM's are fantastic for boilerplate (which is a big part of getting things started). From there, I might not know the right incantation for (say) handling a button click, but it's not too hard to validate that an LLM-generated handler a) works, b) looks sensible, and c) fits reasonably in the codebase. In fact, most of my use of LLMs for coding is about generating s…

Yeah precisely. In my case I’ve been building a Django rest app over the last few years. I started off writing way too much of my own code rather than using plugins I had no idea existed. After finally getting ahold of ChatGPT I was able to expand my knowledge of Django tenfold, and was able to rewrite the app from the ground up using proven libraries and design decisions.

Or you could have gone to https://github.com/shahraizali/awesome-django

Re: If AI is helping people code better, why aren't products getting better?

#155
Based on my experience, at the present moment in time, I think you should be expecting more code (groundbreaking improvements in productivity), but not necessarily significantly better code. But certainly not worse code. Current generation AIs are writing tactical code absolutely brilliantly (often better than my own code), but often making very odd strategic decisions (functionally decomposing code in odd ways, for example).

But, the rate of change in this area is breathtaking. I reasonably expect my AI to improve in the coming months, or even weeks. And I find it difficult to keep up with which AIs are best for generating code at any given moment. There may be AIs that are good at reviewing multi-million-line code bases for security flaws. But I am not currently using one at the present time.

What I do know: my AI coding partner this year is writing code that is more accurate and more stylish than any AIs were producing this time last year. The code that's being produced is often strategically brilliant -- elegant, concise, only very occasionally using hard-coded constants instead of including the correct headers, and almost completely absent of "hallucinations". And I'm using it to regularly generate code in three different languages (C++ for the app server, typescript for the web client, java for the Android client application).

I've frequently found myself adopting coding conventions that my AI has shown me. I particularly like

    namespace fs = std::filesystem;
And the solution it came up with for writing a std:filebuf implementation stills leaves me speechless. I've done that a few times over my career, and the solution the AI uses is infinitely superior to anything I've ever written -- not something I've EVER seen, but clearly the horrible, never documented way the original authors of the iostream libraries MEANT people to do it, which provides substantial advantages over the way I've been doing it. And absolutely nowhere to be found in the first 30 page of google searches, or among the strangely variously broken and obsolete fragments of code on StackExchange.

But my current AI often falls short when it comes to strategic thinking. Functional decomposition is often odd. I often have to refactor code that my AI generates -- sometimes by coaching it through refactoring, and sometimes doing it myself when I move the generated code into production code. But that may change next week. Who knows?

Have I used it for debugging existing code? A couple of times. I'm not currently seeing a huge productivity boost in this area.

Today, I coached it to write me a bash shell script to generate a graph of a key development metric using gnuplot. Bash: as a former Windows programmer, bash still terrifies me. Gnuplot: a documentation set that could be called unforgiveable if you were feeling particularly generous. Took me about 20 minutes. "Change the font of the title, please". "Take input from this program, which produces a column of ISO 8601 dates, and an integer value". "Rotate the date labels 90 degrees anti-clockwise" (It mistakenly rotated them clockwise. The only flaw in an otherwise fantastic performance). Etc. It took me about 20 minutes to do what would have taken me a couple of hours. I wouldn't have done it if I didn't have an AI at my disposal.

Context: Using Claude 3.5, 40 years of very senior Windows development experience, but only about 3 years of Linux development experience.

Re: If AI is helping people code better, why aren't products getting better?

#156

There's a time delay between when an innovation is first adopted and when it has a real impact. You might spend anywhere from 2 months to 8 years [1] developing an application. I've seen numerous VC or bootstrapped companies that took 1.5 years to 2 years to launch, so I wouldn't expect AI to have had much an effect yet. I'm also not sure about "better"; I find Copilot is a good wingman for writing things like shell…

ChatGPT came out in late 2022 so it has been almost 2 years since the "AI is going to change everything" wave started. Looking at myself and my coworkers I don't feel there has been an uptick in developer output in that time. We all have copilot licenses as well, I don't use mine but pair with people who do and it seems cool but not for much more beyond advanced autofill. My big concern with all this stuff is it is p…

I haven't tried copilot yet. I have tried Claude 3.5, which is unexpectedly brilliant. I use it very regularly. But almost entirely useful for writing new code, not fixing old code.

I take your point with respect to understanding large existing codebases. AIs don't yet seem to deal well with strategic large-context thinking. But things are changing at such a furious pace, so that may change sometime much sooner than either of us expect. But you are right. At the present moment in time, AIs are not particularly good at that.

But overall, I am much more optimistic than you are. I think AIs will save us time, so that we can spend more time on issues of overall structure that make "spending months to understand a codebase" a thing of the very distant past. I have certainly worked on codebases that require "months to understand". But I would (perhaps naively) like to think that's a symptom of a disease that codebases shouldn't have, and that modern codebases usually don't have anymore (but sometimes do).

Re: If AI is helping people code better, why aren't products getting better?

#157

Earlier quoted context omitted.

ChatGPT came out in late 2022 so it has been almost 2 years since the "AI is going to change everything" wave started. Looking at myself and my coworkers I don't feel there has been an uptick in developer output in that time. We all have copilot licenses as well, I don't use mine but pair with people who do and it seems cool but not for much more beyond advanced autofill. My big concern with all this stuff is it is p…

I haven't tried copilot yet. I have tried Claude 3.5, which is unexpectedly brilliant. I use it very regularly. But almost entirely useful for writing new code, not fixing old code. I take your point with respect to understanding large existing codebases. AIs don't yet seem to deal well with strategic large-context thinking. But things are changing at such a furious pace, so that may change sometime much sooner than…

Anecdata: Claude 3.5 sometimes nails on the first try something that I can't GPT4 (any version) to ever complete without errors. Most recently, an AHK 2.0 script concatenating some strings.

Re: If AI is helping people code better, why aren't products getting better?

#158
post #9

We're still very much in the early days. Code AI tools today absolutely crush at creating proof of concept apps. You can test your idea and get market validation in days vs months. They are getting better at medium/large codebases, but still have a ways to go before being super useful and it translating to a huge increase in productivity. Currently it's really good for helping with the menial tasks (creating docs, un…

Not discounting your claim, but the fact that you work at a company (Sourcegraph) whose business is literally AI coding / coding assistants calls your objectivity somewhat into question. I will agree that they're very useful for boilerplate tasks particularly around deployment (cloudformation, github actions, etc.)

Def don't take my word at face value. I'm just sharing my experience and knowledge, but the great thing about most of these tools is that many offer usable free tiers or very low-cost plans that don't require any sort of lock in. So try the tools yourself, for your use cases, and see if it's a fit or not.

Personally, I have seen a ton of change in the last 4-5 months, so if you haven't tried these tools recently, I encourage you to try them today and see what's possible.

Re: If AI is helping people code better, why aren't products getting better?

#160
post #125

Because Code ≠ Product. Code ∈ Product, among many other things. I would go even further and say that that relationship between the two is weak, but also very peculiar: bad code can ruin a good product; but good code alone says very little, if anything at all, about the quality of a product.

Another riff on this is that you won’t get better products, but more products. AI reduces the barrier of entry and you’ll have many products doing niche things (often for free!) instead of platforms capturing a lot of the incremental value. It’s like mobile cameras, they reduced barrier of entry to photography, but you didn’t end up getting better photos necessarily. Instead you had more of them.

> It’s like mobile cameras, they reduced barrier of entry to photography, but you didn’t end up getting better photos necessarily. Instead you had more of them.

I like this idea, any new technology that empowers people simply increases volume, there's still an uncaptured element of strategy, design, and taste that doesn't improve with the tools themselves getting better.

Post reply on HN