Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google CEO says more than a quarter of the company's new code is created by AI

businessinsider.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Google CEO says more than a quarter of the company's new code is created by AI

#411

Post by nimithryn »

skrebbel wrote: post #127

To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…

An example:

I'm looking for a new job, so I've been grinding leetcode (oof). I'm an experienced engineer and have worked at multiple FAANGs, so I'm pretty good at leetcode.

Today I solved a leetcode problem 95% of the way to completion, but there was a subtle bug (maybe 10% of the test cases failing). I decided to see if Claude could help debug the code.

I put the problem and the code into Claude and asked it to debug. Over the course of the conversation, Claude managed to provide 5 or 6 totally plausible but also completely wrong "fixes". Luckily, I am experienced enough at leetcode, and leetcode problems are simple enough, that I could easily tell that Claude was mistaken. Note that I am also very experienced with prompt engineering, as I ran a startup that used prompt engineering very heavily. Maybe it's a skill issue (my company did fail, hence why I need a job), but somehow I doubt it.

Eventually, I found the bug on my own, without Claude's help. But leetcode are super simple, with known answers, and probably mostly in the training set! I can't imagine writing a big system and using an LLM heavily.

Similarly, the other day I was trying to learn about e-graphs (the data structure). I went to Claude for help. I noticed that the more I used Claude, the more confused I became. I found other sources, and as it turns out, Claude was subtly wrong about e-graphs, an uncommon but reasonably well-researched data structure! Once again, it's lucky I was able to recognize that something was up. If the problem wasn't limited in scope, I'd have been totally lost!

I use LLMs to help me code. I'm pro new technology. But when I see people bragging on Twitter about their fully automated coding solutions, or coding complex systems, or using LLMs for medical records or law or military or other highly critical domains, I seriously question their wisdom and experience.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#413

Post by jcgrillo »

giantg2 wrote: post #389
jcgrillo wrote: post #353

Earlier quoted context omitted.

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There's this bizarre misconception popular among bigtech managers that there's some tunable tradeoff bet…

"What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti." In my opinion the reason we get enterprise spaghetti is largely due to requirement issues and scope creep. It's nearly impossible to create a streamlined system without knowing what it should look like. And once the system gets to a certain…

When scopes creep and requirements change, simply refactor. Where is it written in The Law that you have to accrue technical debt? EDIT: I'm gonna double down on this one. The fact that your organization thinks they can demand of you that you can magically weathervane your codebase to their changeable whims is evidence that you have failed to realistically communicate to them what is actually possible to do well. The fact that they think it's a move you can make to creep the scope, or change the requirements, is the problem. Every time that happens it should be studied within the organization as a major, costly failure--like an outage or similar.

> it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary

That's a choice. There are some other options:

- Simply don't get business buy-in. Do without. Form a terrorist cell within your organization. You'll likely outpace them. Or you'll get fired, which means you'll get severance, unemployment, a vacation, and the opportunity to apply to a job at a better company.

- Fight viciously for engineering independence. You business people can do the businessing, but us engineers are going to do the engineering. We'll tell you how we'll do it, not the other way.

- Build companies around a culture of doing good, consistent work instead of taking expedient shortcuts. They're rare, but they exist!

Re: Google CEO says more than a quarter of the company's new code is created by AI

#414

Post by delfinom »

gerash wrote: post #177
Twirrim wrote: post #138

Earlier quoted context omitted.

> We work very closely with Google DeepMind to adapt Gemini models for Google-scale coding and other Software Engineering usecases. Considering how terrible and frequently broken the code that the public facing Gemini produces, I'll have to be honest that that kind of scares me. Gemini frequently fails at some fairly basic stuff, even in popular languages where it would have had a lot of source material to work from;…

I believe most people use AI to help them quickly figure out how to use a library or an API without having to read all their (often out dated) documentation instead of helping them solve some mathematical challenge

I've never had an AI not just make up API when it didn't exist, instead of saying "it doesn't exist". Lol

Re: Google CEO says more than a quarter of the company's new code is created by AI

#415

Post by c0balt »

jcgrillo wrote: post #353
toasteros wrote: post #144

Earlier quoted context omitted.

> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There's this bizarre misconception popular among bigtech managers that there's some tunable tradeoff bet…

It's relatively easy to find a programmer(s) who can realize enterprise project X, it's hard to find a programmer(s) who cares about X. Throwing an increased requirement like speed at it makes this worse because it usually ends up burning out both ends of the equation.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#416

Post by tru3_power »

layer8 wrote: post #305
photonthug wrote: post #239

Earlier quoted context omitted.

Maybe it is naive but I think search would probably work again if they could roll back code to 10 or 15 years ago and just make search engines look for text in webpages. Google wasn’t crushed by spam, they decided to stop doing text search and build search bubbles that are user specific, location-specific, decided to surface pages that mention search terms in metadata instead of in text users might read, etc. Oh yeah…

> just make search engines look for text in webpages. Google’s verbatim search option roughly does that for me (plus an ad blocker that removes ads from the results page). I have it activated by default as a search shortcut. (To activate it, one can add “tbs=li:1” as a query parameter to the Google search URL.)

Wow what? Thanks!

Re: Google CEO says more than a quarter of the company's new code is created by AI

#417

Post by steve_adams_86 »

jihadjihad wrote: post #393
jcgrillo wrote: post #353

Earlier quoted context omitted.

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There's this bizarre misconception popular among bigtech managers that there's some tunable tradeoff bet…

> The problem is finding people to do it with who care enough to think rigorously > ... > train them, and reject low quality work. I agree very strongly with both of these points. But I've observed a truth about each of them over the last decade-plus of building software. 1) very few people approach the field of software engineering with anything remotely resembling rigor, and 2) there is often little incentive to tr…

I agree as well. These are actually things that bother me a lot about the industry. I’d love to write software that should run problem-free in 2035, but the reality is almost no one cares.

I’ve had the good fortune of getting to write some firmware that will likely work well for a long time to come, but I find most things being written on computers are written with (or very close to) the minimum care possible in order to get the product out. Clean up is intended but rarely occurs.

I think we’d see real benefits from doing a better job, but like many things, we fail to invest early and crave immediate gratification.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#418

Post by GaggiX »

toasteros wrote: post #144
skrebbel wrote: post #127

To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…

> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…

These models are simply much more powerful than a tradition search engine and stackoverflow, so many people use these models for a reason, a friend of mine that never tried ChatGPT until very recently managed to solve a problem he couldn't find a solution on stackoverflow using GPT-4o, next time he's probably going to ask the model directly.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#420

Post by tru3_power »

consp wrote: post #332
skissane wrote: post #248

Earlier quoted context omitted.

I personally think a big problem with search is major search engines try to be all things to all people and hence suffer as a result. For example: a beginner developer is possibly better served by some SEO-heavy tutorial blog post; an experienced developer would prefer results weighted towards the official docs, the project’s bug tracker and mailing list, etc. But since less technical and non-technical people vastly…

> For example: a beginner developer is possibly better served by some SEO-heavy tutorial blog post Good luck finding those, you end op with SEO spam and clone page spam. These days you have to look for unobvious hidden meanings which only relate to your exact problem to find what you are looking for. I have the strong feeling search these days is back to the Altavista era. You'd have to use trickery to find what you…

Yeah but this is just the name of the game. How can you even stop SEO style gamification at this point? I’m sure even LLMs are vulnerable/have been trained on SEO bs. End of the day it takes an informed user. Remember back in the day? Don’t trust the internet? I think that mindset will become the main school of thought once again. Which tbh, I think maybe a good thing.
Post reply on HN