I feel this made me loose the respect I still had for Google
Google CEO says more than a quarter of the company's new code is created by AI
281–290 of 1001 posts
Re: Google CEO says more than a quarter of the company's new code is created by AI
#282Earlier quoted context omitted.
Writing a prime-number factorization function is hardly "leetcode".
Empirical testing (for example: https://news.ycombinator.com/item?id=33293522 ) has established that the people on Hacker News tend to be junior in their skills. Understanding this fact can help you understand why certain opinions and reactions are more likely here. Surprisingly, the more skilled individuals tend to be found on Reddit (same testing performed there).
Where is the data?
Re: Google CEO says more than a quarter of the company's new code is created by AI
#283To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…
Most programming is trivial. Lots of non-trivial programming tasks can be broken down into pure, trivial sections. Then, the non-trivial part becomes knowing how the entire system fits together. I've been using LLMs for about a month now. It's a nice productivity gain. You do have to read generated code and understand it. Another useful strategy is pasting a buggy function and ask for revisions. I think most programm…
It's reasonable to say that LLMs are not completely useless. There is also a very valid case to make that LLMs are not good at generating production ready code. I have found asking LLMs to make me Nix flakes to be a very nice way to make use of Nix without learning the Nix language.
As an example of not being production ready: I recently tried to use ChatGPT-4 to provide me with a script to manage my gmail labels. The APIs for these are all online, I didn't want to read them. ChatGPT-4 gave me a workable PoC that was extremely slow because it was using inefficient APIs. It then lied to me about better APIs existing and I realized that when reading the docs. The "vibes" outcome of this is that it can produce working slop code. For the curious I discuss this in more specific detail at: https://er4hn.info/blog/2024.10.26-gmail-labels/#using-ai-to...
Re: Google CEO says more than a quarter of the company's new code is created by AI
#284Earlier quoted context omitted.
If the documentation is out of date, such that it doesn't help, this doesn't bode well for the training data of the AI helping it get it right, either?
AI can presumably integrate all of the forum discussions talking about how people really use the code. Assuming discussions don't happen in Slack, or Discord, or...
Unfortunately, it often hallucinates wrong parameters (or gets their order wrong) if there are multiple different APIs for similar packages. For example, there are plenty ML model inference packages, and the code suggestions for NVIDIA Triton Inference Server Python code are pretty much always wrong, as it generates code that’s probably correct for other Python ML inference packages with slightly different API.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#285Earlier quoted context omitted.
Supermaven is an incredible intellisense. Most code IS trivial and I barely write trivial code anymore. My imports appear instantly, with high accuracy. I have lots of embedded SQL queries and it’s able to guess the structure of my database very accurately. As I’m writing a query the suggested joins are accurate probably 80% of the time. I’m significantly more productive and having to type much less. If this is as go…
This is all about the tooling most companies choose when building software: Things with more than enough boilerplate most code is trivial. We can build tools that have far less triviality and more density, where the distance between the code we write and business logic is very narrow.. but then every line of code we write is hard, because it's meaningful, and that feels bad enough to many developers, so we end up wit…
>but then every line of code we write is hard, because it's meaningful, and that feels bad enough to many developers,
I don't know that I've ever even met a developer who wants to be writing endless pools of trivial boilerplate instead of meaningful code. Even the people at work who are willing to say they don't want to deal with the ambiguity and high level design stuff and just want to be told what to do pretty clearly don't want endless drudgery.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#286To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…
Google's internal codebase is nicer and more structured than the average open source code base.
Their internal AI tools are presumably trained on their code, and it wouldn't surprise me if the AI is capable of much more internally than public coding AIs are.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#287Earlier quoted context omitted.
We collectively deride leetcoding interviews yet ask AI to flawlessly solve leetcode questions. I bet I'd make more errors on my first try at it.
Writing a prime-number factorization function is hardly "leetcode".
I didn't say it's hard, but it's most definitely leetcode, as in "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question".
If that doesn't satisfy, here's a similar one at leetcode.com: https://leetcode.com/problems/distinct-prime-factors-of-prod...
I would not expect a programmer of any seniority to churn stuff like that and have it working without testing.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#288Earlier quoted context omitted.
> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…
we've decided we can't be bothered with traditional search Traditional search (at least on the web) is dying. The entire edifice is drowning under a rapidly rising tide of spam and scam sites. No one, including Google, knows what to do about it so we're punting on the whole project and hoping AI will swoop in like deus ex machina and save the day.
> Traditional search (at least on the web) is dying.
That's not my experience at all. While there are scammy sites, using the search engines as an index instead of an oracle still yields useful results. It only requires to learn the keywords which you can do by reading the relevant materials .
Re: Google CEO says more than a quarter of the company's new code is created by AI
#289Aren't we just talking about auto completion?
In that case those 25% are probably the very same 25% that were automatically generated by LTP based auto-completion.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#290this is the 2024 version of "25% of our code is now produced by outsourced resources"