My concern is that "frequently needed and immediately useful results" is strongly correlated to "this code should already be abstracted away into a library by now."
Search Copy-Paste as a Service is hiding a deeper issue.
Google CEO says more than a quarter of the company's new code is created by AI
311–320 of 1001 posts
Re: Google CEO says more than a quarter of the company's new code is created by AI
#312While I get the MBA-speak of lines-of-code that AI is now able to accomplish, it does make me think about their highly-curated internal codebase that makes them well placed to potentially get to 50% AI-generated code. One common misconception is that all LLMs are the same. The models are trained the same, but trained on wildly different datasets. Google, and more specifically the Google codebase is arguably one of th…
> that realistically could easily replace any entry-level or junior developer. This is a massive, unsubstantiated leap.
The issue is it doesn't really replace junior dev. You become one - as you have to babysit it all the time, check every line of code, and beg it to make it work.
In many cases it is counterproductive
Re: Google CEO says more than a quarter of the company's new code is created by AI
#313Earlier quoted context omitted.
> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…
we've decided we can't be bothered with traditional search Traditional search (at least on the web) is dying. The entire edifice is drowning under a rapidly rising tide of spam and scam sites. No one, including Google, knows what to do about it so we're punting on the whole project and hoping AI will swoop in like deus ex machina and save the day.
Without a usable web search index, AI will be in trouble eventually as well. There is no substitute for it.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#314Earlier quoted context omitted.
It's reasonable to say that LLMs are not completely useless. There is also a very valid case to make that LLMs are not good at generating production ready code. I have found asking LLMs to make me Nix flakes to be a very nice way to make use of Nix without learning the Nix language. As an example of not being production ready: I recently tried to use ChatGPT-4 to provide me with a script to manage my gmail labels. Th…
I find a recurring theme in these kind of comments where people seem to blame their laziness on the tool. The problem is not that the tools are imperfect, it’s that you apparently use them in situations where you expect perfection. Does a carpenter blame their hammer when it fails to drive in a screw?
I'd argue that a closer analogy is I bought a laser based measuring device. I point it a distant point and it tells me the distance from the tip of the device to that point. Many people are excited that this tool will replace rulers and measuring tapes because of the ease of use.
However this laser measuring tool is accurate within a range. There's a lot of factors that affect it's accuracy like time of day, how you hold it, the material you point it at, etc. Sometimes these accuracy errors are minimal, sometimes they are pretty big. You end up getting a lot of measurements that seem "close enough". but you still need to ask if each one is correct. "Measure Twice, Cut Once" begins to require one measurement with the laser tool and once with the conventional tool when accuracy matters.
One could have a convoluted analogy where the carpenter has an electric hammer that for some reason has a rounded head that does cause some number of nails to not go in cleanly, but I like my analogy better :)
Re: Google CEO says more than a quarter of the company's new code is created by AI
#315To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…
I'll just answer here, but this isn't about this post in particular. It's about all of them. I've been struggling with a team of junior devs for the past months. How would I describe the experience? It's easy: just take any of these posts, replace "AI" with "junior dev", done.
Except of course AI at least can do spelling. (Or at least I haven't encountered a problem in that regard.)
I'm highly skeptical regarding LLM-assisted development. But I must admit: it works. If paired with an experienced senior developer. IMHO it must not be used otherwise.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#316I’m gonna bet this is a lie.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#317While I get the MBA-speak of lines-of-code that AI is now able to accomplish, it does make me think about their highly-curated internal codebase that makes them well placed to potentially get to 50% AI-generated code. One common misconception is that all LLMs are the same. The models are trained the same, but trained on wildly different datasets. Google, and more specifically the Google codebase is arguably one of th…
Is the public gemini code gen LLM trained on their internal repo? I wonder if one could get it to cough up propriety code with the right prompt.
I’m curious if Microsoft lets OpenAI train on GH private repos.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#318Earlier quoted context omitted.
In lisp about 50% of the code is just closing parentheses.
Heh, but it can't be that, no reason to think llms can count brackets needing a close any more than they can count words.
Logically, it couldn't be 50% since that would imply that the other 50% would be open brackets and that would leave 0% room for macros.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#319Earlier quoted context omitted.
I didn't say it's hard, but it's most definitely leetcode, as in "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". If that doesn't satisfy, here's a similar one at leetcode.com: https://leetcode.com/problems/distinct-prime-factors-of-prod... I would not expect a programmer of any seniority to churn stuff like that and have it working without testing.
> "pointless algorithmic exercise that will only show you if the candidate recently worked on a similar question". I've been able to write one, not from memory but from first principles, any time in the last 40 years.
Curious, I would expect a programmer of your age to remember Knuth's "beware of the bugs in above code, I have only proven it's correct but haven't actually run it".
I'm happy you know math, but my point before this thread got derailed was that we're holding (coding) AI to a higher standard than actual humans, namely to expect to write bug-free code.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#320To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…
I'll just answer here, but this isn't about this post in particular. It's about all of them. I've been struggling with a team of junior devs for the past months. How would I describe the experience? It's easy: just take any of these posts, replace "AI" with "junior dev", done. Except of course AI at least can do spelling. (Or at least I haven't encountered a problem in that regard.) I'm highly skeptical regarding LLM…
> replace "AI" with "junior dev", done.
Damn, that’s a good way of putting it. But I’ll go one further:
replace "AI" with "junior dev who doesn’t like reading documentation or googling how things work so instead confidently types away while guessing the syntax and API so it kind of looks right”