Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google warns its own employees: Do not use code generated by Bard

theregister.com

Post reply on HN
Print view View original

91–100 of 166 posts

Re: Google warns its own employees: Do not use code generated by Bard

#91

Post by wkat4242 »

ilitirit wrote: post #49
redbell wrote: post #35

> Cautioning its own workers not to directly use code generated by Bard undermines Google's claims its chatbot can help developers become more productive The definitive definition of irony o_O This reminds me of two pictures that were circulating on social media a while ago. The first was Zuckerberg holding a book while billions of people are wasting their time in endless scrolling in the Metas; the second was that T…

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 Column2 The suggested improvement: WHERE Column1 Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more produ…

Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.

Re: Google warns its own employees: Do not use code generated by Bard

#92

Post by rsynnott »

ec109685 wrote: post #50
vsskanth wrote: post #3

Yeah Google Bard seems to make up stuff way more than chatgpt and gives just wrong code. I asked for numpy code to fit a polyline rejecting outliers and it just made up an argument "robust" for polyfit. Chatgpt on the other hand generated a function that calculated standard deviation and removed points outside two sigma.

Here’s an example where Bard is making up studies to support RFK’s thesis that drinking water causes transgenderism: https://twitter.com/EbenThurston/status/1670619027608092674?... It’s flat out lying, yet folks are using its output in arguments.

Hadn't realised he was on the General Ripper bandwagon.

Re: Google warns its own employees: Do not use code generated by Bard

#93

Post by giancarlostoro »

martin8412 wrote: post #82
soultrees wrote: post #46

Earlier quoted context omitted.

This is less of a productivity risk than a legal risk. Nobody is denying that these tools enable productivity like we haven’t seen before, but google is afraid of outputting protected code while the courts haven’t fully decided the stance on things. Imagine Google Bard outputs a piece of code for a major new feature, making them a shit ton of money. It’s then found out that code was actually taken copied, almost dire…

Imagine Bard outputs code that is licensed under AGPL.. Suddenly whatever services the code gets included in needs to have their source code made public.

That alone makes me never want to use any LLM for code specifics outside of asking concepts questions when the brains just not remembering.

Re: Google warns its own employees: Do not use code generated by Bard

#94

Post by AmericanChopper »

samwillis wrote: post #31
buro9 wrote: post #21

This is Google, who have a monorepo, who are afraid of the legal risks to the monorepo. The copyright of the output of these tools is not yet determined, it's a risk. The advice not to use the tools in a risky way not only directly mitigates the risk but like an "employees must wash hands" sign in a restaurant restroom, it can be seen as a reasonable step that transfers some of the liability to the staff member. It's…

I doubt the fact they use a monorepo has any impact on this thinking. A versioned sourced repository is no different from a file server. If some "ai copyright infringed" code made it into the monorepo it doesn't compromise the whole thing. There are still clear distinct projects and products. But exactly, this is a hygiene thing. Staff will still be using these tools anyway.

If they were concerned about that as a risk, the thought that came to my mind was exposing the whole codebase to subpoenas.

Re: Google warns its own employees: Do not use code generated by Bard

#95

Post by cmrdporcupine »

wkat4242 wrote: post #91
ilitirit wrote: post #49

Earlier quoted context omitted.

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 Column2 The suggested improvement: WHERE Column1 Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more produ…

Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.

GPT4 produces dubious Rust code all the time for me, especially when I point out mistakes. It gets into a cycle of doubling down on mistakes and degrading into more and more bizarre solutions and gets into a loop of repeating itself.

It gets the rough outline of a solution OK but very easily gets lost if the problem requires 2 or 3 levels of what I guess I'd call nested analysis. Or rather, it's fine for creating a surface solution, but it gets lost in the weeds quite easily for things like type system or borrow checker problems.

Because it doesn't reason -- it (re)produces patterns.

Re: Google warns its own employees: Do not use code generated by Bard

#96

Post by styren »

wkat4242 wrote: post #91
ilitirit wrote: post #49

Earlier quoted context omitted.

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 Column2 The suggested improvement: WHERE Column1 Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more produ…

Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.

ChatGPT is ALWAYS doing stuff like this to me. Suggest code that doesn't compile, suggest SIMD instructions that don't exist, reiterate on incorrect answers with the exact same answer, etc. Granted I use ChatGPT 3.5.

Re: Google warns its own employees: Do not use code generated by Bard

#97

Post by jfk13 »

soultrees wrote: post #46
redbell wrote: post #35

> Cautioning its own workers not to directly use code generated by Bard undermines Google's claims its chatbot can help developers become more productive The definitive definition of irony o_O This reminds me of two pictures that were circulating on social media a while ago. The first was Zuckerberg holding a book while billions of people are wasting their time in endless scrolling in the Metas; the second was that T…

This is less of a productivity risk than a legal risk. Nobody is denying that these tools enable productivity like we haven’t seen before, but google is afraid of outputting protected code while the courts haven’t fully decided the stance on things. Imagine Google Bard outputs a piece of code for a major new feature, making them a shit ton of money. It’s then found out that code was actually taken copied, almost dire…

> Obviously within reason

Oh, so you don't really mean "when all copyright is out of the window", I guess. So.... how are we going to define "within reason"?

Re: Google warns its own employees: Do not use code generated by Bard

#98

Post by chii »

buro9 wrote: post #21

This is Google, who have a monorepo, who are afraid of the legal risks to the monorepo. The copyright of the output of these tools is not yet determined, it's a risk. The advice not to use the tools in a risky way not only directly mitigates the risk but like an "employees must wash hands" sign in a restaurant restroom, it can be seen as a reasonable step that transfers some of the liability to the staff member. It's…

> transfers some of the liability to the staff member.

As an employee, they take zero of the business risk, as no employee is liable for any of business's risks. It's also why CEOs don't go to jail if they're performing their duties, even if the company is liable for damages or commits a crime.

And this is the way it should be.

Re: Google warns its own employees: Do not use code generated by Bard

#99

Post by PeterStuer »

JoshTriplett wrote: post #29
tsimionescu wrote: post #19

Earlier quoted context omitted.

> AI generated works are public domain That's not the status quo. The biggest concern is that AI can sometimes generate code from the training set verbatim, in which case that code has the copyright of the original training set - which might be GPL or proprietary or whatever else.

> The biggest concern is that AI can sometimes generate code from the training set verbatim Not just verbatim: AI-generated code is arguably a derivative work of code in the training set even if it doesn't generate verbatim copies of training data, in the same way that any code can be a derivative work even if it doesn't contain bitwise-identical lines. If you take a piece of Python code, and translate it to C code,…

By the same logic all code I have ever written is a derivative work of all code I've ever read.

Re: Google warns its own employees: Do not use code generated by Bard

#100

Post by NotYourLawyer »

6gvONxR4sf7o wrote: post #25
Lerc wrote: post #6

Is this because of the accuracy of the results, or is it the rights to the content that is the problem? Barring any AI court case setting a precedent, We're still at 'AI generated works are public domain' aren't we? There's probably going to be some big money arguing for the right to own content generated by their AI. I can't really see that being in the public interest though.

> Barring any AI court case setting a precedent, We're still at 'AI generated works are public domain' aren't we? Barring a court case, we’re more in an uncertain place. I think it’s more likely for courts to hold that many model uses aren’t transformative enough to get a fair use exemption. But until a court says either way, it most definitely does NOT just default to public domain.

You’re conflating two issues.

1. Does the output infringe an existing copyright?

2. Is the output copyrightable?

Post reply on HN