Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google warns its own employees: Do not use code generated by Bard

theregister.com

Post reply on HN
Print view View original

71–80 of 166 posts

Re: Google warns its own employees: Do not use code generated by Bard

#71

Post by firesteelrain »

stevejobs69 wrote: post #65
samwillis wrote: post #31

Earlier quoted context omitted.

I doubt the fact they use a monorepo has any impact on this thinking. A versioned sourced repository is no different from a file server. If some "ai copyright infringed" code made it into the monorepo it doesn't compromise the whole thing. There are still clear distinct projects and products. But exactly, this is a hygiene thing. Staff will still be using these tools anyway.

as a google swe I find bard almost useless. it doesnt understand any of our internal tooling or context.

So you broke your company’s policy and Fed code to it?

Re: Google warns its own employees: Do not use code generated by Bard

#72

Post by jprete »

soultrees wrote: post #47
JoshTriplett wrote: post #29

Earlier quoted context omitted.

> The biggest concern is that AI can sometimes generate code from the training set verbatim Not just verbatim: AI-generated code is arguably a derivative work of code in the training set even if it doesn't generate verbatim copies of training data, in the same way that any code can be a derivative work even if it doesn't contain bitwise-identical lines. If you take a piece of Python code, and translate it to C code,…

Everything you do or know is a derivative of your own training set. True original thoughts without context don’t exist, well at least in any way we can find out as every human is the product of their own training set. Just out curiosity, what makes the code you write, more original than an LLMs who’s training set is way bigger than yours and likely will more variance in how to achieve same goal. I’m not trying to be…

That’s not legally true.

I don’t even think it’s factually true.

Re: Google warns its own employees: Do not use code generated by Bard

#73

Post by jprete »

londons_explore wrote: post #64
dacryn wrote: post #61

Earlier quoted context omitted.

as someone who worked for a car manufacturer, you are wrong. What people perceive as a quality car, is often a lack of understanding. Some think they are driving a fine car, but someone who pays attention notices that the door gaps are not uniform, that the plastic on the inside will start to smell bad in summer, that you will get a lot more fatigue because of wind noise and tire noise, that the sound of the door bei…

The very definition of a quality product is one that the user is happy with. If the user doesn't notice the door gaps, then it isn't a quality issue. If the user doesn't drive the car enough to wear out the suboptimal gearbox, then it isn't a quality issue. Yes, this means that the definition of quality depends on the needs of the user.

That’s very explicitly not the definition of “quality”.

Re: Google warns its own employees: Do not use code generated by Bard

#74

Post by threeseed »

bPspGiJT8Y wrote: post #33
buro9 wrote: post #21

This is Google, who have a monorepo, who are afraid of the legal risks to the monorepo. The copyright of the output of these tools is not yet determined, it's a risk. The advice not to use the tools in a risky way not only directly mitigates the risk but like an "employees must wash hands" sign in a restaurant restroom, it can be seen as a reasonable step that transfers some of the liability to the staff member. It's…

> who are afraid of the legal risks to the monorepo You're talking about a company which happily pays billions in fines because its just a fraction of what they otherwise manage to get away with. They're not at all afraid of doing a lot of illegal stuff, I find it highly unlikely they'd be afraid to step into a minor grey area.

Yes their Legal/PR teams may recommend that they quietly settle cases.

But that doesn't mean that the company will tolerate widespread illegality or allow systemic risks to build up. It's simply not how organisations do things especially those are highly public. Poor reputation doesn't just manifest in lower sales but also introduces unnecessary regulatory risks.

Re: Google warns its own employees: Do not use code generated by Bard

#76

Post by michaelt »

ilitirit wrote: post #49
redbell wrote: post #35

> Cautioning its own workers not to directly use code generated by Bard undermines Google's claims its chatbot can help developers become more productive The definitive definition of irony o_O This reminds me of two pictures that were circulating on social media a while ago. The first was Zuckerberg holding a book while billions of people are wasting their time in endless scrolling in the Metas; the second was that T…

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 Column2 The suggested improvement: WHERE Column1 Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more produ…

> I'm trying to understand how this helps me become more productive...

LLMs writing code have strange performance characteristics. Sometimes they produce working output many times faster than the fastest of senior developers, other times they can't understand a bug even when you point it out to them.

Try giving your LLM this prompt:

Please write a python script that loads a file named input.png, converts it to greyscale, plots a chart of the intensity of the pixels of the first row, and saves the chart as output.png . The chart should be a line chart with a blue line, and the lowest and highest points on the chart should be marked with a red cross. Thanks!

So LLMs can sometimes produce working code very fast. Other times, as you've identified, they produce completely broken results.

Re: Google warns its own employees: Do not use code generated by Bard

#77

Post by WeylandYutani »

soultrees wrote: post #47
JoshTriplett wrote: post #29

Earlier quoted context omitted.

> The biggest concern is that AI can sometimes generate code from the training set verbatim Not just verbatim: AI-generated code is arguably a derivative work of code in the training set even if it doesn't generate verbatim copies of training data, in the same way that any code can be a derivative work even if it doesn't contain bitwise-identical lines. If you take a piece of Python code, and translate it to C code,…

Everything you do or know is a derivative of your own training set. True original thoughts without context don’t exist, well at least in any way we can find out as every human is the product of their own training set. Just out curiosity, what makes the code you write, more original than an LLMs who’s training set is way bigger than yours and likely will more variance in how to achieve same goal. I’m not trying to be…

This is an interesting thought. How much do we really owe to our teachers? If it wasn't for the school that taught me to read all those years ago I wouldn't have a job!

Re: Google warns its own employees: Do not use code generated by Bard

#78

Post by weinzierl »

buro9 wrote: post #21

This is Google, who have a monorepo, who are afraid of the legal risks to the monorepo. The copyright of the output of these tools is not yet determined, it's a risk. The advice not to use the tools in a risky way not only directly mitigates the risk but like an "employees must wash hands" sign in a restaurant restroom, it can be seen as a reasonable step that transfers some of the liability to the staff member. It's…

I don't think the comparison with "employees must wash hands" signs is apt. Should we see legal action against large language model of questionable heritage produced code, the consequences will be dire for companies.

Being able to say to their (probably then ex-) employees "I told you so" will gain them nothing. There will not be nearly enough to get from the breaching ex-employee to compensate for any damages.

Re: Google warns its own employees: Do not use code generated by Bard

#79

Post by throwaway2990 »

firesteelrain wrote: post #71
stevejobs69 wrote: post #65

Earlier quoted context omitted.

as a google swe I find bard almost useless. it doesnt understand any of our internal tooling or context.

So you broke your company’s policy and Fed code to it?

You can’t break a policy which didn’t exist at the time.

Re: Google warns its own employees: Do not use code generated by Bard

#80

Post by BoxOfRain »

londons_explore wrote: post #24
lyu07282 wrote: post #23

But has anyone actually tried Bard for coding in practice? It's so awful, practically unusable. GPT4 feels several generations ahead of Bard.

Agreed. I frequently use GPT-4's code. It is especially useful coding in a domain I'm not familiar with. I recently asked it to write a blender plugin to export certain polygons to a spreadsheet (with coordinates projected and transformed in certain ways), and it did admirably. I had never used blender before, and it would have taken me hours otherwise.

I find it good for things where I understand the concept but haven't got to grips with the syntax yet. From the point of view of avoiding googling through endless blog spam and SEO word salad it's great, but it's definitely best to assume it's chatting confident rubbish until proven otherwise.
Post reply on HN