Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

cnbc.com

Post reply on HN
Print view View original

161–170 of 348 posts

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#161

Post by eternalban »

detrites wrote: post #107

So if we divide the cost of training and running a specifically-tailored ChatGPT by $183k, at what point would the company save money were it to go with the AI, versus paying for the engineers (and their office rent, etc...)? Because I suspect that's almost certainly the kind of calculation they hoped to sit down and make were they to conclude this experiment successfully.

Do LLMs suffer from catastrophic interference? A SWE is expected to learn continuously as the dujour stack constantly changes under our feet. This can be brute force mitigated by epochal retraining from ground up but that will cost money. So the cost equation must include a periodic 'rebuild of the dev team'.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#162

Post by jeffbee »

Bukhmanizer wrote: post #79
tbalsam wrote: post #16

Earlier quoted context omitted.

If it helps, this likely is coming. I think we have a tendency to mentally move the goalposts when it comes to this kind of thing as a self-defense mechanism. Years ago this would have been a similar level of impossibility. Since all a codebase like that is is a kind of directed graph, then augmentations to the processing of the network to allow for the simultaneous parsing of and generation of this kind of code may…

I agree this would have been thought to be impossible a few years ago, but I don't think it's necessarily moving the goalposts. I don't think software engineers are really paid for their labour exactly. FAANG is willing to pay top dollar for employees, because that's how they retain dominance over their markets. Now you could say that LLMs enable Google to do what it does now with fewer employees, but the same thing…

Google already published a paper claiming to have deployed an LLM for code generation at full scale to its tens of thousands of software engineers, years ago.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#164

Post by lamontcg »

22SAS wrote: post #5

So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…

> The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the competition, or find bugs in the Linux kernel and solve them

Much more mundanely the thing to focus on would be producing maintainable code that wasn't a patchwork, and being able to patch old code that was already a patchwork without making things even worse.

A particularly difficult thing to do is to just reflect on the change that you'd like to make and determine if there are any relevant edge conditions that will break the 'customers' (internal or external) of your code that aren't reflected in any kind of tests or specs--which requires having a mental model of what your customers actually do and being able to run that simulation in your head against the changes that you're proposing.

This is also something that outsourced teams are particularly shit at.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#165

Post by pixl97 »

wil421 wrote: post #151
pixl97 wrote: post #97

Earlier quoted context omitted.

Why are you making these assumptions? Do you believe that human intelligence is based on something ethereal that cannot be recreated by machines, and if so, why?

It’s a machine doing calculations on inputs you give it. The day it says no I’d rather paint pictures I might be shocked. It’s so bad that we had to redefine the word AI in last 20 years into AIG so we could start saying we have AI.

Pray tell, why would you want to develop a being with informational superpowers and the behavior of a teenager?

There is a problem with AI, but it's not with the A part, it's with the I part. I want you to give me an algorithmic description of scalable intelligence that covers intelligent behaviors at the smallest scales of life all the way to human behaviors. I know you cannot do this has many very 'intelligent' people have been working on this problem for a long time and have not come up with an agreed upon answer. The fact you see an increase and change in definitions as a failure seems pretty sad to me. We have vastly increased our understanding of what intelligence is and that previous definitions have needed to adapt and change to new information. This occurs in every field of science and is a measure of progress, again that you see this differently is worrying.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#166

Post by gedy »

dpedu wrote: post #78

My experience with asking ChatGPT to write code is that is produces code that LOOKS like it will work and solve the question asked but it actually doesn't. For example, I've asked it to create code examples of how to use different features in some Python libraries. The samples it produces make me think "ok, that's exactly how I would expect X feature in this library to work", but upon more a detailed inspection, I fi…

I've asked it to make React components, with unit tests, etc and it works quite well. Also things like image to ASCII converter in a webpage, adding dithering, etc. Found only tiny bugs here and there.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#168

Post by dariusj18 »

morelisp wrote: post #36
mensetmanusman wrote: post #18

Earlier quoted context omitted.

Don’t understand this take. If it was easy to make an LLM that quickly parsed all of StackOverflow and described new answers that most of the time worked in the timeframe of an interview, it would have been done by now. ChatGPT is clearly disruptive being the first useful chatbot in forever.

While I think the jury is still out on whether ChatGPT is truly useful or not, passing an L3 hiring test is not evidence of that one way or another.

I just used to it write a function for me yesterday. I had previously googled a few times and came up dry, asked Chat GPT and it came out with a solution I had not considered, and was better than what I was thinking.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#169

Post by layer8 »

spaceman_2020 wrote: post #114
22SAS wrote: post #5

So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…

I mean building scalable systems is not a new problem. Plenty of individuals and organizations have done it already. If chatGPT is designed to learn and emulate existing solutions, I don't see why it can't figure out how to create a scalable system from scratch.

ChatGPT isn’t designed to learn, though. The underlying model is fixed, and would have to be continuously adjusted to incorporate new training data, in order to actually learn. As far as I know, there is no good way yet to do that efficiently.

Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees

#170

Post by silveroriole »

fatjokes wrote: post #87
fsociety wrote: post #17

This puts the stake in the ground that coding interviews today are more about memorization than testing for understanding.

If a candidate can memorize as much as ChatGPT I think they're worth $183k.

Are your best engineers the ones who have the most facts and algorithms memorised?
Post reply on HN