So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
water is not blue btw
Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
141–150 of 348 posts
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#142My experience with asking ChatGPT to write code is that is produces code that LOOKS like it will work and solve the question asked but it actually doesn't. For example, I've asked it to create code examples of how to use different features in some Python libraries. The samples it produces make me think "ok, that's exactly how I would expect X feature in this library to work", but upon more a detailed inspection, I fi…
I’ve asked ChatGPT to write plenty of Python scripts that uses the AWS SDK and it came back correct most of the time.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#143I think as programmers we may want to rethink the amount of knowledge we share online. That knowledge is how we make money, and people are mining it to build AIs that can replace us.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#144Earlier quoted context omitted.
I've worked in ML for awhile (on the MLOps side of things) and have been in the industry for a bit, and one thing that I think is extremely common is for ML researchers to grossly underestimate the amount of work needed to make improvements. We've been a year away from full self driving cars for the last six years, and it seems like people are getting more cautious in their timing around that instead of getting more…
Based on my experience watching How It’s Made, many factories are extremely automated including lots of robots. Warehouses are not factories though.
Lookup lights out manufacturing. There are factories that often run whole days in the dark because there's no point turning on the nights if there's no one around
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#145Earlier quoted context omitted.
Don’t understand this take. If it was easy to make an LLM that quickly parsed all of StackOverflow and described new answers that most of the time worked in the timeframe of an interview, it would have been done by now. ChatGPT is clearly disruptive being the first useful chatbot in forever.
You don't understand the take that just because ChatGPT can pass a coding interview doesn't mean the coding interview is useless or that ChatGPT could actually do the job? What part of that take do you not understand? It's a really easy concept to grasp, and even if you don't agree with it, I would expect at least that a research scientist (according to your bio) would be able to grok the concepts almost immediately.…
I'll try to phrase it so that even someone who is not a research scientist (?) can understand. I'm not one, whatever that means.
Let's define the interview as useful if the passing candidate can do the job.
Sounds reasonable.
ChatGPT can pass the interview and can't do the job.
The interview is not able to predict the poor working performance of ChatGPT and it's therefore useless.
Some of the companies I worked for hired ex fang people as if it was a mark of quality, but that hasn't always worked out well. There is plenty of people getting out of fangs having just done mediocre work for a big paycheck.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#146Earlier quoted context omitted.
Agree LeetCode is one of the least surprising starting points. Any human that reads the LeetCode books and practices and remembers the fundamentals will pass a LeetCode test. But there is also a ton of code out there for highly scalable client/servers, low latency processing, performance optimizations and bug fixing. Certainly GPT it is being trained on this too. “Find a kernel bug from first principles” maybe not, b…
> Any human that reads the LeetCode books and practices and remembers the fundamentals will pass a LeetCode test. Seems pretty bold to claim "any human" to me. If it were that easy, don't you think alot more people would be able to break into software dev at FAANG and hence drive salaries down?
> Seems pretty bold to claim "any human" to me.
That's obviously not what they claimed. Your quote, "Any human that reads the LeetCode books and practices and remembers the fundamentals".
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#147Earlier quoted context omitted.
We've been saying for years these interviews are not predictive of job performance. Here's the proof. Nothing you do in an interview like this resembles day to day work in this field.
For what it's worth, when I ask these kinds of questions (rarely anymore), I'm looking more at how the problem is solved, not what the solution is. A wrong answer with good thinking is better than a correct answer with no explanation.
Chatgpt can provide you a great explanation of the how.
Oftentimes the explanation is correct, even if there's some mistake in the code (probably because the explanation is easier to generate than the correct code, an artifact of being a high tech parrot)
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#148So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
Correct me if I'm wrong, but answering questions for known answers is precisely the kind of thing a well trained LLM is built for.
It doesn't understand context, and is absolutely unable to rationalize a problem into a solution.
I'm not in any way trying to make it sound like ChatGPT is useless. Much to the opposite, I find it quite impressive. Parsing and producing fluid natural language is a hard problem. But it sounds like something that can be a component of some hypothetical advanced AI, rather than something that will be refined into replacing humans for the sort of tasks you mentioned.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#149As much as we like to say lots of software jobs are just plumbing, the current state of consumer software indicates we have a long way to go in terms of quality.
Whatever training data is fed to an AI will not be better than the data used by human engineers to write code at the macro level. Ergo, the code will be worse in quality.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#150So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
If it helps, this likely is coming. I think we have a tendency to mentally move the goalposts when it comes to this kind of thing as a self-defense mechanism. Years ago this would have been a similar level of impossibility. Since all a codebase like that is is a kind of directed graph, then augmentations to the processing of the network to allow for the simultaneous parsing of and generation of this kind of code may…
I think OP set relatively simple goals. How long until AI can architect, design, build, test, deploy and integrate commercial software systems from scratch, and handle users submitting bug reports that say "The OK button doesn't work when I click it!"?