To me, further proof of how useless ‘coding interviews’ like these really are.
Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
231–240 of 348 posts
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#232That's odd because when I gave ChatGPT my icebreaker interview question that I used a lot at Google it fell right on its face. And this is a question I expect teenagers to ace in one minute.
Without hearing the question it is hard to make judgement.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#233Earlier quoted context omitted.
You don't understand the take that just because ChatGPT can pass a coding interview doesn't mean the coding interview is useless or that ChatGPT could actually do the job? What part of that take do you not understand? It's a really easy concept to grasp, and even if you don't agree with it, I would expect at least that a research scientist (according to your bio) would be able to grok the concepts almost immediately.…
I'll try to phrase it so that even someone who is not a research scientist (?) can understand. I'm not one, whatever that means. Let's define the interview as useful if the passing candidate can do the job. Sounds reasonable. ChatGPT can pass the interview and can't do the job. The interview is not able to predict the poor working performance of ChatGPT and it's therefore useless. Some of the companies I worked for h…
> Let's define the interview as useful if the passing candidate can do the job.
The technical term for this is "construct validity", that the test results are related to something you want to learn about.
> The interview is not able to predict the poor working performance of ChatGPT and it's therefore useless.
This doesn't follow; the interview doesn't need to be able to exclude ChatGPT because ChatGPT doesn't interview for jobs. It's perfectly possible that the same test shows high validity on humans and low validity on ChatGPT.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#234Earlier quoted context omitted.
If a candidate can memorize as much as ChatGPT I think they're worth $183k.
Most candidates can, it's just a matter of how much free time they have to spend memorising.
Challenge. I can barely remember what I had for breakfast yesterday much less.. the entire knowledge base of chatgpt
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#235I feel I'm getting a different vibe that what everyone is getting. So much of leetcode questions/ CS questions aren't hard to interpret. They're hard to solve sure but that's normal. So much in the workplace though that IS hard is finding the actual correct question. I think us as software engineers will be doing more not less. It will give us opportunities to try a variety of solutions quicker. The real work is formulating/distilling the actual problem from the end-user.
We all have in our minds yeah a self-driving car, robots in a warehouse etc. But I truly hope something like Chat-GPT could be made for fields like medicine, geriatric care, education. There are actual jobs that need people that we can't find people for. A LLM to help social workers navigate the mess that is family law. A LLM to help families needing urgent care to make a prescription for a sick kid. There's a lot of opportunities we're missing here.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#236Well it has no face, no skin color, no weird looking. At this point, it reduces huge amount of failure risk already. And interview is the most non-sense process, so what does this benchmark mean?
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#237Earlier quoted context omitted.
I think you’re right in one sense, and we both agree LLMs are not sufficient. I think they are definitely the death knell for the junior python developer that slaps together common APIs by googling the answers. The same way good, optimizing C, C++, … compilers destroyed the need for wide-spread knowledge of assembly programming. 100% agreed on that. Those are the most precarious jobs in the industry. Many of those pe…
I don't think chatgpt lacks an explanation. It can explain what it's doing. It's just that it can be completely wrong or the explanation may be correct and the code wrong. I gave some code to ChatGPT asking to simplify it and it returned the correct code but off by one. It was something dealing with dates, so it was trivial to write a loop checking for each day if the new code matched in functionality the old one. Yo…
It’s a very sophisticated form of a recurrent neural network. We used to use those for generating a complete image based on a partial image. The recurrent network can’t explain why it chose to reproduce one image instead of another. Nor can you look at the network and find the fiddly bit that drive that output. You can ask a human why they chose to use an array instead of a hash map, or why static memory allocation in this area avoids corner cases. ChatGPT simply generates the most likely text as an explanation. That’s what I mean about being able to explain something.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#238[dead]
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#239So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
> or find bugs in the Linux kernel and solve them
Then we won't hear how somebody rewrote pong in Rust on HN. I worry too.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#240Earlier quoted context omitted.
If it helps, this likely is coming. I think we have a tendency to mentally move the goalposts when it comes to this kind of thing as a self-defense mechanism. Years ago this would have been a similar level of impossibility. Since all a codebase like that is is a kind of directed graph, then augmentations to the processing of the network to allow for the simultaneous parsing of and generation of this kind of code may…
I've worked in ML for awhile (on the MLOps side of things) and have been in the industry for a bit, and one thing that I think is extremely common is for ML researchers to grossly underestimate the amount of work needed to make improvements. We've been a year away from full self driving cars for the last six years, and it seems like people are getting more cautious in their timing around that instead of getting more…
Just let them bring on another AI winter.