Did it grind out practice on LeetCode for a few weeks first?
Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
111–120 of 348 posts
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#112Earlier quoted context omitted.
You don't understand the take that just because ChatGPT can pass a coding interview doesn't mean the coding interview is useless or that ChatGPT could actually do the job? What part of that take do you not understand? It's a really easy concept to grasp, and even if you don't agree with it, I would expect at least that a research scientist (according to your bio) would be able to grok the concepts almost immediately.…
> doesn't mean the coding interview is useless or that ChatGPT could actually do the job Aren't these kind of mutually exclusive, at least directionally? If the interview is meaningful you'd expect it to predict job performance. If it can't predict job performance then it is kind of useless. I guess you could play some word games here to occupy a middle ground ("the coding interview is kind of useful, it measures som…
Finding a single counterexample does not disprove correlation or predictive ability. A hiring test can have both false positives and false negatives and still be useful.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#113I think tests should be easy for ChatGPT to pass. It has been trained on data that has the answers and it's good at getting the data. I'm starting to doubt its long term usefulness since it does not seem to have good decision making abilities and even the slightest bit of cognitive ability. I suspect the current crop of AIs will find very specific functions and hit a hard stop. They will change how we function but we…
Why are you making these assumptions? Do you believe that human intelligence is based on something ethereal that cannot be recreated by machines, and if so, why?
More accurately, is it based on something ethereal that cannot be recreated by humans.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#114So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
I mean building scalable systems is not a new problem. Plenty of individuals and organizations have done it already.
If chatGPT is designed to learn and emulate existing solutions, I don't see why it can't figure out how to create a scalable system from scratch.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#115Earlier quoted context omitted.
I don’t think your militant attitude helps them understand any better.
I don't think I had a militant attitude, but I do think saying, "I don't understand..." rather than "I disagree with..." puts a sour note on the entire conversation.
You literally went to their profile and called them out about how they should be able to understand something you’re describing as so easy to understand.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#116Earlier quoted context omitted.
It doesn’t really have a model for choosing. It’s closer to pattern matching. Essentially the pattern is encoded in the training of the networks. So your query most closely matches the stuff about X, where there’s a lot of good quality training data for X. If you want Y, which is novel or rarely used, the quality of the answers varies. Not to say they’re nothing more than pattern matching. It’s also synthesizing the…
>If you want Y, which is novel or rarely used, the quality of the answers varies. Is this any really different than asking a group of humans about the novel and measuring the quality?
Couple differences:
1. Humans aren’t entirely probabilistic, they are able to recognize and admit when they don’t know something and can employ reasoning and information retrieval. We also apply sanity checks to our output, which as of yet has not been implemented in an LLM. As an example in the medical field, it is common to say “I don’t know” and refer to an expert or check resources as appropriate. In their current implementations LLMs are just spewing out BS with confidence.
2. Humans use more than language to learn and understand in the real world. As an example a physician seeing the patient develops a “clinical gestalt” over their practice and how a patient looks (aka “general appearance”, “in extremis”) and the sounds they make (e.g. agonal breathing) alert you that something is seriously wrong before you even begin to converse with the patient. Conversely someone casually eating Doritos with a chief complaint of acute abdominal pain is almost certainly not seriously ill. This is all missed in a LLM.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#117Earlier quoted context omitted.
It doesn’t really have a model for choosing. It’s closer to pattern matching. Essentially the pattern is encoded in the training of the networks. So your query most closely matches the stuff about X, where there’s a lot of good quality training data for X. If you want Y, which is novel or rarely used, the quality of the answers varies. Not to say they’re nothing more than pattern matching. It’s also synthesizing the…
>If you want Y, which is novel or rarely used, the quality of the answers varies. Is this any really different than asking a group of humans about the novel and measuring the quality?
Only when you’re asking for a memorized response. If you were at ask me to create a driver for a novel hardware device in Ada, there’s no memorized answers. I would have to work it out. I do that by creating mental models, which LLM’s don’t really have. It has a statistical encoding over the language space. Essentially, memorization.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#118So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
This comment reads like it was generated by an LLM - well done.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#119So, a LLM, trained extensively on StackOverflow and other data (possibly the plethora of LC solutions out there), is fed a bunch of LC questions and spits out the correct solutions? In other news, water is blue. It is one thing to train an AI on megatons of data, for questions which have solutions. The day ChatGPT can build a highly scalable system from scratch, or an ultra-low latency trading system that beats the c…
Don’t understand this take. If it was easy to make an LLM that quickly parsed all of StackOverflow and described new answers that most of the time worked in the timeframe of an interview, it would have been done by now. ChatGPT is clearly disruptive being the first useful chatbot in forever.
It kind of depends on the frame of the solution. Google can answer leetcode questions, leetcode's answers section can answer them as well. If ChatGPT is solving them, that's one thing, but if it's just mapping the question to a solution found somewhere, then not so impressive.
Re: Google testing ChatGPT-like chatbot 'Apprentice Bard' with employees
#120Earlier quoted context omitted.
I think you’re right in one sense, and we both agree LLMs are not sufficient. I think they are definitely the death knell for the junior python developer that slaps together common APIs by googling the answers. The same way good, optimizing C, C++, … compilers destroyed the need for wide-spread knowledge of assembly programming. 100% agreed on that. Those are the most precarious jobs in the industry. Many of those pe…
I don't think chatgpt lacks an explanation. It can explain what it's doing. It's just that it can be completely wrong or the explanation may be correct and the code wrong. I gave some code to ChatGPT asking to simplify it and it returned the correct code but off by one. It was something dealing with dates, so it was trivial to write a loop checking for each day if the new code matched in functionality the old one. Yo…
Would you trust code coming from a junior developer more?