Earlier quoted context omitted.
Not sure I agree, but to be fair my perspective is informed by this talk I watched not too long ago https://www.youtube.com/watch?v=qmJ4xLC1ObU
I've now watched that video, and I'm going to say what I disagree with. First, he claims that Copilot/GPT only needs more data and more compute to get better. I disagree. It needs both, for sure, but I think it needs more. Also, it won't get any more data! As LLM's are used more and more, the data fed in will be more and more like what they would generate anyway, which will lead to the models overfitting on things an…
>>He claims they have reasoning capabilities.
I think here you touch on the crux of LLM conversation. In my limited experience with GPT, it does appear to have some basic reasoning ability, but that could be that it's really good at regurgitating its trained dataset and it just appears that it's reasoning. I think over time we'll be able to sort this question out.