Seems like it is basically a blog post review of Challenges and Applications of Large Language Models which was published to arXiv last month:
Open Challenges in LLM Research
41–50 of 74 posts
Re: Open Challenges in LLM Research
#42I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require…
While I'm definitely not going to argue that LLMs are inherently 'thinking' like people do, one thing I do find pretty interesting is that all this talk about hallucinations and bias seems to often conveniently ignore the fact that people are often even more prone to these exact same problems - and as far as I know that's also unlikely to be solved.
ChatGPT is often 'confidently wrong' - I'm pretty sure I've been confidently wrong a few times too, and I've met a lot of other people in my life who've express that trait from time to time too, intentionally or otherwise.
I think there is an inherent trade off between 'confidence', 'expression', and of course 'a-priori bias in the input'. You can learn to be circumspect when you are unsure, and you can learn to better measure your level of expertise on a subject.
But you can't escape that uncertainty entirely. On the other hand, I'm not very convinced about efforts to train LLMs on things like mathematical reasoning. These are situations where you really do have the tools to always produce an exact answer. The goal in these types of problems should focus not on holistically learning how to both identify and solve them, but exclusively on how to identify and define them, and then subsequently pass them off to exact tools suitable for computing the solution.
Re: Open Challenges in LLM Research
#43I really like seeing articles or papers that describe the current advances and open challenges in a sub-field (such as [0]). They're underappreciated, but good practice or reading for folks wanting to get in the field. They're also worthwhile and humbling to look back at every few years: did we get the challenges right? How well did we understand the problem at the time? 0: https://arxiv.org/abs/1912.04977
You might like this one; seems to be the basis for the blog post:
Re: Open Challenges in LLM Research
#44Earlier quoted context omitted.
There’s people who thought we could just wire up ChatGPT to a bunch of API calls and have AGI by now. Or some similar version of bootstrapping an LLM.
That could very well be the case.
No, because there are still other issues like context length and performance. AGI is not very useless if you get one token per hour, and when it forgets about what you talk about 10 minutes ago because it ran out of context
Re: Open Challenges in LLM Research
#45Fun fact: I took the photo she used as a cover for one of her books, she asked me if she could use it and I said I'd like to be compensated and her response was something akin to "oh I was just asking assuming you'd say yes, I'm going to do it anyway". Nobody's perfect, maybe she regrets it, and it hasn't really crossed my mind in years, but I guess it still sort of irks me to be reminded of it. Anyway if anyone need…
Just another random anecdotal experience with Chip. I was interviewing with Claypot.ai and when I met her for my first conversation, she was on a walking treadmill and very clearly was more interested in a Slack conversation she was having. She moved me on to the next round which I irrefutably bombed and was respectfully told that I wouldn't be moving on which was the right decision, but I'll never forget watching he…
everyone has bad days/things they regret. i'm not sure this is a relevant discussion to the content, and personal anecdotes can be very damaging to a person's reputation - i've met her in person and she is delightful, but neither of us are here to judge people so lets stick to the content?
Re: Open Challenges in LLM Research
#46Earlier quoted context omitted.
Sounds like a great motivation for academic researchers to find a way to train LLMs with less compute. Or maybe invent something better than transformers. A brain trains on 20 Watts after all.
That's a hardware difference. Brains run at a very low clock speed and make up for it with massive parallelism. They also don't suffer from the vonn neumann bottleneck - today's computers spend most of their time and energy shuffling the network in and out of memory. I believe that better hardware architectures will have more impact on AI than better neural network architectures.
Neuromorphic Computing is already a thing, and Intel's already developing chips (Loihi 2).
But it's not as powerful as GPU's yet, and it only runs spiking neural networks.
Re: Open Challenges in LLM Research
#47"Never before in my life had I seen so many smart people working on the same goal"
I'm not sure why but the assumptions and naivety in this opening line bothers me. There are plenty of goals and problems that orders of magnitude more people are working on today.
Re: Open Challenges in LLM Research
#48Earlier quoted context omitted.
LLMs already know how to distinguish fact from fiction much better than random chance and the base non-RLHF GPT-4 model was excellently calibrated (its predicted confidence in an answer generally matches the probability of being correct). "Eliminating" it is not that important. Getting it to human levels is the goal. and boy do humans often "hallucinate", i.e have a poor grasp of what they do or do not know and confi…
It doesn't matter what humans do or do not do. Human performance as a benchmark is not a useful metric for what machines should or should not do.
It very much does matter sorry.
Re: Open Challenges in LLM Research
#49Earlier quoted context omitted.
Just another random anecdotal experience with Chip. I was interviewing with Claypot.ai and when I met her for my first conversation, she was on a walking treadmill and very clearly was more interested in a Slack conversation she was having. She moved me on to the next round which I irrefutably bombed and was respectfully told that I wouldn't be moving on which was the right decision, but I'll never forget watching he…
everyone has bad days/things they regret. i'm not sure this is a relevant discussion to the content, and personal anecdotes can be very damaging to a person's reputation - i've met her in person and she is delightful, but neither of us are here to judge people so lets stick to the content?
You are very correct. Thank you for the reminder. I can't tell if I'm just missing it and can't find it but I "authorize" a deletion of my comment as off-topic if there's a way for me to do that.
Re: Open Challenges in LLM Research
#50Looking back in 25 years, the "Hallucination Problem" will sound a lot like the "Frame Problem" of the 1970s. Looking back, it's a bit absurd to say that GOFAI would've got to AGI if only the Frame Problem could be solved. But the important point is why that sounds so absurd. It doesn't sound absurd because we found out that the frame problem can't be solved; that's beside the point. It also doesn't sound absurd beca…
My feeling is that GOFAI had a real problem with representing uncertainty, and handling contradiction. So, we tried to approach it theoretically, with fuzzy logic and probability and so on. But the theoretical research on uncertainty didn't reach any clear conclusion. Meanwhile, the neural nets (and ML) researchers just trucked on, with more compute power, and pretty much ignored any theoretical issues with uncertain…
I think that ascribes way too much meaning to hallucinations, which are the artifact of a big fancy markov chain doing what you'd expect a big fancy markov chain to do.