I really like seeing articles or papers that describe the current advances and open challenges in a sub-field (such as [0]). They're underappreciated, but good practice or reading for folks wanting to get in the field. They're also worthwhile and humbling to look back at every few years: did we get the challenges right? How well did we understand the problem at the time?
Open Challenges in LLM Research
21–30 of 74 posts
Re: Open Challenges in LLM Research
#22I have not seen many work on explainable AI regarding large language models. I remember many very nice visualizations and visual analysis tools trying to comprehend, what the network „is seeing“ (eg. in the realm of image classification) or doing
Re: Open Challenges in LLM Research
#23I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require…
Technically, transformers condition on the entire past not just the last step, but RNNs are Markov Chains. RNNs have information bottleneck issues though.
Re: Open Challenges in LLM Research
#24Let me add a few:
- organic data exhaustion - we need to step up synthetic data and its validation
- imbalanced datasets - catalog, assess and fill in missing data
- backtracking - make LLMs better at combinatorial or search problems
- deduction - we need to augment the training set for revealing implicit knowledge, in other words to study the text before learning it
- defragmentation - information comes in small chunks, sits in separate siloes, and context size is short, we need to use retrieval to bring it together for analysis
tl;dr We need quantity, diversity and depth in our training sets
Re: Open Challenges in LLM Research
#25Earlier quoted context omitted.
There are some unsolved practical problems like prompt injection, the difficulty of using them on your own data, etc. But the biggest problem is that they take so much compute, which slows down both research and deployment. Only a handful of giant companies can train their own LLM, and it's a major undertaking even for them. Academic researchers and everyday tinkerers can only run inference on pretrained models.
Sounds like a great motivation for academic researchers to find a way to train LLMs with less compute. Or maybe invent something better than transformers. A brain trains on 20 Watts after all.
That's a hardware difference. Brains run at a very low clock speed and make up for it with massive parallelism. They also don't suffer from the vonn neumann bottleneck - today's computers spend most of their time and energy shuffling the network in and out of memory.
I believe that better hardware architectures will have more impact on AI than better neural network architectures.
Re: Open Challenges in LLM Research
#26Fun fact: I took the photo she used as a cover for one of her books, she asked me if she could use it and I said I'd like to be compensated and her response was something akin to "oh I was just asking assuming you'd say yes, I'm going to do it anyway". Nobody's perfect, maybe she regrets it, and it hasn't really crossed my mind in years, but I guess it still sort of irks me to be reminded of it. Anyway if anyone need…
Just another random anecdotal experience with Chip.
I was interviewing with Claypot.ai and when I met her for my first conversation, she was on a walking treadmill and very clearly was more interested in a Slack conversation she was having.
She moved me on to the next round which I irrefutably bombed and was respectfully told that I wouldn't be moving on which was the right decision, but I'll never forget watching her walking motion while looking at Slack on her second monitor almost the entire time we were talking.
Re: Open Challenges in LLM Research
#27Earlier quoted context omitted.
Does anyone think we would have AGI if only we could solve the hallucination problem?
There’s people who thought we could just wire up ChatGPT to a bunch of API calls and have AGI by now. Or some similar version of bootstrapping an LLM.
That could very well be the case.
Re: Open Challenges in LLM Research
#28I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require…
LLMs already know how to distinguish fact from fiction much better than random chance and the base non-RLHF GPT-4 model was excellently calibrated (its predicted confidence in an answer generally matches the probability of being correct). "Eliminating" it is not that important. Getting it to human levels is the goal. and boy do humans often "hallucinate", i.e have a poor grasp of what they do or do not know and confi…
It doesn't matter what humans do or do not do. Human performance as a benchmark is not a useful metric for what machines should or should not do.
Re: Open Challenges in LLM Research
#29Earlier quoted context omitted.
Sounds like a great motivation for academic researchers to find a way to train LLMs with less compute. Or maybe invent something better than transformers. A brain trains on 20 Watts after all.
That's a hardware difference. Brains run at a very low clock speed and make up for it with massive parallelism. They also don't suffer from the vonn neumann bottleneck - today's computers spend most of their time and energy shuffling the network in and out of memory. I believe that better hardware architectures will have more impact on AI than better neural network architectures.
That would be FPGA's.
Re: Open Challenges in LLM Research
#30Earlier quoted context omitted.
LLMs already know how to distinguish fact from fiction much better than random chance and the base non-RLHF GPT-4 model was excellently calibrated (its predicted confidence in an answer generally matches the probability of being correct). "Eliminating" it is not that important. Getting it to human levels is the goal. and boy do humans often "hallucinate", i.e have a poor grasp of what they do or do not know and confi…
It doesn't matter what humans do or do not do. Human performance as a benchmark is not a useful metric for what machines should or should not do.
“Fake it till you make it”