Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Challenges in LLM Research

huyenchip.com

Post reply on HN
Print view View original

31–40 of 74 posts

Re: Open Challenges in LLM Research

#31

Post by PeterStuer »

fallous wrote: post #10
resonious wrote: post #9

Earlier quoted context omitted.

Humans also very often produce results that don't make sense.

Humans that produce output like LLMs are most likely to be diagnosed as schizophrenic, which I don't believe is the goal.

Confident human bullshitters seem to thrive in business environments, in media and entertainment, in politics ... in fact in any profession where the production is just language instead of doing things. They might be more on the dark triangle spectrum, but I would not call them all "schizophrenic".

The problem is we are so used to yielding to confidence we don't apply the necessary checks even when we know it is projected by machine.

Re: Open Challenges in LLM Research

#32

Post by js8 »

thrwayaistartup wrote: post #2

Looking back in 25 years, the "Hallucination Problem" will sound a lot like the "Frame Problem" of the 1970s. Looking back, it's a bit absurd to say that GOFAI would've got to AGI if only the Frame Problem could be solved. But the important point is why that sounds so absurd. It doesn't sound absurd because we found out that the frame problem can't be solved; that's beside the point. It also doesn't sound absurd beca…

My feeling is that GOFAI had a real problem with representing uncertainty, and handling contradiction. So, we tried to approach it theoretically, with fuzzy logic and probability and so on. But the theoretical research on uncertainty didn't reach any clear conclusion.

Meanwhile, the neural nets (and ML) researchers just trucked on, with more compute power, and pretty much ignored any theoretical issues with uncertainty. And surprisingly, with lots of amazing results.

But now they hit the same wall, we don't actually understand how to do reasoning with uncertainty correctly. LLMs seem to solve this by "just mimic reasoning that humans do". Except because we lack a good theory of reasoning, it can't tell when mimicking is bad and when it's good, unless there is a lot of specific examples. So in the most egregious cases, we get hallucinations but have no clue how to avoid them.

Re: Open Challenges in LLM Research

#33

Post by wanderlust123 »

errantspark wrote: post #19

Fun fact: I took the photo she used as a cover for one of her books, she asked me if she could use it and I said I'd like to be compensated and her response was something akin to "oh I was just asking assuming you'd say yes, I'm going to do it anyway". Nobody's perfect, maybe she regrets it, and it hasn't really crossed my mind in years, but I guess it still sort of irks me to be reminded of it. Anyway if anyone need…

Sounds like a pretty entitled and unpleasant person. At the bare minimum you should have had a say in whether you picture could hve been used.

Re: Open Challenges in LLM Research

#34

Post by inciampati »

One thing I'd like to see is more effort on developing citation systems for these models.

What I mean is that every part of the output of an LLM should be annotated with references to the content that is most important or relevant to it.

Who is leading this effort now?

Re: Open Challenges in LLM Research

#35

Post by astrange »

inciampati wrote: post #34

One thing I'd like to see is more effort on developing citation systems for these models. What I mean is that every part of the output of an LLM should be annotated with references to the content that is most important or relevant to it. Who is leading this effort now?

It's not possible to do this without a completely different and less efficient architecture. You can approximate it, but it won't give you the correct answers, as there are no correct answers insofar as the model is learning to generalize rather than memorize things.

https://twitter.com/AnthropicAI/status/1688946685937090560

Re: Open Challenges in LLM Research

#36

Post by astrange »

fallous wrote: post #10
resonious wrote: post #9

Earlier quoted context omitted.

Humans also very often produce results that don't make sense.

Humans that produce output like LLMs are most likely to be diagnosed as schizophrenic, which I don't believe is the goal.

Untuned LLMs are most like people with Korsakoff's syndrome. "hallucination" is a misleading term.

Re: Open Challenges in LLM Research

#37

Post by josephg »

pishpash wrote: post #29
Legend2440 wrote: post #25

Earlier quoted context omitted.

That's a hardware difference. Brains run at a very low clock speed and make up for it with massive parallelism. They also don't suffer from the vonn neumann bottleneck - today's computers spend most of their time and energy shuffling the network in and out of memory. I believe that better hardware architectures will have more impact on AI than better neural network architectures.

That would be FPGA's.

I doubt it. FPGAs are super inefficient in transistor count in exchange for being dynamically programmable. I suspect a better architecture will be taped out like any other chip.

Re: Open Challenges in LLM Research

#39

Post by josephg »

visarga wrote: post #24

Let me add a few: - organic data exhaustion - we need to step up synthetic data and its validation - imbalanced datasets - catalog, assess and fill in missing data - backtracking - make LLMs better at combinatorial or search problems - deduction - we need to augment the training set for revealing implicit knowledge, in other words to study the text before learning it - defragmentation - information comes in small chu…

And I’ll add some more:

- LLMs aren’t very good at large scale narrative construction. They get too distracted by low level details that they miss the high level details in long text. It feels like the same problem as stable diffusion giving people too many fingers.

- LLMs have 2 kinds of memory: current activations (context) and trained weights. This is like working memory and long term memory. How do we add short term memory? Like, if I read a function, I summarize it in my head and then remember the summary for as long as it’s relevant. (Maybe 20 minutes or something). How do we build a mechanism that can do this?

- How do we do gradient descent on the model architecture itself during training?

- Humans have lots more tricks to use when reading large, complex text - like re-reading relevant sections, making notes, thinking quietly, and so on. Can we introduce these thinking modalities into our systems? I bet they’d behave smarter if they could do this stuff.

- How do we combine multiple LLMs into a smarter overall system? Eg, does it make sense to build committees of “experts” (LLMs taking on different expert roles) to help in decision making? Can we get more intelligence out of chatgpt by using it in a different way in a larger system?

Re: Open Challenges in LLM Research

#40

Post by uoaei »

visarga wrote: post #23
illusionist123 wrote: post #15

I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require…

Technically, transformers condition on the entire past not just the last step, but RNNs are Markov Chains. RNNs have information bottleneck issues though.

Transformers (for NLP) also perform steps on Markov chains. The difference is that with transformers (for NLP), which Markov chain it's moving along changes every step.
Post reply on HN