Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Challenges in LLM Research

huyenchip.com

Post reply on HN
Print view View original

71–74 of 74 posts

Re: Open Challenges in LLM Research

#71

Post by optimalsolver »

emmender1 wrote: post #66
thrwayaistartup wrote: post #2

Looking back in 25 years, the "Hallucination Problem" will sound a lot like the "Frame Problem" of the 1970s. Looking back, it's a bit absurd to say that GOFAI would've got to AGI if only the Frame Problem could be solved. But the important point is why that sounds so absurd. It doesn't sound absurd because we found out that the frame problem can't be solved; that's beside the point. It also doesn't sound absurd beca…

"... .Looking back from the other side of the next Winter, the whole thing will seem a bit goofy." for most of us, what we wish for is what we believe.

Is from ought.

Re: Open Challenges in LLM Research

#72

Post by yeck »

nonameiguess wrote: post #63
yeck wrote: post #55

I have a hard time understanding why mechanistic interpretability has so few eyes on it. It's like trying to build a complex software system without logging or monitoring. Any other improvements you want to make on the system are going to just be trail and error with luck. The hallucination problem is one where interpretability of a model might be able to identify the failure modes that we need to address. Really any…

I'm going to talk out of my ass here because I am not involved enough to know the mechanics of how LLMs are really trained at any deep level, but from the surface level understanding I have, I would expect any attempt to eliminate hallucination to be intractable given the techniques in use. As far as I understand, the initial training run is simply fed raw text and it works on the basis of predicting a next token. Th…

If you could know that this is the case with interpretability tools than we would be able to train new models with purposeful decisions to reduce or remove hallucinations. Narrow the range of the tests and experiments you need to do to solve the problem. Otherwise we are mostly speculating about why stuff doesn't work and play a game of darts in the dark.

Re: Open Challenges in LLM Research

#73

Post by yeck »

jebarker wrote: post #61
yeck wrote: post #55

I have a hard time understanding why mechanistic interpretability has so few eyes on it. It's like trying to build a complex software system without logging or monitoring. Any other improvements you want to make on the system are going to just be trail and error with luck. The hallucination problem is one where interpretability of a model might be able to identify the failure modes that we need to address. Really any…

When I looked into this briefly my impression was that it's extremely hard to do mechanistic interpretation beyond very simple cases like CNN classification or toy problems like arithmetic in transformers. Not to say it's not a worthy pursuit, but I think the difficulty isn't justified for many researchers since the results won't make a big splash like a new model training result.

Yeah, it is harder than other things, but if we can train a model to explain collections of pixels in human language then we might be able to do similar with collections of activations.

I don't know if that is the direction, but just an example that comes to mind easily.

If someone figures out how to do this, I think their models will be far more capable and reliable.

Post reply on HN