Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Challenges in LLM Research

huyenchip.com

Post reply on HN
Print view View original

11–20 of 74 posts

Re: Open Challenges in LLM Research

#11

Post by Legend2440 »

ford wrote: post #5

So far it's been ~8 months since ChatGPT started the (popular) LLM craze. I've found raw GPT to be useful for a lot of things, but have yet to see my most frequently used apps integrate it in a useful way. Maybe I'm using the wrong apps... It'll be interesting to see what improvements (in a lab or at a company) need to happen before most people use purpose-built LLMs (or behind the scenes LLM prompts) in the apps the…

There are some unsolved practical problems like prompt injection, the difficulty of using them on your own data, etc.

But the biggest problem is that they take so much compute, which slows down both research and deployment. Only a handful of giant companies can train their own LLM, and it's a major undertaking even for them. Academic researchers and everyday tinkerers can only run inference on pretrained models.

Re: Open Challenges in LLM Research

#12

Post by Buttons840 »

Our APIs up to this point have been designed for computers. Json input, json output, and those are the nice ones.

I wonder if a deterministic but natural language API would be any better for LLMs to integrate with? Or do LLMs already speak Json well enough?

Re: Open Challenges in LLM Research

#13

Post by JimtheCoder »

p1esk wrote: post #3
thrwayaistartup wrote: post #2

Looking back in 25 years, the "Hallucination Problem" will sound a lot like the "Frame Problem" of the 1970s. Looking back, it's a bit absurd to say that GOFAI would've got to AGI if only the Frame Problem could be solved. But the important point is why that sounds so absurd. It doesn't sound absurd because we found out that the frame problem can't be solved; that's beside the point. It also doesn't sound absurd beca…

Does anyone think we would have AGI if only we could solve the hallucination problem?

We don't even have a generally accepted definition of AGI yet, so...no.

Re: Open Challenges in LLM Research

#15

Post by illusionist123 »

I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require something similar, i.e. a type system or grammar for what should be considered a fact. Another analogy is Prolog and logical resolution to determine consequences from a given database of facts. LLMs do not use logical resolution and they don't have a database of facts to determine whether whatever is generated is actually factual (or logically follows from some set if facts) or not, LLMs are essentially Markov chains and I am certain it is impossible to have Markov chains without hallucinations.

So whoever is working on this problem, good luck because you have you have a lot of work to do to get Markov chains to only output facts and not just correlations of the training data.

Re: Open Challenges in LLM Research

#16

Post by famouswaffles »

illusionist123 wrote: post #15

I think it's not possible to get rid of hallucinations given the structure of LLMs. Getting rid of hallucinations requires knowing how to differentiate fact from fiction. An analogy from programming languages that people might understand is type systems. Well-typed programs are facts and ill-typed programs are fictions (relative to the given typing of the program). To eliminate hallucinations from LLMs would require…

LLMs already know how to distinguish fact from fiction much better than random chance and the base non-RLHF GPT-4 model was excellently calibrated (its predicted confidence in an answer generally matches the probability of being correct). "Eliminating" it is not that important. Getting it to human levels is the goal. and boy do humans often "hallucinate", i.e have a poor grasp of what they do or do not know and confidently spout nonsense.

Re: Open Challenges in LLM Research

#17

Post by famouswaffles »

melenaboija wrote: post #7
resonious wrote: post #4

Earlier quoted context omitted.

I don't know much about AI research but the idea of "measuring" hallucination definitely seems very loaded to me. Humans hallucinate too and I don't think we can measure that. It almost feels like "we need AGI in order to develop AGI".

Comparing human hallucinations with model “hallucinations” does not make sense to me. Model hallucinations seems to me like a fancy way to call the model results that make no sense (ie blatant errors). Plus it makes the model more humanoid.

Most hallucinations make sense. In fact, that is precisely the problem. They make so much sense it's often difficult to distinguish. Most people refer to hallucinations as wrong and often confidently wrong details in a generated reply.

Humans are certainly better but we don't have an absolute sense of what we do or don't know either.

Re: Open Challenges in LLM Research

#18

Post by mlinhares »

fallous wrote: post #10
resonious wrote: post #9

Earlier quoted context omitted.

Humans also very often produce results that don't make sense.

Humans that produce output like LLMs are most likely to be diagnosed as schizophrenic, which I don't believe is the goal.

I really haven’t seen much of that coming from my ChatGPT usage, it’s just someone lying with a lot of confidence, hardly a mental disorder.

Re: Open Challenges in LLM Research

#19

Post by errantspark »

Fun fact: I took the photo she used as a cover for one of her books, she asked me if she could use it and I said I'd like to be compensated and her response was something akin to "oh I was just asking assuming you'd say yes, I'm going to do it anyway". Nobody's perfect, maybe she regrets it, and it hasn't really crossed my mind in years, but I guess it still sort of irks me to be reminded of it. Anyway if anyone needs a portrait for a book cover feel free to hit me up XD.

Re: Open Challenges in LLM Research

#20

Post by p1esk »

Legend2440 wrote: post #11
ford wrote: post #5

So far it's been ~8 months since ChatGPT started the (popular) LLM craze. I've found raw GPT to be useful for a lot of things, but have yet to see my most frequently used apps integrate it in a useful way. Maybe I'm using the wrong apps... It'll be interesting to see what improvements (in a lab or at a company) need to happen before most people use purpose-built LLMs (or behind the scenes LLM prompts) in the apps the…

There are some unsolved practical problems like prompt injection, the difficulty of using them on your own data, etc. But the biggest problem is that they take so much compute, which slows down both research and deployment. Only a handful of giant companies can train their own LLM, and it's a major undertaking even for them. Academic researchers and everyday tinkerers can only run inference on pretrained models.

Sounds like a great motivation for academic researchers to find a way to train LLMs with less compute. Or maybe invent something better than transformers. A brain trains on 20 Watts after all.
Post reply on HN