I think that a lot of hallucination might just be due to not planning ahead - basically a case of running mouth before engaging brain, and then being in a situation where one has uttered a bunch of nonsense - basically backed oneself into a conversational corner. A human might catch themselves with "err, never mind, forget that!", but the LLM's only recourse is to continue extrapolating the nonsense the only way it knows how - i.e. to bullshit/hallucinate the most statistically plausible continuation of the hole it dug for itself.
As a simplistic made up example, say the training set included a bunch of statements about capital cities of various countries of the form "the capital of england is london", "the capital of france is paris", etc, but didn't include any data indicating the capital of australia... Now, if asked "what is the capital of australia?", it may confidently start "the capital of australia is ..." since this matches the pattern it learnt from the training set (with "australia" being copied from the context). However, when generating the next word the LLM (without realizing it) finds itself in the unfortunate situation of not having been trained on data that would let it predict the right answer, but it of course goes ahead and predicts as best it can anyway (i.e. hallucinates/bullshits) and maybe generates the name of some random important city in australia such as "sydney".
The way to fix at least this cause (maybe the primary one) of hallucination is essentially to plan ahead .. don't start off saying "the capital of australia is .." without knowing where you are going with it! One approach might be additional training using tree of thought rollouts (generate multiple possible branching continuations for each training prompt) then evaluate them and use these as RL rewards to learn to predict words leading to good outcomes (e.g. "i don't know", rather than "the capital of australia is ..").