91–100 of 188 posts
Simplified differential T. looks like: (softmax(Q₁K₁) − λ softmax(Q₂K₂)) V
You can factor this into:
x = softmax(Q₁K₁)V
x += -λ softmax(Q₂K₂)V
I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between
> Surely there's a trade-off... For one, speed and memory. They have twice as many Q and K weights in the attention blocks, leading to a ~10% reduction in throughput on their H100 (table 7 in appendix A).
Earlier quoted context omitted.
> Hallucination and creativity feel very related to me. Why? I see them as just sampling errors. Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes. > I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between These language models are next-token predictors. The way the next token is predicted is by sampling a probability…
> Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes. It looks like creativity has many steps but being able to come with novel, unprompted stuff is important, as long as you are able to discard the bullshit earlier. "Hallucination" is only a problem if later layers (or additional networks) can't detect and remove it
Yeah I mean sure. Anything is only a problem if it goes undetected. The issue is that if you rely on statistical model, you’ll always have hallucinations, so you can’t filter statistical output with another statistical model if you need real guarantees.
Many products don’t need those guarantees though.
Earlier quoted context omitted.
> it would probably become obsolete soon Suppose there are many times more posts about something one generation of LLMs can't do (arithmetic, tic-tac-toe, whatever), than posts about how the next generation of models can do that task successfully. I think this is probably the case. While I doubt it will happen, it would be somewhat funny if training on that text caused a future model to claim it can't do something th…
also presumes that the LLM knows it is an LLM
Maybe in the future, those prompts will include motivational phrases, like "You can do it!" or "Believe in yourself, then you can achieve anything."
I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between
> Surely there's a trade-off... For one, speed and memory. They have twice as many Q and K weights in the attention blocks, leading to a ~10% reduction in throughput on their H100 (table 7 in appendix A).
Earlier quoted context omitted.
> it would probably become obsolete soon Suppose there are many times more posts about something one generation of LLMs can't do (arithmetic, tic-tac-toe, whatever), than posts about how the next generation of models can do that task successfully. I think this is probably the case. While I doubt it will happen, it would be somewhat funny if training on that text caused a future model to claim it can't do something th…
also presumes that the LLM knows it is an LLM
Earlier quoted context omitted.
I went to school to learn about the world and the overwhelming majority of that learning was from professors and textbooks. Whether the professors' beliefs and the textbooks' contents reflected the true properties of the world was a completely separate thing, entirely outside of my control. But I did come away with a better understanding of the world and few would say that education is orthogonal to that goal. If you…
> But I did come away with a better understanding of the world and few would say that education is orthogonal to that goal. That's due to the reward function / environment. But even outside extremes like North Korea, lots of education environments value conformity over independent analysis.
Earlier quoted context omitted.
> predict a weight of exactly zero for some of the values Wouldn’t this be pretty unlikely, though?
Quite the opposite — if you have a long sequence only a smattering of the words will influence the meaning of the current word. Everything else is “noise”. Attention is really good at finding this smattering of words (ie assign most weight there). But it struggles to put exactly 0 on the other words.
Hmmm, this could be expressed as 2 consecutive attentions in a residual branch: Simplified differential T. looks like: (softmax(Q₁K₁) − λ softmax(Q₂K₂)) V You can factor this into: x = softmax(Q₁K₁)V x += -λ softmax(Q₂K₂)V which is like 2 subsequent regular attentions added that are sharing V