So softmax is e^x projection followed by l1 norm. Why is e^x projection useful?
Softmax, can you derive the Jacobian? And should you care?
21–30 of 50 posts
Re: Softmax, can you derive the Jacobian? And should you care?
#22> The relative differences between values get exaggerated, which means the largest logit value dominates the output, while smaller values are squashed. This is exactly what we want for confident predictions, but it also explains why softmax can be problematic when you want uncertainty estimates Actually I believe that most of the times even after softmax, sampling is ways too permissive, seldom accepting low quality…
I would love to see more work on beam search/Viterbi decodes rather than just greedy next token output.
Re: Softmax, can you derive the Jacobian? And should you care?
#23Nice article and explanations! On a tangential note, I keep noticing "why x matters", "it's crucial here" that just remind me of Claude. Recently Claude has been gaslighting me in complex problems with such statements and seeing them on an article is low-key infuriating at this point. I can't trust Claude anymore on the most complex problems where it sometimes gets the answer right but completely misses the point and…
I've seen many posts on Reddit in this AI-induced 'psychosis' when people end up believing the words that get generated for them without applying sufficient critical thought. This sycophancy is a serious problem and exploits a weakness in the human psyche (flattery) which may be easier for the RLHF to find reward in than genuinely correct responses.
[deleted]
Re: Softmax, can you derive the Jacobian? And should you care?
#24Earlier quoted context omitted.
Sounds like they're saying that since the distribution doesn't come from measuring or calculating the probability of something , it has the form of a probability distribution but isn't really one. Like saying 5 feet is a height that a person can have, but since I just made up that number it's not actually a person's height.
The soft max is the probability of the next token being whatever in the training data conditioned on the inputs. The author just doesn't know that apparently and thinks it was an arbitrary choice. The author's essay on the sigmoid similarly lacks the deep understanding that it comes from somewhere and isn't an arbitrary choice.
The softmax, after the network has been trained, yields an estimate of the probability in the training data, but it is not that probability itself.
Re: Softmax, can you derive the Jacobian? And should you care?
#25> The relative differences between values get exaggerated, which means the largest logit value dominates the output, while smaller values are squashed. This is exactly what we want for confident predictions, but it also explains why softmax can be problematic when you want uncertainty estimates Actually I believe that most of the times even after softmax, sampling is ways too permissive, seldom accepting low quality…
> most of the times, this is not what the model wanted to say, but sampling that casually selected a low quality token.
How do you identify what the model wanted to say?
Re: Softmax, can you derive the Jacobian? And should you care?
#26Something that really helped me grasp the foundational relevance of the softmax is to justify from first principles why e^x shows up in the preferred mapping function in the numerator (1). The stated problem of mapping raw inputs/scores/logits to a probability distribution can be solved by a bunch of arbitrary functions, and the usual justification given for a softmax is "it has nice derivatives" which is empirically…
> The stated problem of mapping raw inputs/scores/logits to a probability distribution can be solved by a bunch of arbitrary functions, and the usual justification given for a softmax is "it has nice derivatives" which is empirically useful but not satisfying.
Often there isn't any more to it than that. For example, the entire justification for least-squares error measurement is that it has convenient derivatives.
Re: Softmax, can you derive the Jacobian? And should you care?
#27> The relative differences between values get exaggerated, which means the largest logit value dominates the output, while smaller values are squashed. This is exactly what we want for confident predictions, but it also explains why softmax can be problematic when you want uncertainty estimates Actually I believe that most of the times even after softmax, sampling is ways too permissive, seldom accepting low quality…
I would love to see more work on beam search/Viterbi decodes rather than just greedy next token output.
Regret analysis in bandit and similar algorithms shows how inference is connected to loss function. If your loss function is good, greedy inference is as good as joint inference.
Training on cost-to-go loss is good enough. Perfect cost-to-go eliminates the need for global algorithms and allows local decision making. Given “natural” datasets it is probably the best thing to attempt to learn. The fact that probabilistic graphical models never really worked proves it somewhat.
Re: Softmax, can you derive the Jacobian? And should you care?
#28> The relative differences between values get exaggerated, which means the largest logit value dominates the output, while smaller values are squashed. This is exactly what we want for confident predictions, but it also explains why softmax can be problematic when you want uncertainty estimates Actually I believe that most of the times even after softmax, sampling is ways too permissive, seldom accepting low quality…
Yeah, softmax may have useful applications, but anytime you find yourself using the same hammer for everything that looks like a nail it's a bit of a red flag.
If you take instances of softmax that you find in training / inference and there turn out to be a few, and use other things like entmax or sparsemax you see across the board improvements. And like top1 often is just the best answer too, there's a reason why when you're doing tool calls temp=0 is the way to go. Like do you really want creative unicode tokens when writing bash commands. From what I can tell, most of the time softmax is the worst answer that works.
Re: Softmax, can you derive the Jacobian? And should you care?
#29So softmax is e^x projection followed by l1 norm. Why is e^x projection useful?
It maps (-inf, inf) to (0, inf) in about as nice a way as you could expect (addition turns into multiplication). When you want to constrain a value to be positive, parameterizing it with exp is usually a good option.
Re: Softmax, can you derive the Jacobian? And should you care?
#30Earlier quoted context omitted.
The soft max is the probability of the next token being whatever in the training data conditioned on the inputs. The author just doesn't know that apparently and thinks it was an arbitrary choice. The author's essay on the sigmoid similarly lacks the deep understanding that it comes from somewhere and isn't an arbitrary choice.
The softmax, after the network has been trained, yields an estimate of the probability in the training data, but it is not that probability itself.
Which models are not trained with the log softmax as the loss function?