Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Softmax forever, or why I like softmax

kyunghyuncho.me

Post reply on HN
Print view View original

41–50 of 116 posts

Re: Softmax forever, or why I like softmax

#41

Post by yapyap »

jiggawatts wrote: post #31
1dom wrote: post #29

Earlier quoted context omitted.

I agree. I'm all for Graham's pyramid of disagreement: we should focus on the core argument, rather than superfluous things like tone, or character, or capitalisation. But this is too much for me personally. I just realised I consider the complete lack of capitalisation on a piece of public intellectual work to be obnoxious. Sorry, it's impractical, distracting and generates unnecessary cognitive load for everyone el…

It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.

[dead]

Re: Softmax forever, or why I like softmax

#42

Post by saghm »

alchemist1e9 wrote: post #38
jiggawatts wrote: post #31

Earlier quoted context omitted.

It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.

> It's a fad associated with AI, popularised by Sam Altman especially. I know this is true but does anyone understand why they do it? It is actually cognitively disruptive when reading content because many of us are trained to simultaneously proof read while reading. So I also consider it a type of cognitive attack vector and it annoys me extremely as well.

The sibling comment to yours mentions that this is pretty common on Twitter, and I'd guess that it started as a way to making firing off tweets from a phone easier (since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard). Once enough people were doing it there, the style probably became recognizable and evoked a certain "vibe" that people wanted to replicate elsewhere, including in places where the original context of "hitting the shift key is more work than it's worth" doesn't hold as well.

Re: Softmax forever, or why I like softmax

#43

Post by mitthrowaway2 »

LudwigNagasena wrote: post #33
stared wrote: post #21

There are many useful tricks - like cosine distance. In contrast, softmax has a very deep grounding in statistical physics - where it is called the Boltzmann distribution. In fact, this connection between statistical physics and machine learning was so fundamental that it was a key part of the 2024 Nobel Prize in Physics awarded to Hopfield and Hinton.

Study of thermodynamics gave rise to many concepts in information theory and statistics, but I wouldn't say that there is any direct connection per se between thermodynamics and any field where statistics or information theory are applicable. And the reasoning behind the 2024 Nobel Prize in Physics was... quite innovative.

> I wouldn't say that there is any direct connection per se between thermodynamics and any field where statistics or information theory are applicable.

Thermodynamics can absolutely be studied through both a statistical mechanics and an information theory lens, and many physicists have found this to be quite productive and enlightening. Especially when it gets to tricky cases involving entropy, like Maxwell's Demon and Landauer's Eraser, one struggles not to do so.

Re: Softmax forever, or why I like softmax

#44

Post by [deleted] »

alchemist1e9 wrote: post #38
jiggawatts wrote: post #31

Earlier quoted context omitted.

It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.

> It's a fad associated with AI, popularised by Sam Altman especially. I know this is true but does anyone understand why they do it? It is actually cognitively disruptive when reading content because many of us are trained to simultaneously proof read while reading. So I also consider it a type of cognitive attack vector and it annoys me extremely as well.

[deleted]

Re: Softmax forever, or why I like softmax

#45

Post by 4ggr0 »

jiggawatts wrote: post #31
1dom wrote: post #29

Earlier quoted context omitted.

I agree. I'm all for Graham's pyramid of disagreement: we should focus on the core argument, rather than superfluous things like tone, or character, or capitalisation. But this is too much for me personally. I just realised I consider the complete lack of capitalisation on a piece of public intellectual work to be obnoxious. Sorry, it's impractical, distracting and generates unnecessary cognitive load for everyone el…

It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.

i don't want to press the shift-key everytime i need a capitalized letter on my phone and i disable auto-correct because it constantly messes with native languages etc.

wasn't aware that this makes me a steve jobs copier :(

EDIT: people are seriously so emotionally invested in capitalization that i get downvoted into minus, jeez.

Re: Softmax forever, or why I like softmax

#46

Post by jppope »

roger_ wrote: post #27

An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.

Language evolves. Capitalization is an artifact of a period where capitalizing the first letter made a lot of sense for the medium (parchment/paper). Modern culture is abandoning it for speed efficiency on keyboards or digital keyboards. A purist would say that we should still be using all capitals like they did in Greek/Latin which again was related to the medium.

I'll likely continue using Capitalization as a preference and that we use it to express conventions in programming, but I totally understand the movement to drop it and frankly its logical enough.

Re: Softmax forever, or why I like softmax

#47

Post by bicx »

jppope wrote: post #46
roger_ wrote: post #27

An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.

Language evolves. Capitalization is an artifact of a period where capitalizing the first letter made a lot of sense for the medium (parchment/paper). Modern culture is abandoning it for speed efficiency on keyboards or digital keyboards. A purist would say that we should still be using all capitals like they did in Greek/Latin which again was related to the medium. I'll likely continue using Capitalization as a prefe…

It's slower for sure, but capitalization does impart information: beginning of sentences, proper nouns, acronyms, and such. Sure, you could re-read the sentence until you figured all that out, but you are creating unnecessary hitches in the reading process. Capitalization is an optimization for the reader, and lack of capitalization is optimization for the writer.

Re: Softmax forever, or why I like softmax

#48

Post by bowsamic »

saghm wrote: post #42
alchemist1e9 wrote: post #38

Earlier quoted context omitted.

> It's a fad associated with AI, popularised by Sam Altman especially. I know this is true but does anyone understand why they do it? It is actually cognitively disruptive when reading content because many of us are trained to simultaneously proof read while reading. So I also consider it a type of cognitive attack vector and it annoys me extremely as well.

The sibling comment to yours mentions that this is pretty common on Twitter, and I'd guess that it started as a way to making firing off tweets from a phone easier (since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard). Once enoug…

> since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard

I'm a bit confused about this. Do people turn off auto capitalisation on their phones? I very rarely have to press shift on my phone

Re: Softmax forever, or why I like softmax

#49

Post by bowsamic »

PhilippGille wrote: post #37
roger_ wrote: post #27

An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.

https://www.theguardian.com/society/2025/feb/18/death-of-cap...

Well I will fight this trend to the death. Thankfully I don't like to surround myself with philistines

Re: Softmax forever, or why I like softmax

#50

Post by andai »

alchemist1e9 wrote: post #38
jiggawatts wrote: post #31

Earlier quoted context omitted.

It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.

> It's a fad associated with AI, popularised by Sam Altman especially. I know this is true but does anyone understand why they do it? It is actually cognitively disruptive when reading content because many of us are trained to simultaneously proof read while reading. So I also consider it a type of cognitive attack vector and it annoys me extremely as well.

>So I also consider it a type of cognitive attack vector

What does this mean?

Post reply on HN