Funny timing, I just used softmax yesterday to turn a list of numbers (some of which could be negative) into a probability distribution (summing up to 1). So useful. It was the perfect tool for the job.
Softmax forever, or why I like softmax
51–60 of 116 posts
Re: Softmax forever, or why I like softmax
#52Earlier quoted context omitted.
It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years. Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. He's adopting the native culture rather than setting a trend. Sam still uses capitalization in all of his essays, as do most people (including young people). In essays, like this…
>Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use.
Just looked at the algorithmic feed on Twitter to makes sure trends haven't shifted overnight, and zero people in that sample of hundreds of tweets used all lower case in the tweets. Not in science. Not in AI. Not in maths or politics or entertainment or media.
Sam is trying to bE dIFFERENT. He isn't adopting a norm but instead he's trying to make one. It looks ridiculous.
Re: Softmax forever, or why I like softmax
#53Earlier quoted context omitted.
It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years. Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. He's adopting the native culture rather than setting a trend. Sam still uses capitalization in all of his essays, as do most people (including young people). In essays, like this…
Look at AOL Instant Messenger in 1999 and this is how everyone in school wrote.
Guess what… the people who used AIM in 1999 are now middle aged…
Re: Softmax forever, or why I like softmax
#54Off topic: Unlike many out there I'm not usually bothered by lack of capitalization in comments or tweets, but for an essay like this, it makes the paragraphs so hard to read!
If someone can't even put in a minimal amount of effort for basic punctuation and grammar, I'm not going to read their article on something more sophisticated. If you go for the lowercase i's because you want a childish or slob aesthetic, that can be funny in context. But in math or computing, I'm not going to care what someone thinks if they don't know or don't care that 'I' should be capitalized. Grammarly has a fr…
We just had a similar discussion at work the other day when one of our junior engineers noticed that a senior engineer was reflexively tapping the space bar twice after each sentence. That, too, was good style back when we were writing on typewriters or using monospace fonts with no typesetting. Only a child or a slob would fail to provide an extra gap between sentences, it would be distracting to readers and difficult to locate full stops without that!
But it's 2025, and HTML and Word and the APA and MLA and basically everyone agree that times and style guides have changed.
I agree that not capitalizing the first letter in a sentence is a step too far.
For a counter-example, I personally don't care whether they use the proper em-dash, en-dash, or hyphen--I don't even know when or how to insert the right one with my keyboard. I'm sure there are enthusiasts who care very deeply about using the right ones, and feel that my lack of concern for using the right dash is lazy and unrefined. Culture is changing as more and more communication happens on phone touchscreens, and I have to ask myself - am I out of touch? No, it's the children who are wrong. /s
But I strongly disagree that the author should pass everything they write through Grammarly or worse, through ChatGPT.
Re: Softmax forever, or why I like softmax
#55An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
an opinion, and a falsifiable hypothesis:
call me old-fasahioned, but two spaces after a period will solve this problem if people insist on all-lower-case. this also helps distinguish between abbreviations such as st. martin's and the ends of sentences.
i'll bet that the linguistics experimentalists have metrics that quantify reading speed measurements as determined by eye tracking experiments, and can verify this.
Re: Softmax forever, or why I like softmax
#56Earlier quoted context omitted.
Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years. Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. He's adopting the native culture rather than setting a trend. Sam still uses capitalization in all of his essays, as do most people (including young people). In essays, like this…
Look at AOL Instant Messenger in 1999 and this is how everyone in school wrote. Guess what… the people who used AIM in 1999 are now middle aged…
the article has lots of caps it’s a branding and stylistic choice very e e cummings and so on not everyone likes it but clearly the author sees some utility in using caps since they name papers appropriately they also say “UPDATE” in all caps
Re: Softmax forever, or why I like softmax
#57An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
an opinion, and a falsifiable hypothesis: call me old-fasahioned, but two spaces after a period will solve this problem if people insist on all-lower-case. this also helps distinguish between abbreviations such as st. martin's and the ends of sentences. i'll bet that the linguistics experimentalists have metrics that quantify reading speed measurements as determined by eye tracking experiments, and can verify this.
> [I]'ll bet that the linguistics experimentalists have metrics that quantify reading speed measurements as determined by eye tracking experiments, and can verify this.
You appear to be trolling for the sake of trolling, but for reference: reading speed is determined by familiarity with the style of the text. Diverging from whatever people are used to will make them slower.
There is no such thing as "two spaces" in HTML, so good luck with that.
Re: Softmax forever, or why I like softmax
#58Earlier quoted context omitted.
Language evolves. Capitalization is an artifact of a period where capitalizing the first letter made a lot of sense for the medium (parchment/paper). Modern culture is abandoning it for speed efficiency on keyboards or digital keyboards. A purist would say that we should still be using all capitals like they did in Greek/Latin which again was related to the medium. I'll likely continue using Capitalization as a prefe…
It's slower for sure, but capitalization does impart information: beginning of sentences, proper nouns, acronyms, and such. Sure, you could re-read the sentence until you figured all that out, but you are creating unnecessary hitches in the reading process. Capitalization is an optimization for the reader, and lack of capitalization is optimization for the writer.
Its just a convention. You just get used to it. Whether we like it or not, written language is heading that way. Readers don't generally read letters in words anyway, they read the whole word after a certain literacy level has been achieved.
Re: Softmax forever, or why I like softmax
#59An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
This is the norm for Gen Z. We don’t see it because children don’t set social norms where adults are present too, but with the oldest of Gen Z about to turn 30, you and I should expect to see this more and more, and get used to it. If every kid can handle it, I think we can, too.
Re: Softmax forever, or why I like softmax
#60An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
I agree. I'm all for Graham's pyramid of disagreement: we should focus on the core argument, rather than superfluous things like tone, or character, or capitalisation. But this is too much for me personally. I just realised I consider the complete lack of capitalisation on a piece of public intellectual work to be obnoxious. Sorry, it's impractical, distracting and generates unnecessary cognitive load for everyone el…
Graham's Hierarchy, in "How to Disagree": https://paulgraham.com/disagree.html