An aside: please use proper capitalization. With this article I found myself backtracking thinking I’d missed a word, which was very annoying. Not sure what the authors intention was with that decision but please reconsider.
This is the norm for Gen Z. We don’t see it because children don’t set social norms where adults are present too, but with the oldest of Gen Z about to turn 30, you and I should expect to see this more and more, and get used to it. If every kid can handle it, I think we can, too.
Softmax forever, or why I like softmax
61–70 of 116 posts
Re: Softmax forever, or why I like softmax
#62Earlier quoted context omitted.
That would be "welcome to the world of academia". My post-doc friends won't even read a blog post prior to checking author's resume. They are very dismissive every time they notice anything they consider sloppy etc.
You seem to merge two different points. Not reading based on sloppiness is defensible. Not reading based on the author's resume less so.
Re: Softmax forever, or why I like softmax
#63Earlier quoted context omitted.
an opinion, and a falsifiable hypothesis: call me old-fasahioned, but two spaces after a period will solve this problem if people insist on all-lower-case. this also helps distinguish between abbreviations such as st. martin's and the ends of sentences. i'll bet that the linguistics experimentalists have metrics that quantify reading speed measurements as determined by eye tracking experiments, and can verify this.
> [I]'ll bet that the linguistics experimentalists have metrics that quantify reading speed measurements as determined by eye tracking experiments, and can verify this. You appear to be trolling for the sake of trolling, but for reference: reading speed is determined by familiarity with the style of the text. Diverging from whatever people are used to will make them slower. There is no such thing as "two spaces" in H…
Code point 160 followed by 32. In other words ` ` will do it.
Re: Softmax forever, or why I like softmax
#64Earlier quoted context omitted.
It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
i don't want to press the shift-key everytime i need a capitalized letter on my phone and i disable auto-correct because it constantly messes with native languages etc. wasn't aware that this makes me a steve jobs copier :( EDIT: people are seriously so emotionally invested in capitalization that i get downvoted into minus, jeez.
Re: Softmax forever, or why I like softmax
#65Eg. Using Bayesian stats, if i assume an even prior (pretend i have no assumptions about how biased it is), i see a coin flip heads 4 times in a row, what's the probability of it being heads?
Via a long winded proof using the dirichlet distribution Bayesian stats will say "add one to the top and two to the bottom". Here we saw 4/4 heads. So we guess 5/6 chance of being heads (+1 to the top, +2 to the bottom) the next time or a 1/6 chance of being tails. This represents that the statistical model is assuming some bias in the coin.
That's normalized as a probability against 1 which is what we want. It works for multiple probabilities as well, you add to the bottom as many different outcomes as you have. The Dirichlet distribution allows for real numbers and you can support this too. If you feel this gives too much weight to the possibility of the coin being biased you can actually simply add more to the top and bottom which is the same as accounting for this in your prior, eg. add 100 to the top and 200 to the bottom instead.
Now this has a lot of differences with outcomes compared to softmax. It actually gives everything a non-zero chance rather than using the classic sigmoid activation function that softmax has underneath which moves things to almost absolute 0 or 1. But... other distributions like this are very helpful in many circumstances. Do you actually think the chance of tails becomes 0 if you see heads flipped 100 times in a row? Of course not.
So anyway the softmax function fits things to a particular type of distribution but you can actually fit pretty much anything to any distribution with good old statistics. Choose the right one for your use case.
Re: Softmax forever, or why I like softmax
#66Earlier quoted context omitted.
The sibling comment to yours mentions that this is pretty common on Twitter, and I'd guess that it started as a way to making firing off tweets from a phone easier (since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard). Once enoug…
> since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard I'm a bit confused about this. Do people turn off auto capitalisation on their phones? I very rarely have to press shift on my phone
Re: Softmax forever, or why I like softmax
#67Earlier quoted context omitted.
I agree. I'm all for Graham's pyramid of disagreement: we should focus on the core argument, rather than superfluous things like tone, or character, or capitalisation. But this is too much for me personally. I just realised I consider the complete lack of capitalisation on a piece of public intellectual work to be obnoxious. Sorry, it's impractical, distracting and generates unnecessary cognitive load for everyone el…
It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
Re: Softmax forever, or why I like softmax
#68Earlier quoted context omitted.
Twitter and all forms of instant messaging (SMS, WhatsApp, Discord, and the older ones like AIM/MSN/ICQ) have normalized it for many years. Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. He's adopting the native culture rather than setting a trend. Sam still uses capitalization in all of his essays, as do most people (including young people). In essays, like this…
>Sam is just one of the few large company CEOs to tweet in the style other Twitter users usually use. Just looked at the algorithmic feed on Twitter to makes sure trends haven't shifted overnight, and zero people in that sample of hundreds of tweets used all lower case in the tweets. Not in science. Not in AI. Not in maths or politics or entertainment or media. Sam is trying to bE dIFFERENT. He isn't adopting a norm…
Re: Softmax forever, or why I like softmax
#69Earlier quoted context omitted.
It's a fad associated with AI, popularised by Sam Altman especially. It's the new black turtleneck that everyone is wearing, but will swear upon their mother's life isn't because they're copying Steve Jobs.
i don't want to press the shift-key everytime i need a capitalized letter on my phone and i disable auto-correct because it constantly messes with native languages etc. wasn't aware that this makes me a steve jobs copier :( EDIT: people are seriously so emotionally invested in capitalization that i get downvoted into minus, jeez.
I find it weird that you would be surprised that people care about the quality of textual communication
Re: Softmax forever, or why I like softmax
#70Earlier quoted context omitted.
The sibling comment to yours mentions that this is pretty common on Twitter, and I'd guess that it started as a way to making firing off tweets from a phone easier (since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard). Once enoug…
> since the extra effort to hit shift when typing on a phone keyboard is a bit higher, and the additional effort to go back and fix any typos that happen due to trying to capitalize things is also higher compared to using a traditional keyboard I'm a bit confused about this. Do people turn off auto capitalisation on their phones? I very rarely have to press shift on my phone