Attention Is Bayesian Inference
medium.com
Attention Is Bayesian Inference
1–10 of 33 posts
Re: Attention Is Bayesian Inference
#2Pretty interesting. The posterior matching is a big deal, but I'm not convinced by the handwaiving required to demonstrate it in larger models. I'm interested in seeing how direct EM training scales though.
Re: Attention Is Bayesian Inference
#3Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Re: Attention Is Bayesian Inference
#4Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Yeah. The article was clearly "enhanced" with an LLM. Too many inane "this is not just A; this is B" sentences. Also, "why this matters" as final subheading. Fail.
Re: Attention Is Bayesian Inference
#5Last time I look into SoTA Bayesian deep learning, Bayesian output layers seems the most promising and practical. Is that still the case?
Re: Attention Is Bayesian Inference
#6Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Yeah. The article was clearly "enhanced" with an LLM. Too many inane "this is not just A; this is B" sentences. Also, "why this matters" as final subheading. Fail.
this was also my experience and unfortunately, if there were any grains of value to be winnowed from the slop, I lacked the patience to continue grinding at the mill.
Re: Attention Is Bayesian Inference
#7Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Y'all, we need to get away from calling everything written by an LLM "slop". To me, slop is text for the purpose of padding content or getting clicks or whatever. Whether or not this was written in full or in part or 100% by a human who sounds like an LLM, the content here was interesting to think about and was organized and easy to read. Maybe I'm the only person reading past the word choice and grammar to extract the ideas from the article instead of playing a game of "human or AI" with every piece of writing I see.
Re: Attention Is Bayesian Inference
#8Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Just give me an arXiv paper and I'll summarize it myself!
Re: Attention Is Bayesian Inference
#9sure, but this stuff is only obvious post hoc. so many people have tried to "justify" the attention mechanism according to their area of expertise, but none of them came up with it first; ML engineers with ML thinking did.
Re: Attention Is Bayesian Inference
#10Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.
Just give me an arXiv paper and I'll summarize it myself!
The three arxiv links being summarized are included in the article.