Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Attention Is Bayesian Inference

medium.com

Post reply on HN
Print view View original

21–30 of 33 posts

Re: Attention Is Bayesian Inference

#21

Post by Analemma_ »

naasking wrote: post #19
cubefox wrote: post #4

Earlier quoted context omitted.

Yeah. The article was clearly "enhanced" with an LLM. Too many inane "this is not just A; this is B" sentences. Also, "why this matters" as final subheading. Fail.

Who cares if it was enhanced with LLMs? That's not determinative of whether the article is accurate and valuable.

This is kind of a self-defeating argument. If the information is accurate and valuable, why bother with this blog post at all? The papers could speak for themselves.

But a lot of people are of the opinion that for many papers it helps to have a secondary publication where the author puts the work in the appropriate context. I’m trying to build a shared mental model with the author, to help me better understand the underlying work; that is harder to do when there’s no mind behind the words.

Re: Attention Is Bayesian Inference

#23

Post by naasking »

Analemma_ wrote: post #21
naasking wrote: post #19

Earlier quoted context omitted.

Who cares if it was enhanced with LLMs? That's not determinative of whether the article is accurate and valuable.

This is kind of a self-defeating argument. If the information is accurate and valuable, why bother with this blog post at all? The papers could speak for themselves. But a lot of people are of the opinion that for many papers it helps to have a secondary publication where the author puts the work in the appropriate context. I’m trying to build a shared mental model with the author, to help me better understand the un…

Because articles are high level summaries of detailed work. What's self defeating about that?

> that is harder to do when there’s no mind behind the words.

Presumably the author read the text before publish and agreed with the summary. What's the problem exactly?

Re: Attention Is Bayesian Inference

#25

Post by layer8 »

RevEng wrote: post #14
danielscrubs wrote: post #3

Found it interesting and engaging, but having a CS professor at Colombia putting their name to AI “slop” is a bit unnerving. If they are writing papers for work you would hope they would enjoy the process of thinking and writing (journaling) instead of using ChatGPT.

Writing the paper is a very small part of the research. It's entirely likely that - like many of their students - they love the research but hate writing papers. They are very different skill sets.

One would think they’d care about the experience of people actually reading their papers.

Re: Attention Is Bayesian Inference

#26

Post by layer8 »

naasking wrote: post #23
Analemma_ wrote: post #21

Earlier quoted context omitted.

This is kind of a self-defeating argument. If the information is accurate and valuable, why bother with this blog post at all? The papers could speak for themselves. But a lot of people are of the opinion that for many papers it helps to have a secondary publication where the author puts the work in the appropriate context. I’m trying to build a shared mental model with the author, to help me better understand the un…

Because articles are high level summaries of detailed work. What's self defeating about that? > that is harder to do when there’s no mind behind the words. Presumably the author read the text before publish and agreed with the summary. What's the problem exactly?

The problem is that it’s distracting, lowers the quality of the writing, and one has to be cautious that random details might be wrong or misleading in a way that wouldn’t happen if it was completely self-authored.

Re: Attention Is Bayesian Inference

#27

Post by naasking »

layer8 wrote: post #26
naasking wrote: post #23

Earlier quoted context omitted.

Because articles are high level summaries of detailed work. What's self defeating about that? > that is harder to do when there’s no mind behind the words. Presumably the author read the text before publish and agreed with the summary. What's the problem exactly?

The problem is that it’s distracting, lowers the quality of the writing, and one has to be cautious that random details might be wrong or misleading in a way that wouldn’t happen if it was completely self-authored.

That's just not true, and even if LLMs did introduce more errors than humans, if you can't trust the author to proof read a summary article about his own papers, then you shouldn't trust the papers either.

Re: Attention Is Bayesian Inference

#28

Post by layer8 »

naasking wrote: post #27
layer8 wrote: post #26

Earlier quoted context omitted.

The problem is that it’s distracting, lowers the quality of the writing, and one has to be cautious that random details might be wrong or misleading in a way that wouldn’t happen if it was completely self-authored.

That's just not true, and even if LLMs did introduce more errors than humans, if you can't trust the author to proof read a summary article about his own papers, then you shouldn't trust the papers either.

I agree with the latter. The fact that they use an LLM for the summary post without rewriting it in their own words already makes me not trust their papers.

Re: Attention Is Bayesian Inference

#29

Post by dbacar »

iamjs wrote: post #24
dbacar wrote: post #22

Just skimming, noticed lots of em dashes, interesting :).

It's so disappointing that this has become a meme. Lot's of people write with em-dashes. If you want to criticize the _writing_, then do so.

Writing is repetitive, making lots of general false claims out of personal feelings etc... I guess this is enough to criticize a chatbot output.

Re: Attention Is Bayesian Inference

#30

Post by eli_gottlieb »

wrsh07 wrote: post #15
maccam912 wrote: post #7

Earlier quoted context omitted.

Y'all, we need to get away from calling everything written by an LLM "slop". To me, slop is text for the purpose of padding content or getting clicks or whatever. Whether or not this was written in full or in part or 100% by a human who sounds like an LLM, the content here was interesting to think about and was organized and easy to read. Maybe I'm the only person reading past the word choice and grammar to extract t…

I would say that many of the sentences in this essay are not worth reading. Most of them are of the form described, eg not x but y Eg > This suggests that the EM structure isn’t just an analogy — it’s the natural grain of the optimization landscape I don't care if someone uses llm. But it shows a lack of care to do it in this blatant way without noting it. Eg at work I'll often link prompt-response in docs as an appe…

> This suggests that the EM structure isn’t just an analogy — it’s the natural grain of the optimization landscape

As someone in the field, this means nothing, and I'm very suspicious of the article as a whole because it has so many sentences like this.

Post reply on HN