291–300 of 324 posts
Earlier quoted context omitted.
This shows that prediction algorithms (AI models) are also very good at compression, but compression algorithms (like the ones used in gzip) are not likewise very good at prediction. Which is evidence that compression is necessary but not sufficient for prediction.
LLMs are both the best compression and prediction algorithm for English text.
This is the thesis behind the "Information Theory, Inference, and Learning Algorithms" course that was taught at Cambridge University. > Why unify information theory and machine learning? Because they are two sides of the same coin. In the 1960s, a single field, cybernetics, was populated by information theorists, computer scientists, and neuroscientists, all studying common problems. Information theory and machine l…
[1]: https://ashby.info
Earlier quoted context omitted.
What specific historical citations would you like to see in this post? Perhaps you could contact the author with your suggestions?
I'm speaking in general terms and I have not studied much on this subject so I don't have specific suggestions for this post. Please refer to earlier posts in this comment chain for a general idea of what citations would be useful.
Why do you disagree with them?
Earlier quoted context omitted.
Nice one. Yes. Compression constitutes necessary conditions for understanding. But don't forget recomposition as well. Add recursion to the mix and it results in recursive compression and recomposition - making understanding itself a self existing entity. This is as close to an understanding god you can rigorously get to.
This feels like an AI comment, but I feel compelled to respond. Recursion is not needed when you regress system components to its foundamental representations, which can be derived without any recursion involved. In the thread example, I don't need to recursively determine how to compress how a planet orbits a mass. I only need to 'jump to the end' by define the rules of gravity and the mass/velocity of the bodies. W…
Apparently you misunderstood. What made you think I claimed recursion as the key to better intelligence?
Clearly you will have noticed that I agree with compression and merely remind one not to forget "recomposition".
But once you add recursion to the "mix", we immediately enter the domain of self existing entities.
Then things get interesting.
If understanding == compression + recomposition, then recursion(compression + recomposition) yields what can be thought of as a self existing "understanding" entity.
A sort of philosophical "understanding god" of sorts.
Earlier quoted context omitted.
My theory is that anyone wishing to elucidate a complex technical concept is better served with contextual details. I think one method is superior to another. Anybody, anything, obligated, and any related information are hyperbolic terms I would have avoided. I understand with a weak position people often resort to hyperbole. I believe adding more terms to be googled is superior to fewer search terms. When working wi…
If you were trying to describe a preference, you didn't do a very good job. You stated it as a universal. But taking you at your word, if there are things that don't meet your personal tastes, maybe move on to the next thing to read? Not everything has to be for everybody. There are plenty of audiences where people are capable of digging deeper when they want it. Personally, I prefer writers who don't spell everythin…
Earlier quoted context omitted.
I'm speaking in general terms and I have not studied much on this subject so I don't have specific suggestions for this post. Please refer to earlier posts in this comment chain for a general idea of what citations would be useful.
What are your thoughts on the other comments in this chain that claim that the references and citations as they are in the linked post are acceptable? Why do you disagree with them?
> What are your thoughts on the other comments in this chain that claim that the references and citations as they are in the linked post are acceptable?
> Why do you disagree with them?
Which comments in particular are you talking about?
And I mean, the post doesn't have "references and citations" plural that I can see. It has one clear reference. I see two comments defending that reference, and a whole bunch of comments saying that a post like this doesn't need references.
I don't need to disagree with any of those comments. My point isn't about whether a post that lacks citations is "acceptable". I was talking about whether a post could add historical citations without losing momentum, so it could do a better job of fitting into context. A dozen notes sprinkled around about when/where different ideas came from, not just the mention at the very end of what decade arithmetic coding is from and the mention of the 2023 paper.
I will say it's bad that the 2023 paper is described as "arguing that language modeling and compression are two views of the same thing", emphasis in the original. That very much makes it sound like a new idea when it isn't.
Earlier quoted context omitted.
What are your thoughts on the other comments in this chain that claim that the references and citations as they are in the linked post are acceptable? Why do you disagree with them?
First off: What is your goal here? My comment was one sentence and wasn't specific to this article. Why are you acting like I said things I never said? But okay I'll try to answer. > What are your thoughts on the other comments in this chain that claim that the references and citations as they are in the linked post are acceptable? > Why do you disagree with them? Which comments in particular are you talking about? A…
I'm confused by the comments that seem to imply that the author is taking credit for ideas that they're writing about, as I didn't get that impression from writing in the article at all.
I'm also confused by the idea that they didn't provide enough citations for concepts in a non-academic source. I feel like in the day and age of Wikipedia it should be sufficient to colourize / bold terms like "Entropy coders" or "Huffman Encoding" to indicate to the reader that these are important terms that they can research for more details if interested.
Overall I guess my position on this is that ~80/285 comments on this post are related to whether not this is a good post which I think is far too many. There are slight issues with the post but they aren't worth 80 comments.
Earlier quoted context omitted.
First off: What is your goal here? My comment was one sentence and wasn't specific to this article. Why are you acting like I said things I never said? But okay I'll try to answer. > What are your thoughts on the other comments in this chain that claim that the references and citations as they are in the linked post are acceptable? > Why do you disagree with them? Which comments in particular are you talking about? A…
I don't mean to pick on you, it's just then just saw a bunch of negative comments on an article I enjoyed reading and found informative, so I picked a few to reply to for clarification of their position. I'm confused by the comments that seem to imply that the author is taking credit for ideas that they're writing about, as I didn't get that impression from writing in the article at all. I'm also confused by the idea…
Specifically the comment I replied to suggested it was a style tradeoff but I don't see why you can't do citations in either style. That's all I came here for.
> I'm also confused by the idea that they didn't provide enough citations for concepts in a non-academic source.
It's useful information about how we got here. That's good in any long story about an idea.
> I feel like in the day and age of Wikipedia it should be sufficient to colourize / bold terms like "Entropy coders" or "Huffman Encoding" to indicate to the reader that these are important terms that they can research for more details if interested.
It works to some extent but some of the concepts don't have fancy words for them and then they don't get that searchability.
> ~80/285 comments
Sometimes tangents spark a discussion. I don't think those numbers are an issue. And they're mostly not calling it a bad post, they just have an objection.