21–30 of 50 posts
I am not familiar with the standards of publishing in machine learning, but as someone trained in a mathematics background, this paper seems relatively light on details and heavy on exposition. Is that typical? Is this a really novel idea? Not trying to be snarky, just trying to understand how meaningful this is.
Earlier quoted context omitted.
The pace of things is moving along so rapidly right now, I’m not sure that waiting for peer reviews is always a wise move. Doubly so if there’s a paywall; why limit your article’s impact by placing it where practitioners’ agents might not be able to access it? The rapid progress right now is challenging for conventional academic processes. If the value of the paper is difficult to independently verify, for example, i…
// why limit your article's impact // Because...science? It's not science until it passes peer review. I'm not advocating that everybody stops posting to arXiv, and I'm not saying you can't find good stuff there. I'm just saying, it's a vanity press, there is absolutely no guarantee of the paper's quality. And being published by a famous professor from a prestigious university is also no guarantee. If we've learned a…
Earlier quoted context omitted.
/a smaller set of weights for the same LLM./ Distillation is alive and well... Earlier work on model printing also found that it's pretty easy to find smaller sets of parameters which can replicate the behavior of the entire network with pretty good fidelity. Large parameter counts give space to explore, and give routes out of what would be local minima in a lower dimensional space. In other words, there's no guarant…
I'm not claiming any arbitrary set of weights is a minimal representation. But typically, if people could achieve the same quality of results with a smaller set of weights, or weights which have been quantized to lower bit representations, etc, they would have published the smaller one instead.
Your socioeconomic argument just doesn't hold either. People don't delay releasing models until they've minimized it to the theoretical limit. They ship it when it's good enough for whatever job they're making it for.
I am not familiar with the standards of publishing in machine learning, but as someone trained in a mathematics background, this paper seems relatively light on details and heavy on exposition. Is that typical? Is this a really novel idea? Not trying to be snarky, just trying to understand how meaningful this is.
You are not wrong. But this has by no means proven its up to the standard of being publishable in a machine learning journal. Its on arXiv.org, which, lets face it, at the end of the day is a vanity press.
I am not familiar with the standards of publishing in machine learning, but as someone trained in a mathematics background, this paper seems relatively light on details and heavy on exposition. Is that typical? Is this a really novel idea? Not trying to be snarky, just trying to understand how meaningful this is.
What is the (co)homology of this space?
I am not familiar with the standards of publishing in machine learning, but as someone trained in a mathematics background, this paper seems relatively light on details and heavy on exposition. Is that typical? Is this a really novel idea? Not trying to be snarky, just trying to understand how meaningful this is.
It's cool that it proves that a bunch of vectorized outputs from an unknown embedder on an unknown dataset is in no way private, because of this ability to reverse engineer the embedder. I talked to the author at his poster session at neurips and was able to get the gist, though I had read a lot about the platonic representation hypothesis, and this was one of my top 10 favorite papers in the conference.
This, like graph isometry, can be very computationally intensive in the worst case. However, heuristics to aid matching one vertex on one graph to another vertex on another graph using local, semilocal structural signatures can be very effective on particular cases.
One can of course argue that the spaces are not designed as metric spaces. Even if true, these might be metrizable topological spaces.
More generally, if these are indeed non-metric spaces one can still pose it as finding the unknown isomorphism between two poset spaces.
In my other comment I was using the property of maximal chains -- Identify the longest chains in both posets. The isomorphism must map the longest chain in Poset 1 directly to a longest chain in Poset 2, preserving the exact linear order.
Let's assume that monotonocity of pair-wise distances are preserved. Without knowing the details of how the paper solved the problem, my first attempt would be to find the diametrically distant pair of points in the two different embeddings and assume that the pair is the same pair. Then find the next distant pairs and so on. After sufficiently many such pairs have been found, or better still, the largest d-simplex i…
I think a less stringent, but still workable assumption is that for very similair objects, their distances will be small. This is much easier to accomplish than agreement across all pairs.
I explained my thoughts in a comment here