Viewing profile — kherud

Imagine what amazing SVG generators we could have if Simon had randomized the target image from the start (and companies wouldn't just overfit on pelicans).

Feedback about creatine often seems mixed. Many love it, but many also report problems. Just in this thread there are people talking about heart palpitations and sleep problems. I …

LLMs are extremely capable at problem solving. Presumably because you can autonomously learn a lot of it. But can you somehow account for things like long-term maintainability and …

If I'd have to make one recommendation it's David August's Boiler Room set [1]. It has such a coherent flow through the whole set, it makes me fly through multiple hours if not day…

SQLite seems very powerful for building FTS (user enters free text, expects high precision/recall results). Still, I feel like it's non-trivial to get good search quality. I think …

One interesting detail: In previous years, Joscha Bach gave a talk on AI, consciousness, and related topics (see e.g. [0]). A similar talk was planned for this year as well, but af…

I'm a fan of antlr-ng. It's a solid upgrade if you're already using antlr. In my experience, they're fully compatible. antlr's ALL(*) parsing is relatively powerful for a parser ge…

It's probably "Reflections on Palantir" https://news.ycombinator.com/item?id=41855006

Is there a general solution to this problem? I assume you can only start buffering tokens once you see a construct, for which there are continuations, that once completed, would le…

That was my association as well! Dune even uses similar vocabulary. For example someone mentioned "pranayama" in this thread, which sounds a lot like Dune's "Prana-bindu". Really m…

Aren't LLMs much more limited on the amount of output tokens than input tokens? For example, GPT-4o seems to support only up to 16 K output tokens. I'm not completely sure what the…

Maybe you're already aware of it, but there is difftastic [0], which is a syntax aware diff tool that can also be used with git. Its understanding of syntax is based on treesitter,…

Interesting, thanks for sharing! Do you have an explanation or idea why compilation slows some architectures down?

I think it's more about invested work vs. reward. Mindless browsing is one of the lowest work activities, but the influx of information is highly rewarding for the brain. That's wh…

Let's say you want to show a modal, which fetches some data and modifies the state. Based on this, new children are rendered which again fetch state. The problem of "spaghetti fetc…

Shouldn't this theory be testable? The response time for an image of the same size should remain constant (assuming a generated response of constant size). You could then try to pu…

I think this comment explains it https://github.com/ggerganov/llama.cpp/discussions/4130#disc... As far as I understand (and mcharytoniuk should better confirm this), llama.cpp all…

Now that context length seems abundant for most tasks, I'm wondering why sub-word tokens are still used. I'm really curious how character-based LLMs would compare. With 2 M context…

Thank you for sharing! On a tangent: I'm wondering if there are any good open source models/libraries to reconstruct audio quality. I'm thinking about an end-to-end open source alt…

Why is this the expected result? The original transformer algorithm has a n^2 computational complexity, where n is the amount of tokens. As far as I know, there are some improvemen…

Can somebody please explain how quantization below 8 bit works? Since a byte is the smallest addressable unit I think, is the dimensionality of the weights somehow reduced?

Answer Set Programming is an incredibly powerful tool to declaratively solve combinatorial problems. Clingo is one of the best open source implementations in my opinion: https://gi…

I can recommend the puzzle books by Raymond Smullyan [0], for example "Satan, Cantor and Infinity". Among others, this contains puzzles about infinities and apparent paradoxes asso…