Viewing profile — qeternity

LLMs are, in theory, deterministic. Sampling is not intrinsic to LLMs. Greedy decoding a single batch in most libraries will give you mostly deterministic outputs. Higher batch siz…

I don't follow this. Clearly all of the frontier labs are doing these things. When OAI released gpt-oss it was released as an mxfp4 checkpoint. OAI, Ant, et al are also obviously e…

I cannot be sure what the likes of Cursor have done, but I think it's incredibly unlikely that they have trained a QLoRA for Composer. It's almost certainly full parameter post tra…

I think it’s fairly obvious the Chinese labs are doing mass distillation. I also think the Fable accusation is wrong and it was most likely Opus 4.8 which itself is likely a distil…

But these aren't the right services where the test should be, right? There's another service that says "ok we take the 100 bytes from A, and we take the $17 SKU from B, and this sh…

It's a defensive tactic to reduce the effectiveness of distillation. Say of that what you will, but it's not because they want to wrest control from users. It's because they don't …

Public in this context just means publicly listed on a stock exchange. It does not mean it is state-owned.

Empirically this isn't true. However you feel about "true" capitalism vs socialism, countries underpinned by capitalism have prospered, even the socialist flavors (China, Scandinav…

> is supposed to be the central bank's job. What? No it's not, and never has been. Without even getting into the practical vs. theoretical of Fed dual mandate (funding deficits), e…

We are quite literally talking about launching 100k satellites into space. This is necessarily a question that requires thought and speculation about the future. You can't just fli…

It's sort of amazing that people still say these sorts of things today. I don't think it's at all inconceivable where people in the future are streaming high resolution multi modal…

There is a huge amount of vendor financing available for this exact reason. Hetzner presumably have been making use of this for years already. It's far more likely they simply bega…

I think it very much is worth it! But the point was that quality didn't magically increase.

It's not better quality: 59.3% vs 59.4% fp16 on AIME 25

> Venture capitalists & private investors are sucking all of the possible growth and future upside from these companies and then dumping them on retail investors when there's nothi…

I haven't read TFA yet but a common technique is speculative decoding where a fast draft model will generate X tokens, which are then verified by the larger target model. The targe…

> They paid a billion dollars for a vibe coded mess just for the opportunity to associate themselves with the hype. Lol no they didn't. It wasn't even an acquihire. They just hired…

> It's not any company, its Meta and the channels they administrate come with a set of responsibilities and principles Sorry, which laws stipulate these special responsibilities an…

> or if the model might actually have emitted the formatting tokens that indicate a user message. These tokens are almost universally used as stop tokens which causes generation to…

This does not solve the problem at all, it's just another bandaid that hopefully reduces the likelihood.

Yes, it is written for a specific audience. That is not a reason for snark. As other commenters have noted, it’s well written.

Yes, lots of things can create indeterminism. But nothing is inherent.

Yes, lots of things can create indeterminism. But nothing is inherent.

> LLMs are inherently non-deterministic. This isn't true, and certainly not inherently so. Changes to input leading to changes in output does not violate determinism.

> With prompt caching, verbose context that gets reused is basically free. But it's not. It might be discounted cost-wise, however it will still degrade attention and make generati…