Viewing profile — flebron

I think you might be thinking of "AGI" as some sort of point in time, where something happens and everyone all at once has some technology. Not only is the progress towards AGI gra…

Source for this? The notion of attention dates to a content-addressable lookup during sequence alignment (as well as, concurrently, memory lookups in neural Turing machines). Atten…

The standard way of doing MTP is to run the drafter autoregressively for k steps, and then (not concurrently) use the larger model as a verifier for those k tokens at the same time…

The website asks what they do in Haskell. The answer is property modification and reading, as well as very powerful traversal constructs, use lenses ( https://hackage.haskell.org/p…

I think the point is that a monad is a useful concept _purely_ because of what it _allows_ you to do, and _not_ because of anything syntactical. Those rules that you're obviating t…

This is an excellent summary of these techniques :) I like that every single one comes with an example implementation, with shape comments on the tensors. Thanks Stephen!

Perhaps this chapter can help? https://jax-ml.github.io/scaling-book/tpus/ It's a chip (and associated hardware) that can do linear algebra operations really fast. XLA and TPUs wer…

All of them are vectors of embedded representations of tokens. In a transformer, you want to compute the inner product between a query (the token who is doing the attending) and th…

No. In the common use of the word fine-tuning, one is in the supervised learning scenario. One has an input prompt, and an output sentence. One teaches the model to say that output…

"Almost nobody except X does Y." and "Z does Y, with Z != X" are consistent. Is your disagreement entirely due to the (possibly nil) distinction between "Almost nobody except X doe…

I don't see a link between that Bloomberg article and these kidnappings happening mostly to married men cheating. Nor is it the case that "love motels" are primarily used for cheat…

I think k8sToGo's point is that she never "joined Alphabet", Alphabet did not exist when she joined Larry and Sergey. She joined concurrent with Google's founding as a company, she…

The trifecta of: * There's almost always a simple geometric intuition, and low-dimensional intuition can get you quite far even in high dimensional cases. * You can surprisingly of…

That seems a weird use of the word. By that notion, every convicted criminal is a victim, because the court imposed a sentence that they, presumably, do not like. This removes prac…

pc = Parsec :) https://en.wikipedia.org/wiki/Parsec I don't know what the G stands for. Possibly luminosity of some kind?

Well, most interesting properties about computer programs are, in general for all programs, undecidable ( https://en.wikipedia.org/wiki/Rice%27s_theorem ). Undecidability is a clos…

Consider a disconnected domain (say, union of a few open balls in R^n), and f being constant in each connected component, but having different values in each ball. The differential…

I've always found the definition of monoid objects in a category of endofunctors to be tougher to grasp than the Kleisli-category definition of a monad, at least if one is being fo…

Yes.

"1 bowl of rice" gets interpreted as 1 cup of Sake?

To a sufficiently advanced alien species, _we_ are neural networks fed with some pictures (and sounds, smells, touches, and tastes) :)

The report is signed by him, so it's safe to say that report is an accurate recollection of what happened according to him. Having a report on the security of a system be issued mo…

Unfortunately they did. You can see the request for review here[1], and the actual review here[2]. I know the guy who actually did it (Righetti), he sent an email to the department…

Do you also trust your package maintainer to not mismerge patches and leave you open to security bugs? Your argument is no bueno.