Viewing profile — whoateallthepy

I learned Git from an O'Reilly book and I loved that it started with the internals first. The git CLI has some rough edges, but once you have concepts of work tree, index, commits …

One point: if you are re-reviewing, other platforms (e.g. Phabricator, Gerrit) have much more developed ways to compare changes relative to one another.

One thing to bear in mind is that these embedding vectors are high dimensional, so that it is entirely possible that the token embedding and position embedding are near-orthogonal …

The tokenization is done by the tokenizer which can be thought of as just a function that maps strings to integers before the neural network. Tokenizers can be hand-specified or le…

This is a great set of comments/questions! To try and answer this a bit briefly: The input string is tokenized into a sequence of token indices (integers) as the first step of proc…

At the end of last year I put together a repository to try and show what is achieved by self-attention on a toy example: detect whether a sequence of characters contains both "a" a…

I put together a repository at the end of last year to walk through a basic use of a single layer Transformer: detect whether "a" and "b" are in a sequence of characters. Everythin…