Viewing profile — ianand

In that case I humbly suggest my talk from AI Engineer World's Fair https://www.youtube.com/watch?v=ZuiJjkbX0Og Gives you the basics on LLM internals in about 90 minutes and includ…

It's like "GPT is all you need for the backend" [1] on steroids [1] https://news.ycombinator.com/item?id=34503418

Since you mention serverless it might be worth mentioning firecracker and v8 isolates.

I'm not a fan of the database lookup analogy either. The analogy I prefer when teaching attention is celestial mechanics. Tokens are like planets in (latent) space. The attention m…

The site’s domain name is the best use of a .fail tld ever.

Lol. You joke but... https://www.reddit.com/r/GraphicsProgramming/comments/pg9dw8...

Dude, been forever. Thanks. Will DM you.

As the guy who did GPT2 in Excel, very cool and kudos!! Curious why you chose WebGL over WebGPU? Just to show it can be done? (Also see my other comment about fetching weights from…

Checkout https://github.com/jseeio/gpt2-tfjs fetches the weights for GPT2 from huggingface on the fly.

Thanks. Took a quick look and definitely needed to use Google Translate but seems to have worked to get the gist of it.

Sounds like an interesting masters thesis. Is your masters thesis available online somewhere?

Fun fact: A decade ago the designer of HAML and Sass created a modern alternative to XSLT. https://en.wikipedia.org/wiki/Tritium_(programming_language)

The model architecture is the same during RL but the training algorithm is substantially different.

> LLMs that haven't gone through RL are useless to users. They are very unreliable, and will frequently go off the rails spewing garbage, going into repetition loops, etc...RL lear…

hey, creator of spreadsheets-are-all-you-need.ai here. Thanks for mentioning! I now have a web version of GPT2 implemented in pure JavaScript for web developers at https://spreadsh…

Reminder that Microsoft ships RWKV with Windows (~1.5 billion devices), making it probably the most widely deployed non transformer model out there. Amazing work! https://blog.rwkv…

> Even things like how cells are frozen (if you're on B2, are you freezing the first row, or the first and second row?) just feels wrong. FWIW one my frustrations with Excel is how…

Google Sheets does struggle past a certain size. For Spreadsheets-are-all-you-need (GPT2 Small implemented in Spreadsheet, ~1.25GB Excel file, 124M parameters), I have only been ab…

Don't entirely disagree but I think "mad" is an unfair characterization and simplifies how the fascinating competitive dynamics have played out that resulted in the Llama models. M…

This has been my mental model as well. I often use two analogies to describe this. The processed a food is the more likely you’re eating a “Product”, i.e. something that’s been des…

They did invent their own functions to test if the results were due to these functions being on the training date. See the section on data contamination in the paper. Agree it both…

Thanks for this. Listening to an author on a podcast is great hack for getting a good summary of a researcher’s or book author’s views. (Especially business books)

It's actually available in beta. It was announced while I working on this project but I kept going with pure Excel functions because I wanted to illustrate the transformer without …

This is so awesome!! I'm going to have to show this in the embeddings video I'm working on when I discuss non-text embeddings and CLIP. While I created spreadsheet-are-all-you-need…

Hopefully this outlasts the harlem shake bookmarklet...