Viewing profile — sasjaws

The idea i tried to express was purely the loss function thing you mentioned, and how both tasks (1 vs 2 vs n) lead to identical training runs. At least with nanogpt. I dont know i…

No blog post, my llm expert friend told me this was kinda obvious when i shared it with him so i didnt think it was worth it. I can tell you how i got there, i did nanogpt, then tr…

This is about reasoning tokens right? I didnt mean that, nanogpt doesnt do that. Nanogpt inference just outputs letters directly, no intermediate tokens.

Thats actually an interesting way to look at it. However i just posted that because i often see articles expressing amazement at how training an llm at next token prediction can ta…

Thats the one, lots of fun and a great entrypoint for experimentation.

A while ago i did the nanogpt tutorial, i went through some math with pen and paper and noticed the loss function for 'predict the next token' and 'predict the next 2 tokens' (or n…

Personalized audio streams for language learners. Ideal for during driving or while doing chores. https://listen.longyan.io/ At the intermediate level lots of learners struggle to …

I've added support for Lithuanian and created a stream about version control for you to try it out. Just 'select language' -> Lithuanian -> Play If you find it useful, you can regi…

I'm building a service that generates audio streams about subjects and vocab of your choosing, currently notebookLM based. If you have intermediate listening skills its pretty usef…

Thanks for having a look, I actually started out from traditional characters, but once I realized >90% of the students only do simplified I switched. I also tend to believe to just…

I'm building a reader app that tries to solve this exact problem by providing a range of gradually simplified versions of each article to match your proficiency. So you can stay in…