Viewing profile — croqaz

Thank you very much! It is humbling and motivating to see other people interested in this.

Do share! I read all the blog posts where people share their experiences of building small scale LLMs "from scratch".

That's a fair point TBH. I said in my post that this LLM is first of all a learning project and I skipped an important step: the training loop. But on the other hand, how many data…

"A more granular followup would be cool too" Do you mind expanding this question? More granular in what way? what would you like to know that is missing from the post?

That's exactly what I had in mind. When I started this, I was jumping back and forth between this thought: "Can this model size actually generate logical English text?" and I playe…

It looks like ROT13 text to me, I hope it's not Welsh. Don't want to offend anyone if that's their actual language :)

I am creating my tiny Llama 340M base model from scratch. If you're curious about the steps, challenges and cost, read on. I am still working on the instruct model.

TwoFold is a small command line app that allows plain text files to behave like dynamic files. It is a hybrid between a text expande, a template engine and a mini programming langu…

What are web snapshots; comparison of the most popular methods: WARC, HTML, rrWeb and a better alternative called "recorded".

DYI blocking, a few methods

Tomb consists of a simple shell script (Zsh) using standard filesystem tools (GNU) and the cryptographic API of the Linux kernel (cryptsetup and LUKS).

The same in Romania. In the 90s, everyone I know had to learn how to write with a fountain pen. For me at least, I write horribly when I use a ballpoint pen, vs a fountain pen. Bec…

I love my Plaisir too! I have a green color and of course, it's always inked with green :)

Wow, I had no idea my crappy post ended on Hacker news! I just collected a bunch of info and links for myself :) I guess I have to update the post with some pictures, at least...