Viewing profile — lewtun

Shameless plug: https://huggingface.co/spaces/smolagents/ml-intern It’s a simple harness around Opus, but with tight integration to Hugging Face infra, so the agent can read papers…

Hugging Face Buckets are pretty simple: https://huggingface.co/docs/huggingface_hub/en/guides/bucket... Disclaimer: I work at HF

The analogy stems from the notion that neural nets are "grown" rather than "engineered". Chris Olah has an old, but good post with some specific examples: https://colah.github.io/n…

Thanks! I expect the book will remain relevant as long as the Transformers architecture does. That’s why we mostly focus on topics we think will stand the test of time, but let’s s…

In the specific case of SmolLM, it originates from the meme in this dataset https://huggingface.co/datasets/bigcode/the-stack-smol

Hi, Lewis here (one of the co-authors). Happy to answer any questions people have about the book :)

For those interested in playing with an implementation of these ideas, my colleagues at HF made some recipes here: https://github.com/huggingface/trl/blob/main/docs/source/lor...

“QED and the Men Who Made It” [1] might be close to what you’re after for quantum theory at least. Unlike other popular accounts, it gets quite technical and covers a lot of the hi…

> We instantiate this idea through Preference-prior Informed Linucb fOr adaptive rouTing (PILOT), a novel extension of LinUCB Academics are pretty creative at naming their creation…

Indeed we opted for offline methods like Anchored Preference Optimization as we found in the Open R1 project that doing multi-task RL on small models is quite a hassle to get right…

> The absolute best way of doing this is these days is likely through a vision based machine learning model, but that is an approach that is very far away from scaling to processin…

I gave the demo a spin and it’s pretty nice! One thing I noticed is that the avatar doesn’t seem to be aware of it’s surroundings- for example, I asked it why it was wearing a cowb…

> I expect language models to also get crazy good at mathematical theorem proving Indeed, systems like AlphaProof / AlphaGeometry are already able to win a silver medal at the IMO,…

Hello everyone, we just did a speed run with Argilla and KAIST AI to fine-tune the beefy new Mixtral model with some new techniques that came out recently. More details in the mode…