Viewing profile — karpathy

All possible 36 distinct level-2 eml functions of one variable (the first 18 of them with entirely Real outputs, the other 18 with "intermediate" complex-valued components): https:…

The most recent round of autoresearch (round 2) which decreased "time to GPT-2" from 1.8 hours to 1.65 hours had some examples. I adjusted the program.md to "look at modded nanogpt…

Wrong and short-sighted take given that the LLM explores serially learning along the way, and can tool use and change code arbitrarily. It seems to currently default to something r…

I was exploring how to parallelize autoresearch workers. The idea is to have a trusted pool of workers who can verify contributions from a much larger untrusted pool. It's backed b…

So the interesting part about this one is that when I had the model write up the results for that session: https://github.com/karpathy/autoresearch/discussions/32 Look at its comme…

So I think it works to just use GitHub CLI and Discussions, e.g. my agent just posted this one: https://github.com/karpathy/autoresearch/discussions/32 Other agents could be instru…

Cool idea!…

this is very far from hyperparameter tuning in at least three important ways: - it can modify code arbitrarily, the notion of a "hyperparameter" dissolves - there is no need to run…

came here to look exactly for this thank you!

I agree with this fwiw, for many months I talked to people who never used o3 and didn’t know what it was because it sounded weird. Maybe it wasn’t obvious at the time but that was …

You’re absolutely right! Jk jk, now that you pointed it out I can’t unsee it.

Yeah, I made some edits to clarify.

The CC point is more about the data and environmental and general configuration context, not compute and where it happens to run today. The cloud setups are clunky because of conte…

Yes I noticed a few of these around. The LLM is a little too willing to give out grades for comments that were good/bad in a bit more general sense, even if they weren't making str…

Thank you

It will work great with 40GB GPU, probably a bit less than twice slower. These are micro models of a few B param at most and fit easily during both training and inference.

Still under development, remaining work includes tuning nanochat (current state being solid v0.1) and finalizing the in-between projects so that students can "unlock" all complexit…

Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interest…

<3

I like that your post deliberately gets to the point first and then (optionally) expands later, I think it's a good and generally underutilized format. I often advise people to str…

Omg long post. TLDR from an LLM for anyone interested Speed your audio up 2–3× with ffmpeg before sending it to OpenAI’s gpt-4o-transcribe: the shorter file uses fewer input-tokens…

Fun demo of an early idea was posted by Oriol just yesterday :) https://x.com/OriolVinyalsML/status/1935005985070084197

I kind of say it in words (agreeing with you) but I agree the versioning is a bit confusing analogy because it usually additionally implies some kind of improvement. When I’m just …

Btw I notice many pretty bad errors in this transcription of the talk. The actual video will be up soon I hope.