151–160 of 326 posts
Earlier quoted context omitted.
> the repo is too far off the data distribution ah, this explains why these models have been useless to me this whole time. everything i do is just too far off the data distribution!
Everything is unless your app is a React todolist or leatcode questions.
Earlier quoted context omitted.
As noble as the goal sounds, I think it's wrong. Software is just a tool. Much like a hammer, a knife, or ammonium nitrate, it can be used for both good or bad. I say this as someone who has spent almost 15 years writing software in my free time and publishing it as open source: building software and allowing anyone to use it does not automatically make other people's lives better. A lot of my work has been used for…
"With what is called AI today, only the largest corporations can afford to train the models" I take it you're very positive about Andrej's new project which allows anyone to train a model for a few hundred dollars which is comparable to the state-of-the-art from just 5 years ago then.
Can I run it on my local hardware (nvidia consumer card, AMD cpu)? No. When could that corporation cut off my access to that hardware if I did anything it didn't like? Anytime.
Lots of things have started off cheap / subsidized to put competitors out of business, and then the prices go up, up and up..
Earlier quoted context omitted.
That is a good thing to hear from someone as reputable as Karpathy. The folks who think we're on the cusp of AGI may want to temper their expectations a bit. I do love Claude Code, because one thing I periodically need to do is write some web code, which is not my favorite type of coding but happens to have incredibly good coverage in the training data. Claude is a much better web developer than I am. But for digging…
> But for digging into the algorithmic core of our automation tooling What I find fascinating is reading this same thing in other context like “UI guru” will say “I would not let CC touch the UI but I let it rip on algorithmic core of our automation tooling cause it is better at it than me…”
But 'mediocre' isn't 'useless'.
Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904
Earlier quoted context omitted.
You wildly misunderstand pytorch.
What is there to misunderstand? It doesn't even install properly most of the time on my machine. You have to use a specific python version. I gave up on all tools that depend on it for inference. llama-cpp compiles cleanly on my system for Vulkan. I want the same simplicity to test model training.
Earlier quoted context omitted.
Everything is unless your app is a React todolist or leatcode questions.
or a typical CRUD app architecture, or a common design pattern, or unit/integration test scaffolding, or standard CI/CD pipeline definitions, or one-off utility scripts, etc... Like 80% of writing coding is just being a glorified autocomplete and AI is exceptional at automating those aspects. Yes, there is a lot more to being a developer than writing code, but, in those instances, AI really does make a difference in…
I wonder, if something like this were trained on Wikipedia, could it become a reliable local Wikipedia search engine, basically?
I don't think so. Training on documents is not a great way of building a search engine for those for the information in those documents, because the training process mixes all of that information together in ways that detach the individual words from the source documents they came from. As usual, if you want an LLM to be able to help search a corpus of text the best way to achieve that is to teach it how to use a sea…
Any examples of this?
Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904
AI can write better code than 99% of developers. This embarrassingly anti-AI shill included.
If he used the AI tool my company is developing the code would have been better and shipped sooner.
Interesting exchange on the use of AI coding tools: curious how much did you write the code by hand of it? Karpathy: Good question, it's basically entirely hand-written (with tab autocomplete). I tried to use claude/codex agents a few times but they just didn't work well enough at all and net unhelpful, possibly the repo is too far off the data distribution. https://x.com/karpathy/status/1977758204139331904
This makes sense, right? It's a relatively novel thing to be writing. I don't find it to be a damning remark like other comments here seem to be concluding. If anything, the fact that Karpathy reached towards Claude/Codex in an attempt to gain value is indicative that, in previous coding efforts, those tools were helpful to him.
This is good for bitcoin.