Anthropic spokesman [0] Andrej Karpathy is here to tell you about token-wasting loops, and insists on the weird idea that they are "~free", when in fact, they are fuelled by expensively burning investor money. [0] Seriously. Get used to mentally prefixing his and Boris Cherny's name like this, every time you see them quoted. These people are speaking while employed; there is no chance they are not aligned with the em…
Karpathy’s Pelican
311–320 of 461 posts
Re: Karpathy’s Pelican
#312There is a tipping point between procedurally generating everything in SVG to maybe giving them tool access to something like 3dsmax (or having them build and then use a tool to do the thing vs doing the thing).
I’d love to see this benchmark using Blender. Asking a model to animate a scene in Three.js is a square peg/round hole; it doesn’t convey much when the model can’t get it to fit. With Blender the human expert ceiling for this task has been proven to be very high
Re: Karpathy’s Pelican
#313Earlier quoted context omitted.
> I think it's touching the limit of what one can reasonably expect any intelligent thing to produce with the only direction being "produce an svg of a pelican riding a bicycle". Oh come now. I am extremely confident that if I hired a professional artist to draw a picture of a pelican riding a bicycle, I would get something inarguably much better than what today's best coding LLMs can produce.
> if I hired a professional artist to draw a picture of a pelican riding a bicycle, I think AI folks have done a terrible job of communicating this, but replacing a professional simply isn't the point. The point is to serve all the situations where people would've never considered hiring a professional, and where perfection or artistic merit isn't the point (say a personal throwaway recreation of an LOTR world). And…
I'm not saying it is, just that there's obviously still room for the models to improve on this task.
Re: Karpathy’s Pelican
#314Earlier quoted context omitted.
Agreed. It's far from solved. Modern LLMs still generate pelican bike SVGs with obvious errors: * some omitted the bottom of the diamond which connects from the pedals to the rear wheel * some added an extra connection from the pedals to the front wheel, making it impossible to steer * none could align the head tube with the fork * none added a correct offset to the fork * none could generate the chain properly in a…
what's fable at, does anyone know?
The bike is generally okay, apart from medium which derped hard. Max has correct diamond, correct head tube, and so on. Only nitpicks are that the front fork offset isn't there and the chain doesn't touch the rear sprocket correctly.
Re: Karpathy’s Pelican
#315Earlier quoted context omitted.
Aren't they still bad at understanding how bicycle frame works? Especially the steering part?
I don’t know what datasets are available to these LLMs, but I’d imagine if there was training on CAD code, text, and images, a prompt steered towards that probably could get it pretty good. I am not a mechanical engineer, so even prompting well with ME lingo probably will take some effort.
Re: Karpathy’s Pelican
#316Earlier quoted context omitted.
Gotta love when a techbro just says some complete nonsense like this with total confidence
Grok build me a spaceship to mars, make no mistakes
Re: Karpathy’s Pelican
#317Earlier quoted context omitted.
> I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle Please remember, we've started from there : https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ When it started, it was clear what LLM would stand out, its style, etc. Nowadays, the pelicans look similar, the difference is in details and sometimes hard to catch. Sure, the task is not completed perfectly, but that's not…
Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.
It can certainly do it better than I can. Sometimes you don't have a human handy with the required skills to do something.
Re: Karpathy’s Pelican
#318It seems pretty clear that Anthropic models have been specifically trained to be good at generating three.js (JavaScript 3-D Graphics) code, so given current state of AI code generation in general, I don't find three.js models/animations as indicative of anything other than the model's ability to write three.js code. When Fable was first released the day-1 demos of it on Twitter (presumably from people who were given…
Re: Karpathy’s Pelican
#319Earlier quoted context omitted.
I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle. Not if you look at the image long enough to take it in. Even the best ones have something wrong with them. Not a matter of taste but a matter of having both legs peddling on the viewer's side of the bicycle or having two beaks. I'm actually beginning to wonder if some people who ignore these things have a different, somewhat less…
Pelicans don't ride bicycles. It's physically impossible. The problem is to draw it in the least disturbing way possible.
https://ridesabike.com/donald-duck-daisy-duck-huey-dewey-and...
Re: Karpathy’s Pelican
#320Earlier quoted context omitted.
What you’re saying is that you’re not interested in benchmarks. But then you go a step further and state that this particular benchmark is entirely inconsequential. That’s like telling you that if you didn’t exist, nothing would change. Even if that were true, it would still be an insensitive and rude thing to say, wouldn’t it?
It's okay to be rude to benchmarks though, they don't have feelings.