Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Karpathy’s Pelican

twitter.com

Post reply on HN
Print view View original

371–380 of 461 posts

Re: Karpathy’s Pelican

#371

Post by didibus »

skygazer wrote: post #286
Wowfunhappy wrote: post #283

Earlier quoted context omitted.

> I think it's touching the limit of what one can reasonably expect any intelligent thing to produce with the only direction being "produce an svg of a pelican riding a bicycle". Oh come now. I am extremely confident that if I hired a professional artist to draw a picture of a pelican riding a bicycle, I would get something inarguably much better than what today's best coding LLMs can produce.

At that rate you could use an image model, which was designed for the task. Thats the absurdity of this test. It’s often a text only generation model that has never seen a pelican, coerced into creating an xml graphical representation that a human might recognize. That it does anything passable is already astounding.

Most of the models are multi-modal and trained on images no? That's what they claim at least.

Re: Karpathy’s Pelican

#372

Post by fwlr »

Buttons840 wrote: post #310
reaperducer wrote: post #279

Earlier quoted context omitted.

Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.

Hugely profitable companies leak half the nation's personal data every month. Tell me more about how being wrong at scale is not valuable.

So what if it is more profitable or valuable? It is still not better. Something being more profitable/valuable does not make it better, just like something being better does not make it more profitable/valuable. Sometimes, in some pursuits, for some outputs under some circumstances, the two are correlated. In others, the two are anti-correlated.

Re: Karpathy’s Pelican

#373

Post by nomel »

didibus wrote: post #370

Did I miss when they managed to draw a pelican? Cause all the ones I've seen are wrong in some way.

Perfection is such an absolutely wild requirement for what we're seeing happen here, with the level of understanding required, from a tech that was complete fiction 5 years ago.

Maybe excitement and wonder, in tech, is just something for us old guys, that watched it all be birthed. Get off my lawn!

Re: Karpathy’s Pelican

#374

Post by 0x1ceb00da »

anigbrowl wrote: post #207
dundarious wrote: post #51

I can forgive the modeling being godawful jank (windows floating in the air, disconnected from the house). But I expected it to have a better understanding of the text. Instead, we have Bilbo's "disappearance" interpreted as him magically transporting or cloaking, and similarly for his reappearance.

But this only seems wrong to you because you're familiar with the prior context. When there's only a single paragraph to work from, and it's a drily humorous text, why not employ comic literalism and lean into the perplexity with which his neighbors viewed him?

I haven't read the book or watched the movies and even then it looked weird as hell.

Re: Karpathy’s Pelican

#376

Post by crabmusket »

nomel wrote: post #373
didibus wrote: post #370

Did I miss when they managed to draw a pelican? Cause all the ones I've seen are wrong in some way.

Perfection is such an absolutely wild requirement for what we're seeing happen here, with the level of understanding required, from a tech that was complete fiction 5 years ago. Maybe excitement and wonder, in tech, is just something for us old guys, that watched it all be birthed. Get off my lawn!

I don't think didibus is saying they expect perfection, just that there is still room for the pelican prompt to measure meaningful future improvement.

Re: Karpathy’s Pelican

#377

Post by the_af »

tayo42 wrote: post #307
revel wrote: post #61

Earlier quoted context omitted.

LLMs are bad at creative work and I don’t see them improving any time soon. Try asking an agent to write a story about raccoons. It will almost certainly involve either stealing food or raiding trash with a 50% chance of having a character named Pip. If a location is mentioned, it’ll be Elm Street. I guess this is the average story and, similarly, the average game is boring and predictable

1/3 I did get pip, but it's a story about trying to rescue the moon from a lake and the raccoon lives on moonberry lane What's pip from?

I got a story about a raccoon named Pip, and friends, trying to catch moon beans.

So "moon" is also part of the predigested artifacts. Maybe because raccoons are nocturnal?

Re: Karpathy’s Pelican

#378

Post by necovek »

jkahrs595 wrote: post #212
garganzol wrote: post #184

Earlier quoted context omitted.

This is a bizarre claim to make in this case, AI tech is used on all platforms universally.

Literally just google “mac users more likely to pay” and you will find many such instances

One of the hits I got was https://www.networkworld.com/article/665211/byod-mac-users-p... — so Linux users obviously pay even more when they are interested ;)

My point being that this is a bit niche, and overall trends might not apply.

Re: Karpathy’s Pelican

#379

Post by singpolyma3 »

charcircuit wrote: post #164
jmugan wrote: post #93

A lot of people are posting here about how bad the end product is, but that is kind of the point. Models have moved beyond generating images to a new kind of benchmark that better exposes understanding of the physical world, and we can use benchmarks like this to measure future progress. (Of course, it will have to be a qualitative/subjective measurement.)

Bad? It has a charming style. I would watch the whole book if it was made like this.

Yeah. It's pretty good. I've seen worse commercial adaptations of this story
Post reply on HN