Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are AI labs pelicanmaxxing?

dylancastillo.co

Post reply on HN
Print view View original

201–210 of 253 posts

Re: Are AI labs pelicanmaxxing?

#201

Post by somenameforme »

nl wrote: post #197
somenameforme wrote: post #190

Earlier quoted context omitted.

Which models have you found good for working with 3d stuff?

ChatGPT 5.5 and Sol 5.6, Fable are good. I haven't really tested Opus 4.8, but 4.7 wasn't nearly as good as ChatGPT 5.5.

Interesting! The reason I asked is I've had poor results with Fable and 3d stuff. Its spatial awareness seems poor - doing things like rotating left and then right back, and then pitching nonsensically, just to try to capture a segment of a scene for a verification pass. And its placement often results in clipping, misrotations, and so on.

It could well be that the exact domain matters more than the bigger picture concepts, like 3d. One of the ever fewer reminders that this tech is still just fundamentally a token prediction algorithm.

Re: Are AI labs pelicanmaxxing?

#202

Post by seebeen »

qq66 wrote: post #59
lukev wrote: post #44

Earlier quoted context omitted.

What if they’re not pelicanmaxxing, but svgmaxxxing in general? Because otherwise using a LLM to generate complex svgs is pretty niche and what I thought made this a good benchmark when it was new - generalized programming and spatial knowledge. Obviously image gen in svg format is not a particularly hard problem if tackled directly on its own.

But that's a genuine worthwhile capability. It's like benchmarkmaxxing on a weightlifting competition by getting really strong.

I love this comment.

Re: Are AI labs pelicanmaxxing?

#204

Post by lionkor »

mauvehaus wrote: post #27

> All 21 pelican-bicycle images, across all seven labs, face right. No other animal/vehicle combination does that. > However, facing right is common: 60% of all 1,008 images do it. How common depends on the animal and the vehicle, and bicycles are one of the two vehicles where it’s strongest Of course the pelican on the bicycle is facing right. The drivetrain on a bicycle is on the right side. If you want any represe…

> This suggests a pretty serious lack of actual understanding of how a bicycle works

an... understanding? there is no understanding here at all

Re: Are AI labs pelicanmaxxing?

#205

Post by pbhjpbhj »

oasisbob wrote: post #121

As a unicyclist, all the sideways-riding caught my eye for being especially silly. However, I'm very surprised that most of the models make the same sideways mistake with only some of the animals, and they do it consistently. With most of the models, cat, raccoons, and otters are almost always riding sideways. Why is that?

Pose bias? Most pictures of animals like penguins are face on (I think they're mainly at 45°, and birds are general profile; so perhaps it's a more generalised bias)? Or perhaps the 'recognisable' is correlated with face-on?

Re: Are AI labs pelicanmaxxing?

#207

Post by reilly3000 »

I fear there this reveals something else - not about the nature of models but of our community. The intro of the article seemed to affirm the importance of HN as a tastemaker, potentially influencing decision-makers on the scale of billions or trillions. For the 15 or so years I’ve been hanging around here, that only feels like mild hyperbole… a good LaunchHN can reshape the future, right?

It made me realize that for this time around, we’re not at the center anymore. The future of LLMs is the stuff of nations and AIG is what the labs actually care about. They aren’t pelicanmaxxing just as much as they really aren’t revenue/margin/marketing maxing. They just want our attention and ideas so they can show growth and acquire FLOPS.

The apparent fact that they aren’t catering to this community (who frankly decides what goes and stays in prod) leaves me feeling a bit defeated somehow. And also awesome?! Like there is a culture here that runs deeper than any technology and it has fought hard to maintain its identity. Props to dang and all for keeping the astroturfing so imperceptible that I can say this.

In any case, a great little piece of citizen-science dcastm. Lmk if there is a way I can chip in towards token costs.

Re: Are AI labs pelicanmaxxing?

#208

Post by amelius »

sandcat_ wrote: post #157
qq66 wrote: post #59

Earlier quoted context omitted.

But that's a genuine worthwhile capability. It's like benchmarkmaxxing on a weightlifting competition by getting really strong.

Yes, but the original purpose of the benchmark (simonw, please correct me if I'm wrong!) was to test whether new models were good at novel problem solving. Things they haven't been trained on. So yes, getting better at generating SVGs is great news (and it seems they have been) but this particular benchmark still strikes me as largely worthless now, unless SVGs happen to be what you care about in particular when a ne…

How they will approach it eventually will probably be to let a diffusion style model render the image, then run some algorithm to convert that to svg.

Re: Are AI labs pelicanmaxxing?

#209

Post by amelius »

sheept wrote: post #167
cheesecakegood wrote: post #138

Earlier quoted context omitted.

Interestingly, there was an artist a few years back who (for an unrelated project) had almost 400 people across a range of ages draw a bicycle and 75% of those faced left! So this seems to actually go slightly against the human drawing intuition. https://www.gianlucagimini.it/portfolio-item/velocipedia/ On the other hand, I notice that the prompt says to draw a pelican riding a bicycle, implying motion... and since m…

That's interesting. I think the bikes-facing-left bias probably comes from how humans use bikes: the kickstand is on the left, so people likely hold and approach bikes from its left. Looking online, the kickstand is apparently on the left to avoid the gears. Based on the other comment about bike photography, it's interesting the same design choice makes humans and cameras/LLMs see bikes from different sides.

It's the other way around: the kickstand is on the left because people jump on their bikes by swinging their right leg. Therefore looking at a bike from the other side feels awkward because you don't feel the same urge to jump on it and ride it.

Re: Are AI labs pelicanmaxxing?

#210

Post by gpjanik »

TLDR: the experiment asks for in-distribution responses and gets those.

The right answer here is to ask a LLM to create a scene similar in quality to those, but completely out of distribution.

I asked GPT 5.6 Sol to give me a pelican playing football on San Siro while smoking a cigarette, in AC Milan's t-shirt. While this sounds like higher complexity of a problem, the generations from current models often include additional details like scene composition, scarf, etc., I don't ask for, so I wanted to see what here is memorization vs. composition skill.

"write svg code of a fish playing football on san siro in ac milan's t shirt, with raybans on and a cigarette."

Try that on GPT 5.6 Sol, Fable, or whatever other model. It's chaos.

Post reply on HN