Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra on OpenRouter

openrouter.ai

Post reply on HN
Print view View original

161–170 of 255 posts

Re: GPT-6 Astra on OpenRouter

#161

Post by stymaar »

kibae wrote: post #89
steve-atx-7600 wrote: post #77

Earlier quoted context omitted.

You would not expect the developers of the model to optimize for a well known benchmark?

There was a HN post that tested this hypothesis a few weeks ago: https://news.ycombinator.com/item?id=49010129

I don't think it's a great rebuttal, quite the opposite actually: it's the same prompt with the most obvioy tweak you can imagine and the results are all the same. That definitely looks like it's been something the models have been trained on, and not some emergent capability (and if you take into account how nonthinking Luna compares to the SotA models from a year an half ago: https://simonwillison.net/tags/pelican-riding-a-bicycle/?pag... it becomes even more obvious)

Re: GPT-6 Astra on OpenRouter

#162

Post by maleldil »

stymaar wrote: post #153
dector wrote: post #100

Earlier quoted context omitted.

Btw, they had 3D pelican-on-the-bicycle easter egg in one of the promo videos: https://youtu.be/bOC3DisEOfg?t=117 so I'm pretty sure that they spent some small amount of resources to train the model to produce good svg version as well. :D

When you see how good the output of the Luna model without reasoning is compared to SotA just a year and a half ago, it's pretty clear that it's been trained on explicitly.

I don't see how this is proof, and not just that the model got the better. You're comparing models a year and a half apart; this is a lifetime in LLM development.

Re: GPT-6 Astra on OpenRouter

#163

Post by scotty79 »

simonw wrote: post #19

I posted this in the other Astra thread but it's just fallen off the homepage, so... Pelicans from Astra, plus 5.6 Sol, Terra, Luna for comparison: https://static.simonwillison.net/static/2026/gpt-6-and-5.6-p... I think this is a genuinely interesting comparison grid. Astra may be more expensive, but if you have a budget of 10 cents for a Pelican Astra low gives you something SO much better than the other models. Ast…

They really can't decide which way pelican's knees should bend, do they?

Re: GPT-6 Astra on OpenRouter

#164

Post by spockz »

Gander5739 wrote: post #124
bahmboo wrote: post #94

Earlier quoted context omitted.

Would be great to spend some money (other people's money) on paying some illustrators and artists to do the same for comparison. It could be straight images converted to SVG or direct vector graphics.

Doesn't that somewhat miss the point? The models can't see and check their work, for instance.

They can. Ask them to generate SVG , call a renderer and let them inspect the bitmap/png. They can learn about it.

Re: GPT-6 Astra on OpenRouter

#165

Post by boxed »

pointitkememe wrote: post #138
simonw wrote: post #96

Earlier quoted context omitted.

Here's 'Generate an SVG of a ring-tailed lemur riding an electric scooter' at reasoning level max: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... Quote from the thinking trace: > I’m thinking about how a helmet would obscure lemur ears, but using an electric scooter helmet seems responsible. It's pretty solid - face is a little wonky but excellent tail and scooter.

[flagged]

The lemur is worse though. The hands don't follow physics for example. You claim this would be easier, but then the model is worse.

Re: GPT-6 Astra on OpenRouter

#166

Post by theturtletalks »

dector wrote: post #100
simonw wrote: post #19

I posted this in the other Astra thread but it's just fallen off the homepage, so... Pelicans from Astra, plus 5.6 Sol, Terra, Luna for comparison: https://static.simonwillison.net/static/2026/gpt-6-and-5.6-p... I think this is a genuinely interesting comparison grid. Astra may be more expensive, but if you have a budget of 10 cents for a Pelican Astra low gives you something SO much better than the other models. Ast…

Btw, they had 3D pelican-on-the-bicycle easter egg in one of the promo videos: https://youtu.be/bOC3DisEOfg?t=117 so I'm pretty sure that they spent some small amount of resources to train the model to produce good svg version as well. :D

This is one of those benchmarks I don’t mind if they benchmax because it would mean models are actually good at creating svg graphics.

Re: GPT-6 Astra on OpenRouter

#167

Post by boxed »

stymaar wrote: post #161
kibae wrote: post #89

Earlier quoted context omitted.

There was a HN post that tested this hypothesis a few weeks ago: https://news.ycombinator.com/item?id=49010129

I don't think it's a great rebuttal, quite the opposite actually: it's the same prompt with the most obvioy tweak you can imagine and the results are all the same. That definitely looks like it's been something the models have been trained on, and not some emergent capability (and if you take into account how nonthinking Luna compares to the SotA models from a year an half ago: https://simonwillison.net/tags/pelican-…

How about you come up with something yourself instead of just moving the goalpost?

Re: GPT-6 Astra on OpenRouter

#168

Post by theturtletalks »

theturtletalks wrote: post #166
dector wrote: post #100

Earlier quoted context omitted.

Btw, they had 3D pelican-on-the-bicycle easter egg in one of the promo videos: https://youtu.be/bOC3DisEOfg?t=117 so I'm pretty sure that they spent some small amount of resources to train the model to produce good svg version as well. :D

This is one of those benchmarks I don’t mind if they benchmax because it would mean models are actually good at creating svg graphics.

Look at how well it made this Xbox controller SVG:

https://www.svgviewer.dev/s/i8t1VXzQ

Re: GPT-6 Astra on OpenRouter

#169

Post by TiredOfLife »

pizza234 wrote: post #60
vb-8448 wrote: post #51

Earlier quoted context omitted.

I find curios that Astra's pelicans are basically the same (yellow sun top right corner, green bike, same bike shape, same legs style, very similar background) while in other there is more randomness.

Astra is the only model that correctly depicts occlusion of crank and leg, although interestingly, at max and medium levels (not in between).

Kimi k3 and Fable max also

Re: GPT-6 Astra on OpenRouter

#170

Post by DrProtic »

logged4upvoting wrote: post #143
upcoming-sesame wrote: post #135

Any tips on using Astra as orchestrator with Luna workers efficiently in codex?

(This applies to Sol but probably works for Astra too) I've created with Sol a skill called Low Quota Mode that intends to reduce the use of tokens usages by the frontier (intelligent model) and delegate the use of bulk reading of docs/code and implementation to a sub-agent running Luna Max. Sol is asked to supervise, read the diffs and approves the commit/pr. The skill might need some iterations while you use it, fo…

Could you shoot us the github gist?
Post reply on HN