Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Code and Muse Spark 1.2

research.meta.ai

Post reply on HN
Print view View original

221–230 of 265 posts

Re: Muse Code and Muse Spark 1.2

#221

Post by lacker »

villish wrote: post #94
lacker wrote: post #56

Earlier quoted context omitted.

My conclusion is the opposite. If benchmarks were meaningless, surely Meta would be able to find some benchmark that shows they are better than Sol and Fable. The fact that they can't do that tells me that benchmarks still do mean something.

Muse 1.1 performed relatively well according to benchmarks, putting it within spitting distance of the premier models. However, based on the results I got from it and the review videos I watched, it wasn’t even close. Opus 5 is incredible at making games. Almost like a generation better than other models from my experience. You won't see that if you just look at the popular benchmarks.. You have to test each model on…

Yeah, totally. But... the problem is that I don't have time to test every single model that comes out. So I rely on reports like this to decide, should I even bother testing out Muse?

Re: Muse Code and Muse Spark 1.2

#223

Post by deaux »

minimaxir wrote: post #11
wmf wrote: post #4

Earlier quoted context omitted.

Clearly they're positioning it as a mid model.

Which is in itself a bit weird as mid models nowadays are a golden mean fallacy. Terra is much less popular than both Luna (cost-sensitive) and Sol (performance-sensitive). Claude Sonnet is a weird exception to the mid models because Anthropic doesn't do much with Haiku and Opus is too big.

It's a little of the opposite to what you're saying. By performance you seem to mean only "intelligence" i.e. pass rate. But there's a 3rd factor, time to task completion. So it's three-dimensional rather than two. And on that spectrum there are areas where Terra is optimal.

Sonnet on the other hand never is, it's far from the best pick anywhere on the spectrum.

Re: Muse Code and Muse Spark 1.2

#224

Post by qlte »

embedding-shape wrote: post #211
ersiees wrote: post #180

Meta, please go back to publishing weights. Your models aren't top-tier, this wouldn't hurt you at all at your current position in the rankings. They wouldn't have the best open-weight models, but the best western open-weight models.

> the best western open-weight models If they were sitting on that and could release those, they for sure would. AFAIK, they still haven't released Llama 4 Behemoth, so kind of feels like it's evident what has happened, they aren't able to compete anymore.

Would they?

To me it feels like we’re past the dreamy early days of the AI boom (in the US) and now investors want to see the tech monopolies actually start building the new cash printing machines they’ve been promising with the hundreds of billions in capex burned over the last few years.

Maybe a Gemma sized model that couldn’t self-cannibalize but releasing a top open Kimi K3 sized model seems like it could start to wear on Meta investors’ patience without signaling how it fits into a new profitable business line.

Re: Muse Code and Muse Spark 1.2

#225

Post by deaux »

greyb wrote: post #37
giancarlostoro wrote: post #21

Will someone at Meta for the love of God make it so none of this stuff goes through Facebook.com? You want customers but most corporate firewalls block social media. Also, a lot of devs do not want their work stuff tied up to their facebook account. For the love of all things show the IG / FB logins as optional and do email as primary. I am not a fan of Meta but I do cheer for any competitors against OpenAI and Anthr…

I honestly think they're kinda banking on piggybacking off of Facebook account integrity systems to avoid the problems that other LLM providers are facing in trying to prevent mass free trial signups for token relays and so forth. It's not a good system obviously. Google did this as well for Gemini-CLI, but forced it to be linked to personal Google accounts (which caused a great deal of onboarding friction).

They're spending $100 billion on this stuff and they're worried about free trial signups? Genuinely my opinion couldn't be lower about Meta's AI strategy but even I don't believe they're that stupid. They'd 100x rather have that problem than nobody using it.

Re: Muse Code and Muse Spark 1.2

#226

Post by deaux »

giancarlostoro wrote: post #72
hahahaa wrote: post #36

Earlier quoted context omitted.

China says hi.

Not in my case, I don't see any of my employers (past or current) trusting a country like China with their data.

Hello Chinese models run by US providers.

If they trust Chinese models run by US providers less than Meta then your employers are flat-earther level loonies.

Re: Muse Code and Muse Spark 1.2

#227

Post by 6thbit »

discordance wrote: post #151
tristanj wrote: post #42

Meta is offering a 10x discount on input ($0.10 vs. $1.25/Mtok) and 20x discount on output ($0.20 vs. $4.25/Mtok) if you opt in to let them train on your data. https://developer.meta.com/ai/models/muse-spark/

Call me childish but it was worth a shot... "Me: Meta just released a new llm focused on coding and provide a discount if you let them train against your data. I don't like Meta and I think they are a net negative in our world. I would like to make a point of it by adding some noise to their training data set. Think of this as a protest and perhaps a bit of a marketing campaign to remind Meta employees (and others) o…

This is likely an legitimate risk, not from you exactly, but if there's unethical competitors with unlimited budgets, their training could indeed be poisoned.

Not sure if anyone would bother.

Re: Muse Code and Muse Spark 1.2

#228

Post by deaux »

Stunning to see so many comments here very eager to try it out and cheering for it. Meta is not one bit better (and in reality worse) than Larry Elisson or Palantir but I doubt we'd be seeing this if it'd be Oracle Code or Palantir Code.

Though I guess that there's such a big population of (ex-)Meta employees here that this might explain it.

Re: Muse Code and Muse Spark 1.2

#229

Post by 6thbit »

nthypes wrote: post #217

It's useless. Tried with OpenCode + OpenRouter and it couldn't complete an simple task. It stuck using grep/search tools. I think Muse Spark was so heavily RL'd on the Meta harness that it make it useless or very token inneficient to use in other harness like Opencode.

is this their way to push their harness?

people interested in the discounted -contributor model would increase their harness beta testers. But it'd be a terrible strategy to capture the better paying customer base through openrouter and similar...

Re: Muse Code and Muse Spark 1.2

#230

Post by 6thbit »

tristanj wrote: post #42

Meta is offering a 10x discount on input ($0.10 vs. $1.25/Mtok) and 20x discount on output ($0.20 vs. $4.25/Mtok) if you opt in to let them train on your data. https://developer.meta.com/ai/models/muse-spark/

So what Meta believes fair for "paying" for your data is $0.1/Mtok plus the opportunity cost of $3/Mtok in output?
Post reply on HN