Live data from Hacker News

Project Fetch: Phase Two

anthropic.com

21–29 of 29 posts

Re: Project Fetch: Phase Two

#23
post #15

Fast? Sure. Good maintainable code? Doubted. I think they skip the right metrics there. So that's just their AI promo.

Why does the code need to be maintainable by humans?

It needs to be maintainable by someone, and AI, left to its own devices, does not write code that is maintainable by AI or humans.

Re: Project Fetch: Phase Two

#25
post #13

Earlier quoted context omitted.

> Its not disguised. Corporate blogs exist overtly to promote the company and its work. It is. That makes the "research" heavily biased. If xAI did the same thing, with Elon Musk screaming about that it is "AGI", you would not believe them at all. Given that the work is not independent, such articles of this "research" can easily be manipulated or the results being massaged to promote the company positively. But when…

You or some other interested person could go do that experiment and publish the results. It shouldn't be hard to figure out what hardware exactly they were using and get a copy, and the prompt also doesn't have to be exactly what they used, just similar enough in spirit. See just how similar/different the outcome is.

Using their own time and money to disprove corporate propaganda. By the time you’ve disproven it they’ve already released 10 new claims. “Why are you still talking about that old news, nobody cares.”

It’s like the firehose of lies but done by corps.

Re: Project Fetch: Phase Two

#26

Earlier quoted context omitted.

You or some other interested person could go do that experiment and publish the results. It shouldn't be hard to figure out what hardware exactly they were using and get a copy, and the prompt also doesn't have to be exactly what they used, just similar enough in spirit. See just how similar/different the outcome is.

Using their own time and money to disprove corporate propaganda. By the time you’ve disproven it they’ve already released 10 new claims. “Why are you still talking about that old news, nobody cares.” It’s like the firehose of lies but done by corps.

In this case at least it isn't that hard. Opus is available to pretty much everyone, and anyone of sufficient means (I'm guessing at least 95% of active HN) can also easily afford the hardware.

And obviously this isn't something that's being iterated on rapid-fire; there have been 2 relevant publications roughly a year apart. Absolutely no firehose of anything here. As such there should be no problem for someone with enough interest to attempt to disprove the claims, and hopefully share the results regardless of their findings.

Re: Project Fetch: Phase Two

#27
post #13

Earlier quoted context omitted.

> I'm getting a bit tired of these disguised adverts. Its not disguised. Corporate blogs exist overtly to promote the company and its work. Disguised promotions where notionally independent media publish promotional pieces as news concealing that they were fed to them by party whose products they promote area thing, but this is just the most overt undisguised promotion.

> Its not disguised. Corporate blogs exist overtly to promote the company and its work. It is. That makes the "research" heavily biased. If xAI did the same thing, with Elon Musk screaming about that it is "AGI", you would not believe them at all. Given that the work is not independent, such articles of this "research" can easily be manipulated or the results being massaged to promote the company positively. But when…

> If xAI did the same thing, with Elon Musk screaming about that it is "AGI", you would not believe them at all.

I’m not saying it is trustworthy or that I believe them, I am saying the advertising isn’t even a little bit disguised when it is communicated directly from what is overtly a promotional channel for the company involved.

It's like calling the “9 out of 10 dentist prefer...” claim in a TV commercial “disguised advertising” and then coming back with arguments about how it isn't trustworthy reaearch wheb it is pointed out that TV commercials are openly ads. Yeah, its not trustworthy, but the fact that it is corporate promotional material and not a neutral third-party report is not at all concealed.

It is overt advertising communicated through a channel whose sole and open purpose is advertising for the company whose products it advertises.

Re: Project Fetch: Phase Two

#28
post #14
post #13

Earlier quoted context omitted.

> Its not disguised. Corporate blogs exist overtly to promote the company and its work. It is. That makes the "research" heavily biased. If xAI did the same thing, with Elon Musk screaming about that it is "AGI", you would not believe them at all. Given that the work is not independent, such articles of this "research" can easily be manipulated or the results being massaged to promote the company positively. But when…

You’re writing with the assumption that this is “research” in the first place. This is advertising first, “research” second.

Then Anthropic and all the other sketch-marketing AI companies should stop calling these "experiments" and start calling them "staged demos".

Re: Project Fetch: Phase Two

#29
post #2

> Preliminary trials with Claude Mythos Preview showed that it would not provide an apples-to-apples comparison with other models because of how we had set up the experiment and how the model was served. What does this mean? My guess is they couldn’t co-locate Mythos close enough to reduce latency? (I’m assuming this experiment pre-dates the export controls)

Because this was a staged demo, not an experiment. Mythos performed more poorly but they don't want to admit it. The phrase "because of how we had set up the experiment" means "we didn't have experimental controls and got a bunch of bullshit noise that we cherry picked."

At least that's my guess.

Post reply on HN