Project Fetch: Phase Two
anthropic.com
Project Fetch: Phase Two
1–10 of 29 posts
Re: Project Fetch: Phase Two
#2> Preliminary trials with Claude Mythos Preview showed that it would not provide an apples-to-apples comparison with other models because of how we had set up the experiment and how the model was served.
What does this mean? My guess is they couldn’t co-locate Mythos close enough to reduce latency?
(I’m assuming this experiment pre-dates the export controls)
Re: Project Fetch: Phase Two
#3> Preliminary trials with Claude Mythos Preview showed that it would not provide an apples-to-apples comparison with other models because of how we had set up the experiment and how the model was served. What does this mean? My guess is they couldn’t co-locate Mythos close enough to reduce latency? (I’m assuming this experiment pre-dates the export controls)
> My guess is they couldn’t co-locate Mythos close enough to reduce latency?
I doubt network latency is the reason. Even when connecting from literally across the world network latency is lost in the noise of overall response latency of even fast models.
The overall response latency of the model very well could have been the difference, though. AFAIK Mythos is structured to do relatively slow "deep thinking".
Re: Project Fetch: Phase Two
#4stop trying to make fetch happen
Re: Project Fetch: Phase Two
#5This mostly reads as a comparison between Opus 4.7 and 4.1 it would be more interesting if they reran the experiment against a team of humans with 4.7 and see how much the humans still improve the results today.
Re: Project Fetch: Phase Two
#6[flagged]
Re: Project Fetch: Phase Two
#7> Preliminary trials with Claude Mythos Preview showed that it would not provide an apples-to-apples comparison with other models because of how we had set up the experiment and how the model was served. What does this mean? My guess is they couldn’t co-locate Mythos close enough to reduce latency? (I’m assuming this experiment pre-dates the export controls)
> My guess is they couldn’t co-locate Mythos close enough to reduce latency? I doubt network latency is the reason. Even when connecting from literally across the world network latency is lost in the noise of overall response latency of even fast models. The overall response latency of the model very well could have been the difference, though. AFAIK Mythos is structured to do relatively slow "deep thinking".
Depending on the timeline, it could be that they're not allowed to access Mythos because of something like non-US citizens on the team or the lack of some way for them to meet the constraint DOD has them under.
Re: Project Fetch: Phase Two
#8Do you want Terminators? Because this is how you get Terminators.
Re: Project Fetch: Phase Two
#9Earlier quoted context omitted.
> My guess is they couldn’t co-locate Mythos close enough to reduce latency? I doubt network latency is the reason. Even when connecting from literally across the world network latency is lost in the noise of overall response latency of even fast models. The overall response latency of the model very well could have been the difference, though. AFAIK Mythos is structured to do relatively slow "deep thinking".
Depending on the timeline, it could be that they're not allowed to access Mythos because of something like non-US citizens on the team or the lack of some way for them to meet the constraint DOD has them under.
I strongly suspect if that was the case they would have just directly mentioned that Mythos couldn't be used because of that reason, it would be less confusing and less suspect messaging than saying it wasn't an "apples-to-apples comparsion".
Re: Project Fetch: Phase Two
#10I'm getting a bit tired of these disguised adverts.
Here's how non robotics engineers used AI to do a short robot integration task faster than other non robotics engineers without AI.
Where "better" mostly means faster, and who knows what happens on longer horizons, with actual robotics experts, robustness requirements, or tasks where the hard part is control rather than API spelunking.