Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon launches Trainium3

techcrunch.com

Post reply on HN
Print view View original

51–60 of 75 posts

Re: Amazon launches Trainium3

#51

Post by klysm »

mountainriver wrote: post #49
ZeroCool2u wrote: post #10

I've had to repeatedly tell our AWS account reps that we're not even a little interested in the Trainium or Inferentia instances unless they have a provably reliable track record of working with the standard libraries we have to use like Transformers and PyTorch. I know they claim they work, but that's only on their happy path with their very specific AMI's and the nightmare that is the neuron SDK. You try to do any…

Agree, Google put a ton of work into making TPUs usable with the ecosystem. Given Amazon’s track record I can’t imagine they would ever do that.

There might be enough market pressure right now to make them think about it, but the stock price went up enough from just announcing it so whatever

Re: Amazon launches Trainium3

#52

Post by [deleted] »

ecshafer wrote: post #15
ZeroCool2u wrote: post #10

I've had to repeatedly tell our AWS account reps that we're not even a little interested in the Trainium or Inferentia instances unless they have a provably reliable track record of working with the standard libraries we have to use like Transformers and PyTorch. I know they claim they work, but that's only on their happy path with their very specific AMI's and the nightmare that is the neuron SDK. You try to do any…

IMO AWS once you get off the core services is full of beta services. S3, Dynamo, Lambda, ECS, etc are all solid. But there are a lot of services they have that have some big rough patches.

[deleted]

Re: Amazon launches Trainium3

#53

Post by raw_anon_1111 »

jeffparsons wrote: post #41
ecshafer wrote: post #15

Earlier quoted context omitted.

IMO AWS once you get off the core services is full of beta services. S3, Dynamo, Lambda, ECS, etc are all solid. But there are a lot of services they have that have some big rough patches.

RDS, Route53, and Elasticache are decent, too. But yes, I've also been bitten badly in the distant past by attempting to rely on their higher-level services. I guess some things don't change. I wonder if the difference is stuff they dogfood versus stuff they don't?

Amazon Connect is a solid higher level offering. But only because it is a productized version of Amazon Retail’s call center

Re: Amazon launches Trainium3

#55

Post by trebligdivad »

Symmetry wrote: post #17
aaa_aaa wrote: post #3

Interesting that in the article, they do not say what the chip actually does. Not even once.

A bunch of 128x128 systolic arrays at its heart. More details: https://newsletter.semianalysis.com/p/amazons-ai-self-suffic...

Nice article; I hate to think what the DC bus bars look like! ~50v at ~25kW/rack; 500A bus bars - I guess split, but still!

Re: Amazon launches Trainium3

#56

Post by phantasmish »

jeffparsons wrote: post #41
ecshafer wrote: post #15

Earlier quoted context omitted.

IMO AWS once you get off the core services is full of beta services. S3, Dynamo, Lambda, ECS, etc are all solid. But there are a lot of services they have that have some big rough patches.

RDS, Route53, and Elasticache are decent, too. But yes, I've also been bitten badly in the distant past by attempting to rely on their higher-level services. I guess some things don't change. I wonder if the difference is stuff they dogfood versus stuff they don't?

I once used one of their services (I forget which, but I think it was there serverless product) that “supported” Java.

… but the official command line tools had show-stopper bugs if you were deploying Java to this service, that’d been known for months, and some features couldn’t be used in Java, and the docs were only like 20% complete.

But this work-in-progress alpha (not even beta quality because it couldn’t plausibly be considered feature complete) counted as “supported” alongside other languages that were actually supported.

(This was a few years ago and this particular thing might be a lot better now, but it shows how little you can trust their marketing pages and GUI AWS dashboards)

Re: Amazon launches Trainium3

#57

Post by hustwindmaple »

cmiles8 wrote: post #16
giancarlostoro wrote: post #14

Earlier quoted context omitted.

Not just AWS, looks like Anthropic uses it heavily as well. I assume they get plenty of handholding from Amazon though. I'm surprised any cloud provider does not invest drastically more into their SDK and tooling, nobody will use your cloud if they literally cannot.

Well AWS says Anthropic uses it but Anthropic isn’t exactly jumping up and down telling everyone how awesome it is, which tells you everything you need to know. If Anthropic walked out on stage today and said how amazing it was and how they’re using it the announcement would have a lot more weight. Instead… crickets from Anthropic in the keynote

I met a AWS engineer a couple of weeks ago and he said Trainium is actually being used for Anthropic model inference, not for training. Inferentia is basically defected Trainiums chips that nobody wants to use.

Re: Amazon launches Trainium3

#58

Post by smallmancontrov »

ozten wrote: post #45
jeffparsons wrote: post #41

Earlier quoted context omitted.

RDS, Route53, and Elasticache are decent, too. But yes, I've also been bitten badly in the distant past by attempting to rely on their higher-level services. I guess some things don't change. I wonder if the difference is stuff they dogfood versus stuff they don't?

A big problem for a when three AWS teams launch the same thing. Lowers confidence in dogfooding the “right” one.

Or when your AWS account rep is schmoozing your boss trying to persuade them to use something that is officially deprecated, lol.

Re: Amazon launches Trainium3

#59

Post by weird-eye-issue »

ecshafer wrote: post #15
ZeroCool2u wrote: post #10

I've had to repeatedly tell our AWS account reps that we're not even a little interested in the Trainium or Inferentia instances unless they have a provably reliable track record of working with the standard libraries we have to use like Transformers and PyTorch. I know they claim they work, but that's only on their happy path with their very specific AMI's and the nightmare that is the neuron SDK. You try to do any…

IMO AWS once you get off the core services is full of beta services. S3, Dynamo, Lambda, ECS, etc are all solid. But there are a lot of services they have that have some big rough patches.

True with Cloudflare too. Just stick with Workers, R2, Durable Objects, etc...
Post reply on HN