Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advice to Tenstorrent

github.com

Post reply on HN
Print view View original

41–50 of 67 posts

Re: Advice to Tenstorrent

#41

Post by sothatsit »

throwaway314155 wrote: post #12

Guy would really benefit from learning some manners. Just comes across as painfully toxic no matter how correct he is. edit: For what it's worth, if you can't see that this language is rude or think it is somehow acceptable for people of a certain caliber to talk this way - you're also probably toxic.

> edit: For what it's worth, if you can't see that this language is rude or think it is somehow acceptable for people of a certain caliber to talk this way - you're also probably toxic.

Personally, I prefer direct language if it gets to the root of the problem quicker. It's more pragmatic. You just have to pick your audience, because some people get offended by it. But the most productive discussions I've had have been arguments where you can both quickly find the holes in each other's positions, and then move forwards from there. As long as no one is taking it personally, this is very effective.

OTOH, I've been in many meetings where people talk around a problem for an hour, never reaching the conflict about what their disagreement actually is. To me, that is much more frustrating than someone risking offending someone by being direct. But it really depends upon the people you work with and the team you have.

Re: Advice to Tenstorrent

#42

Post by Aurornis »

latchkey wrote: post #19
imtringued wrote: post #10

Earlier quoted context omitted.

I think he is right about AMD but completely misses the mark when it comes to tenstorrent. He is ranting about exponential linear unit (elu), which hardly seems to be something that could possibly hold an AI company back. If the hardware is running and training models reliably, then it's just a matter of pricing to stay competitive. Poor optimization cuts your margins, but the incentives are aligned. With AMD the exp…

People refuse to believe it, but the AMD experience is getting better every day by leaps and bounds. Over the last few months, there is a brand new focus on improving the software. There is still a long way to go, but the company is absolutely trying to save itself.

geohot got a lot of press and attention by coming out aggressively at AMD during a moment when their software really was weak.

The short-term payoff of that drama created a long-term problem where the only way they could look good was by outrunning the progress of the engineers who were inside the company, well funded, and already familiar with everything. It was an impossible goal from the start but he made it even more impossible by attacking AMD. AMD was smart to basically ignore them and wait for him to give up, as opposed to inviting that drama to crossover in-house or split their user base.

Re: Advice to Tenstorrent

#43

Post by Aurornis »

roenxi wrote: post #23
htrp wrote: post #7

Has geohot done anything since the original iphone jailbreak? The ventures he has started (I can think of tinygrad and comma ai) all seem like half finished tech demos.

How many times does he have to make international news before he is qualified to let off steam in a github README? He owns a company that works in this space and his opinions usually offer insights into the ML hardware world. Although he is a little bombastic and his grammar is vulnerable to criticism. His AMD rants were a valuable warning about the quality of their hardware. I wish he'd done that maybe 10 years ago…

> His AMD rants were a valuable warning about the quality of their hardware.

The rants weren’t breaking news to anyone who was familiar with PyTorch or adjacent communities. He seized upon a weak moment for AMD to try to launch his own company. Unfortunately he launched his effort with an attack on the company he was effectively trying to partner with, making the entire venture DOA.

It’s too bad, too, because it would have been interesting to see if anything could have been accomplished with a more friendly offer of cooperation. He’s obviously talented as a developer, but effectively going on the attack for the company that forms the foundation of the business you’re trying to build is obviously not going to end well.

Re: Advice to Tenstorrent

#44

Post by latchkey »

Aurornis wrote: post #42
latchkey wrote: post #19

Earlier quoted context omitted.

People refuse to believe it, but the AMD experience is getting better every day by leaps and bounds. Over the last few months, there is a brand new focus on improving the software. There is still a long way to go, but the company is absolutely trying to save itself.

geohot got a lot of press and attention by coming out aggressively at AMD during a moment when their software really was weak. The short-term payoff of that drama created a long-term problem where the only way they could look good was by outrunning the progress of the engineers who were inside the company, well funded, and already familiar with everything. It was an impossible goal from the start but he made it even…

They didn't ignore him and he didn't give up. They gave him two boxes and he added MI300x support to tinygrad.

Re: Advice to Tenstorrent

#45

Post by georgehotz »

latchkey wrote: post #44
Aurornis wrote: post #42

Earlier quoted context omitted.

geohot got a lot of press and attention by coming out aggressively at AMD during a moment when their software really was weak. The short-term payoff of that drama created a long-term problem where the only way they could look good was by outrunning the progress of the engineers who were inside the company, well funded, and already familiar with everything. It was an impossible goal from the start but he made it even…

They didn't ignore him and he didn't give up. They gave him two boxes and he added MI300x support to tinygrad.

We got the MI300X box on MLPerf too, and every MLPerf from here on general tinygrad improvements should bring down the times. We're still quite focused on AMD.

Like it's strange people think I give up on things, I think they listen to the media too much. This is a 2+ year long project that I've worked on almost every day. https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...

Re: Advice to Tenstorrent

#46

Post by coolThingsFirst »

roenxi wrote: post #23
htrp wrote: post #7

Has geohot done anything since the original iphone jailbreak? The ventures he has started (I can think of tinygrad and comma ai) all seem like half finished tech demos.

How many times does he have to make international news before he is qualified to let off steam in a github README? He owns a company that works in this space and his opinions usually offer insights into the ML hardware world. Although he is a little bombastic and his grammar is vulnerable to criticism. His AMD rants were a valuable warning about the quality of their hardware. I wish he'd done that maybe 10 years ago…

> his grammar is vulnerable to criticism

He definitely writes in a below-HS level.

Re: Advice to Tenstorrent

#47

Post by roenxi »

Aurornis wrote: post #43
roenxi wrote: post #23

Earlier quoted context omitted.

How many times does he have to make international news before he is qualified to let off steam in a github README? He owns a company that works in this space and his opinions usually offer insights into the ML hardware world. Although he is a little bombastic and his grammar is vulnerable to criticism. His AMD rants were a valuable warning about the quality of their hardware. I wish he'd done that maybe 10 years ago…

> His AMD rants were a valuable warning about the quality of their hardware. The rants weren’t breaking news to anyone who was familiar with PyTorch or adjacent communities. He seized upon a weak moment for AMD to try to launch his own company. Unfortunately he launched his effort with an attack on the company he was effectively trying to partner with, making the entire venture DOA. It’s too bad, too, because it woul…

> The rants weren’t breaking news to anyone who was familiar with PyTorch or adjacent communities.

How many people in the PyTorch and adjacent communities are trying to port the stack to AMD cards? I'd guess less than 100. That is a pretty small community. They don't do that much in the way of publicity (and, case in point, George does and he attracts a certain number of haters).

George is probably the first public reference I saw who wasn't pointing at CUDA as the problem but tagging kernel & hardware bugs on AMD cards as the blocker. Those are really different things; CUDA isn't that complicated to implement as a half-baked thing. I can implement naive matrix multiplication if I need to and it'd be a fun month of reading academic papers to learn better techniques. But that does nothing to work around hardware and firmware bugs which are an order of magnitude harder to deal with and not something I want to deal with.

I'm happy to buy in to bad software libraries. I don't want a bar of hardware bugs.

Re: Advice to Tenstorrent

#48

Post by liaopeiyuan »

samsartor wrote: post #13

I'm doing my PhD in ML shit. Before that I was a systems programming guy, lots of C++, bit of CUDA, big fan of Rust. On the side I'm obsessed with RISC-V. Own a couple of boards. I made a stupid little cuda-like-compiler on top of the RISC-V vector extensions, just for fun. What I'm saying is, tensorrent couldn't find a more excitable third-party developer if they grew one in a lab. And you know what? I can't make he…

This is my sentiment too after trying to get a Blackhole to run a recent VLM (like Pixtral) over the weekend. Not just unit tests, but actual training loops. I write a lot of JAX in my day job to train large models but I used to do a bit of ML compiler development, which I guess also puts me in the dumb people crowd. I'm equally impressed by how smooth the lower-level setup is and frustrated by how little progress I was able to make towards the seemingly last mile of "just rewrite the code a little bit more bro I just need to get rid of this one hlo op because it's not supported."

I don't think anyone is seriously training an NN on TT hardware at the moment and I think that's an issue. I think tinygrad works not only because geohot is one hell of an engineer but also because comma dogfoods it. TT's engineers are absolutely brilliant (from reading their commits) but I think they are stretched too thin. Bounties are not gonna work - you can't expect an outsider with no internal access/bandwidth/knowledge to suddenly make e.g. Mixtral work as the issue spans at least across tt-xla/tt-mlir. And to agree with ^ training is a kind of artifact where good CX can only be derived from strong leadership and a leaner view of the stack. NVIDIA accumulated that over the decades and the rest are trying to catch up by aggressive hiring (not to say that hiring is necessary). e.g. Annapurna has a presence on the CMU campus when I was there and has the Anthropic team to test it out.

I'm an incredibly excited third-party developer as I think the pitch appeals a lot to grad students (who do model research) who need to run small experiments within the 13B range and reasonably scale them up to draw the first half of the scaling curve.

I lose too much productivity to abstractions and incomplete e2e support in TT's current shape. I'd love to give it another go in 6 months.

Re: Advice to Tenstorrent

#49

Post by yallpendantools »

sothatsit wrote: post #41
throwaway314155 wrote: post #12

Guy would really benefit from learning some manners. Just comes across as painfully toxic no matter how correct he is. edit: For what it's worth, if you can't see that this language is rude or think it is somehow acceptable for people of a certain caliber to talk this way - you're also probably toxic.

> edit: For what it's worth, if you can't see that this language is rude or think it is somehow acceptable for people of a certain caliber to talk this way - you're also probably toxic. Personally, I prefer direct language if it gets to the root of the problem quicker. It's more pragmatic. You just have to pick your audience, because some people get offended by it. But the most productive discussions I've had have be…

There's a line between rude/toxic and direct/pragmatic. The whole world at large would be much better off if we made that distinction. Also, that both tones can be found in the same piece of prose; we don't have to label a whole piece as one or the other.

> If you want a dataflow graph compiler, build a dataflow graph compiler. > This is not 6 layers of abstraction, it's 3 (and only 2 you have to build).

This is direct and pragmatic. It states the writer's justified true beliefs and opinions as plainly as possible.

> Plz bro one more stack this stack will be good i promise > bro bro bro plz one more make it all back one trade type beat

This is just toxic. The writer is making assumptions about other people's position that he does not (and probably could not) substantiate.

Ironically, sprinkling in toxic comments and back-handed insults in any piece has the effect of making said piece less direct and pragmatic.

Re: Advice to Tenstorrent

#50

Post by [deleted] »

yallpendantools wrote: post #49
sothatsit wrote: post #41

Earlier quoted context omitted.

> edit: For what it's worth, if you can't see that this language is rude or think it is somehow acceptable for people of a certain caliber to talk this way - you're also probably toxic. Personally, I prefer direct language if it gets to the root of the problem quicker. It's more pragmatic. You just have to pick your audience, because some people get offended by it. But the most productive discussions I've had have be…

There's a line between rude/toxic and direct/pragmatic. The whole world at large would be much better off if we made that distinction. Also, that both tones can be found in the same piece of prose; we don't have to label a whole piece as one or the other. > If you want a dataflow graph compiler, build a dataflow graph compiler. > This is not 6 layers of abstraction, it's 3 (and only 2 you have to build). This is dire…

[deleted]
Post reply on HN