51–60 of 70 posts
Earlier quoted context omitted.
I found that Lex interview enlightening for two reasons. Firstly, Keller is a profoundly talented and interesting guy. And secondly, Lex is a truly painful and incompetent interviewer; virtually every time Lex spoke I begged my screen for him to stay quiet and let Keller speak.
Lex is also absolutely boring to listen to. His tone of voice puts you to sleep.
To address the article, this will be very interesting because in the market Tensortorrent is targetting the real challenge is software support for PyTorch, TensorFlow, distributed training, etc... and less about the hardware itself. It's not like AMD GPUs can't do matrix multiplications at competitive TFLOPS but instead that if I wanted to use them to run a model I'd have to work through several workarounds to get si…
Earlier quoted context omitted.
I was at Intel when Jim Keller came. He was one of the most inspiring technical leaders I have ever encountered. He obviously had very strong engineering knowledge as well. When he came in, he took some time to really understanding our process and challenges. Then he created 5 areas that Intel needed to improve. He said “we are going to fix these, or burn down the place trying”. And these weren’t vague goals like “ex…
Moar. Maybe you have some more stories or insights from him. I'm a shameless fanboy. I also listened to I think every podcast (not many of them unfortunately, Lex ones seem the best as expected). This guy oozes wisdom.
"This is how were going to validate this chip:
1. go to the beach
2. open your laptop
3. run the chip in emulation through boot
4. tape-in! (send to the fab)
5. take the rest of the year off at the beach "
His point was that we need a simple focus on proving the chip works and then shipping it. If any validation milestone or objective is not directly connected to that, it's not worth doing. We don't need all this extraneous validation and multiple layers of rechecking. Just emulate the thing and then your done.
In his interview with Jordan Petersen he talked about how some people spend all their time "admiring the problem"; Trying to understand and categorize it. Keller didn't want to spend any more time talking about the problem then needed. He was focused on finding good solutions (and it had to be a good solution), and then executing them.
I recommend Keller's appearances on the Lex Fridman podcast, and also the chat with his (Keller's) brother-in-law Jordan Peterson.
Thumbs up for courage to even mention Jordan Peterson. As he is extremely unpopular on HN.
Earlier quoted context omitted.
Sure, but Tesla doesn’t sell that many vehicles. They only went over a million last year (will be closer to 2 million this year it would appear.
Yes, they were planning ahead. They knew they would sell more vehicles in the future, and thus the effort was worth it. Not just for V1, but the following version as well. Once you have an internal team, you can vertically integrate and costume the solution to your needs exactly rather then buying a generic part from a supplier. Things like OTA updates are just easier if you control the software and hardware stack yo…
I recommend Keller's appearances on the Lex Fridman podcast, and also the chat with his (Keller's) brother-in-law Jordan Peterson.
I found that Lex interview enlightening for two reasons. Firstly, Keller is a profoundly talented and interesting guy. And secondly, Lex is a truly painful and incompetent interviewer; virtually every time Lex spoke I begged my screen for him to stay quiet and let Keller speak.
The man is a legend! He made an amazing go work across Apple, AMD and recently at Intel.
To address the article, this will be very interesting because in the market Tensortorrent is targetting the real challenge is software support for PyTorch, TensorFlow, distributed training, etc... and less about the hardware itself. It's not like AMD GPUs can't do matrix multiplications at competitive TFLOPS but instead that if I wanted to use them to run a model I'd have to work through several workarounds to get si…
Does AMD have similar tensor core stuff?
And they're blurring the line by adding more AI-focused acceleration instructions to their GPUs (maybe focused on their CDNA line rather than the consumer RDNA, but they're expanding their capabilities on both).
There seems to be a bit of a philosophy difference between NVidia and AMD acceleration - NVidia add new specialised "cores", while AMD adds those similar capabilities as accelerated instructions to their shader units.
Though this might just be marketing - I don't think we've got an open reference for the NVidia shader ISA like we do with AMD so difficult to compare, but the capabilities and integration in shaders of many of these acceleration paths seems to imply they might also be implemented as shader instructions, which stretches the definition of a separate "core" to me, and so might just be a marketing difference rather than anything technical. Same with things like the BVH acceleration used in ray tracing - AMD added shader instructions, while NVidia talks loudly about "RT Cores".
To address the article, this will be very interesting because in the market Tensortorrent is targetting the real challenge is software support for PyTorch, TensorFlow, distributed training, etc... and less about the hardware itself. It's not like AMD GPUs can't do matrix multiplications at competitive TFLOPS but instead that if I wanted to use them to run a model I'd have to work through several workarounds to get si…
Does AMD have similar tensor core stuff?
Earlier quoted context omitted.
I found that Lex interview enlightening for two reasons. Firstly, Keller is a profoundly talented and interesting guy. And secondly, Lex is a truly painful and incompetent interviewer; virtually every time Lex spoke I begged my screen for him to stay quiet and let Keller speak.
His guests are often so interesting that it's worth persevering through Lex's un-profound musings. (Every time he uses the word 'beautiful' or 'love' it reminds me of a pretentious teenager who lacks life experience)
He is ok. Not that bad.