Long $amd?
HipKittens: Fast and furious AMD kernels
91–94 of 94 posts
Re: HipKittens: Fast and furious AMD kernels
#92Earlier quoted context omitted.
I got one of those running whisper yesterday, hopeful the bigger llms will run shortly. You'd need rocm 7 which seems to be much better than 6.4 was.
Is the performance decent? I'm looking at using it with 30b coding models with a local agent framework like goose to see if we can do this locally as developers instead of risking leaking code to the big models.
The chip in general is fast, it builds llvm in ~12m or so. Whisper on it is at least real time but I only ran the stream binary before sending the box away to SC25. I'm expecting it to need some work to exploit the zero copy the APU permits. So it probably will be fast but isn't just yet, at least on my toolchain.
Re: HipKittens: Fast and furious AMD kernels
#93Earlier quoted context omitted.
Will it catch up or will it forever chase nvidia's tail ? I'm betting on the latter unless another AI winter happens. And contrary to anti-generative AI social media talking points, the literature suggests The Red Queen's race is continuing apace IMO. Nvidia remains undefeated at responding to hardware threats with hardware diving catches to this day. What scenario prevents them from yet another one of their diving c…
Nvidia remains undefeated at responding to hardware threats with hardware diving catches to this day. What scenario prevents them from yet another one of their diving catches? The fact that they make roughly the same hardware as AMD for the last 2 decades, and even today. There was no diving catch, AMD just ignored what the hardware was capable of and didn't reinforce OpenCL. There was literally no diving catch. For…
Except no, AMD 100% played follow the leader with technology like CUDA, NVLink, and tensor cores.
Even paying paying someone in academia to get s** to work on their hardware is yet another example of follow the leader.
What exactly do you think is coming? I think the biggest threat is one or more Chinese companies catching up on both hardware and ecosystem in the next half decade or so myself, mostly because of the state level support for making that so. But I absolutely don't expect an x86_64 moment for GPUs here given past results and the current bias against software in AMD's HW culture. Convince me otherwise.
Re: HipKittens: Fast and furious AMD kernels
#94Earlier quoted context omitted.
> We haven't had a fully funded bonus in the past 4+ years. This is WILD to hear considering how well it appears AMD is executing from the outside.
> considering how well it appears AMD is executing from the outside. The party line is that the stock price is up because the market expects us to perform well in the future, and we won't get a bonus until we actually perform well.
OOF.