PyTorch Library for Running LLM on Intel CPU and GPU
1–10 of 98 posts
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#2Looking forward to reviewing!
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#3I'd be interested in seeing benchmark data. The speed seemed pretty good in those examples.
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#4Hm, no major cloud provider offers intel gpus.
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#5Hm, no major cloud provider offers intel gpus.
Lots offer Intel CPUs though...
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#6Hm, no major cloud provider offers intel gpus.
No, but for consumers they're a great offering.
16GB RAM and performance around a 4060ti or so, but for 65% of the price
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#7Hm, no major cloud provider offers intel gpus.
Intel GPUs got quite a bit of penetration in the SE Asian market, and Intel is close to releasing a new generation. In addition, Intel's allowing for GPU virtualization without additional license fees (unlike Nvidia and GRID licenses), allowing hosting operators to carve up these cards. I have a feeling we're going to see a lot more Intel offerings available.
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#8Any performance benchmark against 'llamafile'[0] or others?
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#9Hm, no major cloud provider offers intel gpus.
No, but for consumers they're a great offering. 16GB RAM and performance around a 4060ti or so, but for 65% of the price
and 65% of the software support, less I'm inclined to believe? Although having more players in the fold is definitely a good thing.
Re: PyTorch Library for Running LLM on Intel CPU and GPU
#10Are there any Intel GPUs with a lot of vRAM that someone could recommend that would work with this?