101–102 of 102 posts
Earlier quoted context omitted.
That's about the same price as a 3090 and it's also 24GB. Are they faster at inference?
it is not, at least in llama.cpp/llamafile https://benchmarks.andromeda.computer/compare
People use "Docker-based" all the time but what they mean is that they ship $SOFTWARE in a Docker image. "Docker-based" reads, to me, as if you were doing Inference on AMD cards with Docker somehow, which doesn't make sense.
Yeah, they're using docker to wrap up the software packages, which is what Docker is used for. I don't understand why that confuses you or what you think Docker is otherwise used for.