Considering the server version of llama.cpp for a commercial use case over bulkier options like vllm. But wondering if it's been battle-tested in production environments.
Ask HN: Is anybody using llama.cpp for production?
1–4 of 4 posts
Re: Ask HN: Is anybody using llama.cpp for production?
#2[dead]
Re: Ask HN: Is anybody using llama.cpp for production?
#3[dead]
Re: Ask HN: Is anybody using llama.cpp for production?
#4No, i think llama.cpp only using testing model