Viewing profile — kiview

Yeah, we are on it. In the current version, things are hardcoded and implicit (we are also in experimental preview), but soon it will be configurable and explicit.

You can run the same model in production, using Docker Model Runner in Docker CE. This is assuming, the model works for your production use case :)

You can check out my blog post on publishing models to Docker Hub using Docker Model Runner :) https://www.docker.com/blog/publish-ai-models-on-docker-hub/

Unfortunately not, since the container won't have access to the Apple silicon GPU. That's why in our architecture, we have to run llama.cpp as a host process and wire it up with th…

(disclaimer: I'm leading the Docker Model Runner team at Docker) We decided to start with Apple silicon Macs, because they provide one of the worst experiences of running LLMs in a…

Better late than never ;)

(disclaimer: I'm leading the Docker Model Runner team at Docker) We are not packaging models as Docker images, since indeed that is the wrong fit and comes with all kinds of techni…

(disclaimer: I'm leading the Docker Model Runner team at Docker) It's fine to disagree of course, but we envision Docker as a tool that has a higher abstraction level than just con…

(disclaimer: I'm leading the Docker Model Runner team at Docker) You were always able to manually install Docker CE within WSL2 on Windows. But if you want to have an integrated Do…