Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Run Hugging Face models with a single command

llmpm.co

Post reply on HN
Print view View original

1–10 of 11 posts

Re: Show HN: Run Hugging Face models with a single command

#4

Post by dataversity »

Hi folks, I’ve been working on llmpm, a CLI tool that makes open-source LLMs installable like packages. The idea came from the frustration of trying to run different models locally. Every model tends to have slightly different instructions, dependencies, or runtimes. I wanted something that worked more like npm or pip, install a model and run it immediately. With llmpm you can do things like:

llmpm install llama3

llmpm run llama3

Re: Show HN: Run Hugging Face models with a single command

#7

Post by dataversity »

Some of the features llmpm supports today:

• Install and run open-source models (almost all Hugging Face models)

• Package models with projects so environments are reproducible

• Serve models through an OpenAI compatible API

• Built-in chat UI for interacting with running models

• CLI workflow designed for developers

• Model benchmarking and rankings to help choose models Example:

llmpm serve llama3

This exposes an API endpoint similar to:

http://localhost:8080/v1/chat/completions

Post reply on HN