21–28 of 28 posts
It's a really beautiful project, and I’d like to ask something purely out of curiosity and with the best intentions. What’s the name of the design trend you used for your website? I really loved the website too.
it appears to be using Infima, which is Docusaurus's default CSS framework plus a standard system font stack [0] font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
Earlier quoted context omitted.
Strong disagree on this. Ollama is great for moderately technical users who aren't really programmers or proficient with the command line.
You can disagree all you want, but Ollama does not keep their llama.cpp vendored copy up to date, and also ships, via their mirror, completely random badly labeled models claiming to be the upstream real ones, often misappropriated from major community members (Unsloth, et al). When you get a model offered by Ollama's service, you have no clue what you're getting, and normal people who have no experience aren't even…
Earlier quoted context omitted.
Strong disagree on this. Ollama is great for moderately technical users who aren't really programmers or proficient with the command line.
You can disagree all you want, but Ollama does not keep their llama.cpp vendored copy up to date, and also ships, via their mirror, completely random badly labeled models claiming to be the upstream real ones, often misappropriated from major community members (Unsloth, et al). When you get a model offered by Ollama's service, you have no clue what you're getting, and normal people who have no experience aren't even…
Earlier quoted context omitted.
You can disagree all you want, but Ollama does not keep their llama.cpp vendored copy up to date, and also ships, via their mirror, completely random badly labeled models claiming to be the upstream real ones, often misappropriated from major community members (Unsloth, et al). When you get a model offered by Ollama's service, you have no clue what you're getting, and normal people who have no experience aren't even…
I thought the models were like HuggingFace, where anyone can upload a model and you choose which you pull. The Unsloth ones look like this to me, eg: https://ollama.com/secfa/DeepSeek-R1-UD-IQ1_S
When R1 first came out, for example, their official copy of it was one of the distills labeled as "R1" instead of something like "R1-qwen-distill". They've done this more than once.
Ooh this looks really neat! I'd love to see more content in the future on Structured outputs/Guided generation and sampling. Another great reference on inference-time algorithms for sampling is here: https://rentry.co/samplers
If I remember, BentoML was about MLOps, I remember trying it about a year back. Did the company pivot ?
Hi everyone. I'm one of the maintainers of this project. We're both excited and humbled to see it on Hacker News! We created this handbook to make LLM inference concepts more accessible, especially for developers building real-world LLM applications. The goal is to pull together scattered knowledge into something clear, practical, and easy to build on. We’re continuing to improve it, so feedback is very welcome! GitH…
Thanks a lot for putting this together! I have a question. In https://github.com/bentoml/llm-inference-in-production/blob/... , you have a single picture that defines TTFT and ITL. That does not match my understanding (but you guys know probably more than me): In the graphic, it looks like that the model is generating 4 tokens T0 to T3, before outputting a single output token. I'd have expected that picture for ITL (…
Thanks for putting this together! From now on I only need one link to point interested ones to learn. Only one suggestion: On page "OpenAI-compatible API" it would be great to have also a simple example for the pure REST call instead of the need to import the OpenAI package.