Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

81–90 of 251 posts

Re: Ollama Turbo

#81

Post by ekianjo »

steren wrote: post #62
jnmandal wrote: post #29

I see a lot of hate for ollama doing this kind of thing but also they remain one of the easiest to use solutions for developing and testing against a model locally. Sure, llama.cpp is the real thing, ollama is a wrapper... I would never want to use something like ollama in a production setting. But if I want to quickly get someone less technical up to speed to develop an LLM-enabled system and run qwen or w/e locally…

> I would never want to use something like ollama in a production setting. We benchmarked vLLM and Ollama on both startup time and tokens per seconds. Ollama comes at the top. We hope to be able to publish these results soon.

you need to benchmark against llama.cpp as well.

Re: Ollama Turbo

#82

Post by paxys »

polarbear67 wrote: post #13

Why does everything AI-related have to be $20? Why can't there be tiers? OpenAI setting the standard of $20/m for every AI application is one of the worst things to ever happen.

https://openai.com/chatgpt/pricing/ - $0 / $20 / $200 / $25 (team) / custom enterprise pricing / on-demand API pricing

https://www.anthropic.com/pricing - $0 / $17 (if billed annually) / $20 (if billed monthly) / $100 / $25 (team) / custom enterprise pricing / on-demand API pricing

Sounds like tiers to me.

Re: Ollama Turbo

#83

Post by pagekicker »

liuliu wrote: post #33

Any more information on "Privacy first"? It seems pretty thin if just not retaining data. For Draw Things provided "Cloud Compute", we don't retain any data too (everything is done in RAM per request). But that is still unsatisfactory personally. We will soon add "privacy pass" support, but still not to the satisfactory. Transparency log that can be attested on the hardware would be nice (since we run our open-source…

I see no privacy advantage to working with Ollama, which can sell your data or have it subpoenaed just like anyone else.

Re: Ollama Turbo

#86

Post by coderatlarge »

extr wrote: post #10

Nice release. Part of the problem right now with OSS models (at least for enterprise users) is the diversity of offerings in terms of: - Speed - Cost - Reliability - Feature Parity (eg: context caching) - Performance (What quant level is being used...really?) - Host region/data privacy guarantees - LTS And that's not even including the decision of what model you want to use! Realistically if you want to use an OSS mo…

true but ignores handing over all your prompt traffic without any real legal protections as sama has pointed out:

[1] https://californiarecorder.com/sam-altman-requires-ai-privil...

Re: Ollama Turbo

#87

Post by paxys »

A subscription fee for API usage is definitely an interesting offering, though the actual value will depend on usage limits (which are kept hidden).

Re: Ollama Turbo

#88

Post by _--__--__ »

jacekm wrote: post #63

What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?

Groq seems to do okay with a similar service but I think their pricing is probably better.

Re: Ollama Turbo

#89

Post by mchiang »

paxys wrote: post #87

A subscription fee for API usage is definitely an interesting offering, though the actual value will depend on usage limits (which are kept hidden).

we are learning the usage patterns to be able to price this more properly.
Post reply on HN