Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

51–60 of 251 posts

Re: Ollama Turbo

#51

Post by bangaladore »

bangaladore wrote: post #22
satellite2 wrote: post #12

"All hardware is located in the United States." If I use local/OSS models it's specifically to avoid running in a country with no data protection laws. It's a big close miss here.

I think what matters more here is "All hardware is located outside of China". Located in the US means little because that's not good enough for many regulated industries even within the US. All things considered though, Europe is getting confusing. They have GDPR but now pushing to backdoor encryption within the EU? [1] At least there isn't a strong movement in the US trying to outlaw E2E encryption. [1] https://www.…

Maybe I hit a nerve with the EU part? I thought it was a fair observation, but I'm open to being corrected if there's more nuance I missed.

Re: Ollama Turbo

#52

Post by criddell »

mchiang wrote: post #32
recursivegirth wrote: post #26

Earlier quoted context omitted.

Ollama, run by Facebook. Small company, huh.

Ollama is not run by Facebook. We are a small team building our dreams.

I thought it was a Meta company because the name is so close to Llama which is a Meta product.

I looked up the Ollama trademark and was surprised to see it's a Canadian company.

Re: Ollama Turbo

#53

Post by colesantiago »

No matter if a project is "open source" as long as they announce that they have raised millions amount of dollars from investors...

It is completely compromised, especially if it is an AI company.

How do you think ollama was able to provide the open source AI models to everyone for free?

I am pretty sure ollama was losing money on every pull of those images from their infrastructure.

Those that are now angry at ollama charging money or not focusing on privacy should have been angry when they raised money from investors.

Re: Ollama Turbo

#54

Post by llmtosser »

Distractions like this probably the reason they still, over a year now, do not support sharded GGUF.

https://github.com/ollama/ollama/issues/5245

If any of the major inference engines - vLLM, Sglang, llama.cpp - incorporated api driven model switching, automatic model unload after idle and automatic CPU layer offloading to avoid OOM it would avoid the need for ollama.

Re: Ollama Turbo

#55

Post by ciaranmca »

mchiang wrote: post #21
decide1000 wrote: post #19

It was fun because it was open. Now it's just another brand seeking dollars.

Ollama at its core will always be open. Not all users have the computer to run models locally, and it is only fair if we provide GPUs that cost us money and let the users who optionally want it to pay for it.

I think it’s the logical move to ensure Ollama can continue to fund development. I think you will probably end up having to add more tiers or some way for users to buy more credits/gpu time. See anthropic’s recent move with Claude code due to the usage of a number of 24/7 users.

Re: Ollama Turbo

#56

Post by llmtosser »

otabdeveloper4 wrote: post #36
smlacy wrote: post #15

Watching ollama pivot from a somewhat scrappy yet amazingly important and well designed open source project to a regular "for-profit company" is going to be sad. Thankfully, this may just leave more room for other open source local inference engines.

[flagged]

This is not true.

No inference engine does all of:

- Model switching

- Unload after idle

- Dynamic layer offload to CPU to avoid OOM

Re: Ollama Turbo

#57

Post by observationist »

pphysch wrote: post #49
riazrizvi wrote: post #42

Earlier quoted context omitted.

No I think the point is to choose the best jurisdiction to have cloud hosted data where your data is best protected from access by very wealthy entities via intelligence services bribery. That’s still hands down the USA.

Any evidence for this claim that e.g. Mossad has less penetration into digital systems of USA than it does RF or PRC?

They might have access to any given machine, but they lack the broad scope of general surveillance. If they want to get you, just like most of the other nation state level threats, you will get got. For other threat models, the US works pretty well.

I guarantee that nobody cares about or will be surveilling your private AI use unless you're doing other things that warrant surveillance.

The reason big providers suck, as OpenAI is so nicely demonstrating for us, is that they retain everything, the user is the product, and court cases, other situations can unmask and expose everything you do on a platform to third parties. This country seriously needs a digital bill of rights.

Re: Ollama Turbo

#58

Post by jychang »

llmtosser wrote: post #54

Distractions like this probably the reason they still, over a year now, do not support sharded GGUF. https://github.com/ollama/ollama/issues/5245 If any of the major inference engines - vLLM, Sglang, llama.cpp - incorporated api driven model switching, automatic model unload after idle and automatic CPU layer offloading to avoid OOM it would avoid the need for ollama.

That’s just llama-swap and llama.cpp

Re: Ollama Turbo

#59

Post by joecot »

polarbear67 wrote: post #13

Why does everything AI-related have to be $20? Why can't there be tiers? OpenAI setting the standard of $20/m for every AI application is one of the worst things to ever happen.

I strongly recommend together.ai, which allows you to use a lot of different open source models and charges for usage, not a monthly fee.

Re: Ollama Turbo

#60

Post by [deleted] »

dang wrote: post #61
otabdeveloper4 wrote: post #36

Earlier quoted context omitted.

[flagged]

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html

[deleted]
Post reply on HN