Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Web Search

ollama.com

Post reply on HN
Print view View original

171–180 of 182 posts

Re: Ollama Web Search

#171

Post by dcreater »

mchiang wrote: post #119
dcreater wrote: post #113

Earlier quoted context omitted.

This information is very useful to the open source community. Whats the rationale in not "building in the public"? Is Ollama turning its back on the open source community? Also why should we believe ollama web search is better than my locally run searxng server?

Oh yes! that is why I want to provide the names of the providers we use. I do believe in building in the open. The web search functionality has a very generous free tier (it is behind Ollama's free account to prevent abuse) that allows you to give it a try comparing to running a searxng server locally. On making the search functionality locally -- we made considerations and gave it a try but had trouble around result…

> I'd want us to revisit this at some point. I believe in having the power of local

Thanks! please do!

Re: Ollama Web Search

#172

Post by andai »

hombre_fatal wrote: post #157
andai wrote: post #95

Earlier quoted context omitted.

I've been wondering can't this be done p2p? Didn't we solve most of the technical problems in the late 90s / early 2000s? And then just abandoned that entire way of thinking for some reason? If many thousands of people care about having a free / private / distributed search engine, wouldn't it make sense for them to donate 1% of their CPU/storage/network to an indexer / db that they they then all benefit from?

Well, flesh it out more and it doesn't sound solved at all. How do you make it trustless. How do you fetch/crawl the index when it's scattered across arbitrary devices. How do you index the decentralized index. What is actually stored on nodes. When you want to do something useful with the crawled info, what does that look like.

I think you could do it hierarchically, and with redundancy.

You'd figure out a replication strategy based on observed reliability (Lindy effect + uptime %).

It would be less "5 million flaky randoms" and more "5,000 very reliable volunteers".

Though for the crawling layer you can and should absolutely utilize 5 million flaky randoms. That's actually the holy grail of crawling. One request per random consumer device.

I think the actual issue wouldn't be the technical issue but the selection. How do you decide what's worth keeping.

You could just do it on a volunteer basis. One volunteer really likes Lizard Facts and volunteers to host that. Or you could dynamically generate the "desired semantic subspace" based on the search traffic...

Re: Ollama Web Search

#173

Post by yggdrasil_ai »

parthsareen wrote: post #37
yggdrasil_ai wrote: post #32

I wish they would instead focus on local tool use. I could just use my own web search via brave api.

Hey! Author of the blogpost and I also work on Ollama's tool calling. There has been a big push on tool calling over the last year to improve the parsing. What's the issues you're running into with local tool use? What models are you using?

I went back and had another look at my implementation, and got it to work. Sorry I was mistaken!

Re: Ollama Web Search

#174

Post by andai »

andai wrote: post #172
hombre_fatal wrote: post #157

Earlier quoted context omitted.

Well, flesh it out more and it doesn't sound solved at all. How do you make it trustless. How do you fetch/crawl the index when it's scattered across arbitrary devices. How do you index the decentralized index. What is actually stored on nodes. When you want to do something useful with the crawled info, what does that look like.

I think you could do it hierarchically, and with redundancy. You'd figure out a replication strategy based on observed reliability (Lindy effect + uptime %). It would be less "5 million flaky randoms" and more "5,000 very reliable volunteers". Though for the crawling layer you can and should absolutely utilize 5 million flaky randoms. That's actually the holy grail of crawling. One request per random consumer device.…

Let me illustrate this with a more poetic example.

In 2015, I was working at a startup incubator hosted inside of an art academy.

I took a nap on the couch. I was the only person in the building, so my full attention was devoted to the strange sounds produced by the computers.

There were dozens of computers there. They were all on. They were all wasting hundreds of watts. They were all doing essentially nothing. Nothing useful.

I could feel the power there. I could feel, suddenly, all the computers in a thousand mile radius. All sitting there, all wasting time and energy.

Re: Ollama Web Search

#175

Post by avmich »

simonw wrote: post #149
userbinator wrote: post #118

Earlier quoted context omitted.

(IANAL) You can normally safely ignore such things.

My nightmare scenario is that I build my own crucial database of information partially derived from a search API... and then later get into legal trouble which forces me to delete that data, which is now intermingled with other information I've collected.

So we don't have just data now, but data-obtained-by-particular-process? If you have a database, should it matter how it was gathered?

Re: Ollama Web Search

#176

Post by simonw »

avmich wrote: post #175
simonw wrote: post #149

Earlier quoted context omitted.

My nightmare scenario is that I build my own crucial database of information partially derived from a search API... and then later get into legal trouble which forces me to delete that data, which is now intermingled with other information I've collected.

So we don't have just data now, but data-obtained-by-particular-process? If you have a database, should it matter how it was gathered?

Yes - it's important to me that I understand the source of the data I've collected and if that source results in restrictions on what I can do with that data.

Especially when I'm building databases that I want other organizations to be able to use.

Fun fact: many geocoding APIs have restrictions on what you can do with the data you get back from that geocoder - including how long you can store it and whether you are allowed to re-syndicate to other people. That's one of the reasons I like OpenCage: https://opencagedata.com/guides/how-to-compare-and-test-geoc...

Re: Ollama Web Search

#177

Post by tripplyons »

pzo wrote: post #73
throwaway12345t wrote: post #28

Earlier quoted context omitted.

We need more indexes

perplexity added API today, got the following email: > Dear API user, We’re excited to launch the Perplexity Search API — giving developers direct access to the same real-time, high-quality web index that powers Perplexity’s answers.

This doesn't mean they run their own index. They are likely just reselling access to whatever index they are using for their product.

Re: Ollama Web Search

#178

Post by jpencausse »

mchiang wrote: post #58
simonw wrote: post #9

I'd love to know what search engine provider they're using under the hood for this. I asked them on Twitter and didn't get a reply (yet) https://twitter.com/simonw/status/1971210260015919488 Crucially, I want to understand the license that applies to the search results. Can I store them, can I re-publish them? Different providers have different rules about this.

We work with search providers and ensure that we have zero data retention policies in place. The search results are yours to own and use. You are free to do what you want with it. Of course you are bound by local laws of the legal jurisdiction you are in.

Would be curious about legal statement with EU AI Act that kills Bing API (Microsoft switch to Grounding Bing that rephrase the content)

Yes, Ephemeral queries must not retain any data, but there is also other rules, for instance it is forbidden for commercial services (where Ollama have a pricing model ?).

Re: Ollama Web Search

#180

Post by kgeist »

Tepix wrote: post #132

Looks like Ollama is focusing more and more on non-local offerings. Also their performance is worse than say vLLM. What's a good Ollama alternative (for keeping 1-5x RTX 3090 busy) if you want to run things like open-webui (via an OpenAI compatible API) where your users can choose between a few LLMs?

At work I've set up LibreChat + LlamaSwap + llama.cpp

200 weekly users :)

Post reply on HN