Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vector indexing all of Wikipedia on a laptop

foojay.io

Post reply on HN
Print view View original

131–140 of 146 posts

Re: Vector indexing all of Wikipedia on a laptop

#131

Post by riku_iki »

janalsncm wrote: post #53
traverseda wrote: post #12

Earlier quoted context omitted.

This is pretty early in the game to be relying on proprietary embeddings, don't you think? If if they are 20% better, blink and there will be a new normal. It's insane to me that someone, this early in the gold rush, would be mining in someone else's mine, so to speak

It’s not just that. Embeddings aren’t magic. If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? There are a lot of related sources of similarity, but they’re slightly different. And I have no idea what Cohere is doing. Additionally, it’s not clear to me how queries can and s…

> here are a lot of different hash functions. Cryptographic hashes, locality sensitive hashes, hashes for checksum, etc.

and there are some standard hash functions in the lib, which cover 98% of usecases. I think the same is embeddings, you can train some foundational multitask model, and embedding will work for variety of tasks too.

Re: Vector indexing all of Wikipedia on a laptop

#132

Post by Mathnerd314 »

> Enough RAM to run a JVM with 36GB of heap space

Are there laptops like that? Maybe an upgraded MacBook, but I have been looking for Windows/Linux laptops and they generally top out at 32GB. I checked Lenovo's website and everything with 64GB and up is not called a laptop but a "mobile workstation".

Re: Vector indexing all of Wikipedia on a laptop

#133

Post by cbolton »

Mathnerd314 wrote: post #132

> Enough RAM to run a JVM with 36GB of heap space Are there laptops like that? Maybe an upgraded MacBook, but I have been looking for Windows/Linux laptops and they generally top out at 32GB. I checked Lenovo's website and everything with 64GB and up is not called a laptop but a "mobile workstation".

You can configure a Lenovo Z13 Gen 2 with 64GB for little extra money (and choose between Windows, Ubuntu, Fedora, or no OS preinstalled).

Re: Vector indexing all of Wikipedia on a laptop

#134

Post by riku_iki »

jbellis wrote: post #123
noufalibrahim wrote: post #121

What are the good solutions in this space? Vector databases I mean. Mostly for semantic search across various texts. I have a few projects I'd like to work on. For typical web projects, I have a "go to" stack and I'd like to add something sensible for vector based search to that.

JVector (the index used in TFA) is available as a service with a friendly API from DataStax. https://www.datastax.com/products/datastax-astra [article author, I work on JVector and Astra]

Could you tell how scalable JVector is? How many vectors it can handle, like millions, billions, hundreds of billions?

Re: Vector indexing all of Wikipedia on a laptop

#135

Post by _zoltan_ »

Mathnerd314 wrote: post #132

> Enough RAM to run a JVM with 36GB of heap space Are there laptops like that? Maybe an upgraded MacBook, but I have been looking for Windows/Linux laptops and they generally top out at 32GB. I checked Lenovo's website and everything with 64GB and up is not called a laptop but a "mobile workstation".

You can buy an M3 Max with 128GB memory.

Re: Vector indexing all of Wikipedia on a laptop

#136

Post by gfourfour »

solarengineer wrote: post #118
gfourfour wrote: post #59

Earlier quoted context omitted.

I will probably make a post when I launch my app! For now I’m trying to figure out how I can host the whole system for cheap because I don’t anticipate generating much revenue

I'm interested in hearing about what you will be hosting. Would Digital Ocean or Hetzner meet your needs?

I was going to use ec2 and s3, should I look at digital ocean or hetzner instead?

Re: Vector indexing all of Wikipedia on a laptop

#138

Post by pradn »

janalsncm wrote: post #53
traverseda wrote: post #12

Earlier quoted context omitted.

This is pretty early in the game to be relying on proprietary embeddings, don't you think? If if they are 20% better, blink and there will be a new normal. It's insane to me that someone, this early in the gold rush, would be mining in someone else's mine, so to speak

It’s not just that. Embeddings aren’t magic. If you’re going to be creating embeddings for similarity search, the first thing you need to ask yourself is what makes two vectors similar such that two embeddings should even be close together? There are a lot of related sources of similarity, but they’re slightly different. And I have no idea what Cohere is doing. Additionally, it’s not clear to me how queries can and s…

I'm with you on this. The vector embedding craze seems to be confusing mechanism and problem. The problem is semantic similarity search. One mechanism is vector embedding. I think all this comes from taking LLMs as a given, seeing that they work reasonably well with phrase-input semantic retrieval, and then hyper-optimizing vector embedding / search to achieve it.

Are there other semantic search systems? What happened to the entire field of Information Retrieval - is vector search the only method? Are all the stemming, linguistic analysis, all that - all obsoleted by vectors?

Or is it purely because vector search is quick? That's just an engineering problem. I'm not convinced it's the only method here. Happy to be corrected!

Re: Vector indexing all of Wikipedia on a laptop

#139

Post by isoprophlex »

criddell wrote: post #137
isoprophlex wrote: post #31

$5000?! I indexed all of HN for ... $50 I think. And that's tens of millions of posts.

How are you using that index?

https://www.searchhacker.news/

A tool that (hopefully) surfaces interesting HN discussion threads; I wanted an excuse to investigate (hybrid) full text and vector search at a substantial scale beyond toy datasets.

Sadly (well not really) I changed jobs soon after building the first version. Life caught up and I never got around to adding more features and polishing up the frontend (eg. the broken back button

Ideas for new features are very welcome :)

Re: Vector indexing all of Wikipedia on a laptop

#140

Post by HammadB »

jbellis wrote: post #124
HammadB wrote: post #119

Earlier quoted context omitted.

Sure, I see. I think this is an area where complexity analysis doesn’t lead to useful information. To be more correct it’s O(N/C log C) where C is the capacity of a segment. In this case you can ignore 1/C and log C as constant. So now sure, you actually just have O(N). But this is not super useful as it says that a segmented hnsw approach and brute force approach are the same - when this is really not the case in pr…

> I’m not sure why we would ever do anything with segmentation according to that analysis if it were correct. What's your alternative when you can't build an index larger than C?

If segmented hsnw indices were O(N log N) - it would make no sense to build the index at all - brute force would be better as O(N log N) > O(N)
Post reply on HN