Vector indexing all of Wikipedia on a laptop
1–10 of 146 posts
Re: Vector indexing all of Wikipedia on a laptop
#2Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
Re: Vector indexing all of Wikipedia on a laptop
#3Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
Got any details?
Re: Vector indexing all of Wikipedia on a laptop
#4Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
Did you do use the same method, i.e. split by chunks each article and vectorize each chunk?
Re: Vector indexing all of Wikipedia on a laptop
#5Re: Vector indexing all of Wikipedia on a laptop
#6Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
How?
Re: Vector indexing all of Wikipedia on a laptop
#7>Disable swap before building the index. Linux will aggressively try to cache the index being constructed to the point of swapping out parts of the JVM heap, which is obviously counterproductive. In my test, building with swap enabled was almost twice as slow as with it off.
This is an indication to me that something has gone very wrong in your code base.
Re: Vector indexing all of Wikipedia on a laptop
#8Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
Got any details?
Nothing too crazy, just downloading a dump, splitting it into manageable batch sizes, and using a lightweight embedding model to vectorize each article.
Using the best GPU available on colab it takes maybe 8 hours if I remember correctly? Vectors can be saved as NPY files and loaded into something like FAISS for fast querying.
Re: Vector indexing all of Wikipedia on a laptop
#9Maybe I’m missing something but I’ve created vector embeddings for all of English Wikipedia about a dozen times and it costs maybe $10 of compute on Colab, not $5000
This is covering 300+ languages, not just English, and it's specifically using Cohere's Embed v3 embeddings, which are provided as a service and currently priced at US$0.10 per million tokens [1]. I assume if you're running on Colab you're using an open model, and possibly a relatively lighter weight one as well?
Re: Vector indexing all of Wikipedia on a laptop
#10>Disable swap before building the index. Linux will aggressively try to cache the index being constructed to the point of swapping out parts of the JVM heap, which is obviously counterproductive. In my test, building with swap enabled was almost twice as slow as with it off. This is an indication to me that something has gone very wrong in your code base.
As a workaround, you can mlock your process which should prevent the application pages from being evicted by swap.