Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Turbopuffer: Fast search on object storage

turbopuffer.com

Post reply on HN
Print view View original

41–50 of 68 posts

Re: Turbopuffer: Fast search on object storage

#41

Post by AYBABTME »

nh2 wrote: post #28

> $3600.00/TB/month It doesn't have to be that way. At Hetzner I pay $200/TB/month for RAM. That's 18x cheaper. Sometimes you can reach the goal faster with less complexity by removing the part with the 20x markup.

200$/TB/month for raw RAM, not RAM that's presented to you behind a usable API that's distributed and operated by someone else, freeing you of time.

It's not particularly useful to compare the cost of raw unorganized information medium on a single node, to highly organized information platform. It's like saying "this CPU chip is expensive, just look at the price of this sand".

Re: Turbopuffer: Fast search on object storage

#42

Post by formerly_proven »

nh2 wrote: post #28

> $3600.00/TB/month It doesn't have to be that way. At Hetzner I pay $200/TB/month for RAM. That's 18x cheaper. Sometimes you can reach the goal faster with less complexity by removing the part with the 20x markup.

You seem to be quoting the highest figure from the article out of context as-if that is their pricing, but the opposite is the case.

> $3600.00/TB/month (incumbents)

> $70.00/TB/month (turbopuffer)

That's still 3x cheaper than your number and it's a SaaS API, not just a piece of rented hardware.

Re: Turbopuffer: Fast search on object storage

#43

Post by kirmerzlikin »

AYBABTME wrote: post #41
nh2 wrote: post #28

> $3600.00/TB/month It doesn't have to be that way. At Hetzner I pay $200/TB/month for RAM. That's 18x cheaper. Sometimes you can reach the goal faster with less complexity by removing the part with the 20x markup.

200$/TB/month for raw RAM, not RAM that's presented to you behind a usable API that's distributed and operated by someone else, freeing you of time. It's not particularly useful to compare the cost of raw unorganized information medium on a single node, to highly organized information platform. It's like saying "this CPU chip is expensive, just look at the price of this sand".

AFAIU, 3600$ is also a price for "raw RAM" that will be used by your common database via sys calls and not via a "usable API operated by someone else"

Re: Turbopuffer: Fast search on object storage

#44

Post by hobofan »

eknkc wrote: post #38
tionis wrote: post #22

Earlier quoted context omitted.

You could use a sqlite database and use range queries using something like this: https://github.com/psanford/sqlite3vfshttp https://github.com/phiresky/sql.js-httpvfs Simon Willison wrote about it: https://simonwillison.net/2022/Aug/10/sqlite-http/

Yep this thing is the reason I thought about doing it in the first place. Tried duckdb which has built in support for range requests over http. Whole idea makes sense but I feel like the file format should be specifically tuned for this use case. Otherwise you end up with a lot of range requests because it was designed for disk access. I wondered if anything was actually designed for that.

Parquet and other columnar storage formats are essentially already tuned for that.

A lot of requests in themselves shouldn't be that horrible with Cloudfront nowadays, as you both have low latency and with HTTP2 a low-overhead RPC channel.

There are some potential remedies, but each come with significant architetural impact:

- Bigger range queries; For smallish tables, instead of trying to do point-based access for individual rows, instead retrieve bigger chunks at once and scan through them locally -> Less requests, but likely also more wasted bandwidth

- Compute the specific view live with a remote DuckDB -> Has the downside of having to introduce a DuckDB instance that you have to manage between the browser and S3

- Precompute the data you are interested into new parquest files -> Only works if you can anticipate the query patterns enough

I read in the sibling comment that your main issue seems to be re-reading of metadata. DuckDB is AFAIK able to cache the metadata, but won't across instances. I've seen someone have the same issue, and the problem was that they only created short-lived DuckDB in-memory instances (every time the wanted to run a query), so every time the fresh DB had to retrieve the metadata again.

Re: Turbopuffer: Fast search on object storage

#45

Post by eknkc »

hobofan wrote: post #44
eknkc wrote: post #38

Earlier quoted context omitted.

Yep this thing is the reason I thought about doing it in the first place. Tried duckdb which has built in support for range requests over http. Whole idea makes sense but I feel like the file format should be specifically tuned for this use case. Otherwise you end up with a lot of range requests because it was designed for disk access. I wondered if anything was actually designed for that.

Parquet and other columnar storage formats are essentially already tuned for that. A lot of requests in themselves shouldn't be that horrible with Cloudfront nowadays, as you both have low latency and with HTTP2 a low-overhead RPC channel. There are some potential remedies, but each come with significant architetural impact: - Bigger range queries; For smallish tables, instead of trying to do point-based access for i…

Thanks for the insights. Precomputing is not really suitable for this and the thing is, I'm mostly using it as a lookup table on key / value queries. I know Duckdb is mostly suitable for aggregation but the http range query support was too attractive to pass on.

I did some tests, querying "where col = 'x'". If the database was a remote duckdb native db, it would issue a bunch of http range requests and the second exact call would not trigger any new requests. Also, querying for col = foo and then col = foob would yield less and less requests as I assume it has the necesary data on hand.

Doing it on parquet, with a single long running duckdb cli instance, I get the same requests over and over again. The difference though, I'd need to "attach" the duckdb database under a schema name but would query the parquet file using "select from 'http://.../x.parquet'" syntax. Maybe this causes it to be ephemeral for each query. Will see if the attach syntax also works for parquet.

Re: Turbopuffer: Fast search on object storage

#47

Post by hobofan »

eknkc wrote: post #45
hobofan wrote: post #44

Earlier quoted context omitted.

Parquet and other columnar storage formats are essentially already tuned for that. A lot of requests in themselves shouldn't be that horrible with Cloudfront nowadays, as you both have low latency and with HTTP2 a low-overhead RPC channel. There are some potential remedies, but each come with significant architetural impact: - Bigger range queries; For smallish tables, instead of trying to do point-based access for i…

Thanks for the insights. Precomputing is not really suitable for this and the thing is, I'm mostly using it as a lookup table on key / value queries. I know Duckdb is mostly suitable for aggregation but the http range query support was too attractive to pass on. I did some tests, querying "where col = 'x'". If the database was a remote duckdb native db, it would issue a bunch of http range requests and the second exa…

I think both should work, but you have to set the object cache pragma IIRC: https://duckdb.org/docs/configuration/pragmas.html#object-ca...

Re: Turbopuffer: Fast search on object storage

#48

Post by tiew9Vii »

cdchn wrote: post #31
eknkc wrote: post #9

Is there a good general purpose solution where I can store a large read only database in s3 or something and do lookups directly on it? Duckdb can open parquet files over http and query them but I found it to trigger a lot of small requests reading bunch of places from the files. I mean a lot. I mostly need key / value lookups and could potentially store each key in a seperate object in s3 but for a couple hundred mi…

>Is there a good general purpose solution where I can store a large read only database in s3 or something and do lookups directly on it? I think this is pretty much what AWS Athena is.

Cloud backed SQLLite looks like it might be good for this. Doesn’t support S3 though

https://sqlite.org/cloudsqlite/doc/trunk/www/index.wiki

Re: Turbopuffer: Fast search on object storage

#50

Post by lyu07282 »

omneity wrote: post #19
jbellis wrote: post #15

Earlier quoted context omitted.

Have you tried it? pgvector performance falls off a cliff once you can't cache in ram. Vector search isn't like "normal" workloads that follow a nice pareto distribution.

Tried and deployed in production with similar sized collections. You only need enough memory to load the index, definitely not the whole collection. A typical index would most likely fit within a few GBs. And even if you need dozens of GBs of RAM it won’t cost nearly as much as $20k/month as the article surmises.

How do you get to "a few GBs"? A hundred million embeddings, if you have 4 byte floats 1024 dimensions would be >400 GB alone.
Post reply on HN