Live data from Hacker News

How we replaced Elasticsearch and MongoDB with Rust and RocksDB

radar.com

61–70 of 104 posts

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#61
post #47

Earlier quoted context omitted.

Lots of other options here if you don't like managing. You can use Elastic cloud, Bonsai.io, and others

A lot of places can't put their data just anywhere.

They manage it in your GCP project, so you can also make use of your commitments etc.

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#62

Lol I "love" that the first benefit this company lists in their jobs page is "In-Office Culture". Do people actually believe that having to commute is a benefit ?

In-office culture would be dope if there were actual benefits to an office like maybe

Learning from smart people, making friends, free food and drinks, a DDR machine

My last office job had none of that. Instead it was just sort of like a depressing scaled up version of my home office

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#63

Lol I "love" that the first benefit this company lists in their jobs page is "In-Office Culture". Do people actually believe that having to commute is a benefit ?

You can't reduce the in-office or remote experience purely to commuting. It's just one aspect about how and where you work and work life balance in general.

But since you asked, yes, I actually enjoy commuting when it is less than 30 minutes each way and especially when it involves physical activities. My best commutes have been walking and biking commutes of around 20-25 minutes each way. They give me exercise, a chance to clear my head, and provide "space" between work and home.

During 2020, I worked from home the entire time and eventually I found it just mentally wasn't good for me to work and live in the same space. I couldn't go into the office, so I started taking hour long walks at the end of every day to reset. It helped a lot.

That said, I've also done commutes of up to an hour each way by crowed train and highway driving and those are...not good.

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#64
post #63

Lol I "love" that the first benefit this company lists in their jobs page is "In-Office Culture". Do people actually believe that having to commute is a benefit ?

You can't reduce the in-office or remote experience purely to commuting. It's just one aspect about how and where you work and work life balance in general. But since you asked, yes, I actually enjoy commuting when it is less than 30 minutes each way and especially when it involves physical activities. My best commutes have been walking and biking commutes of around 20-25 minutes each way. They give me exercise, a ch…

[flagged]

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#65

Lol I "love" that the first benefit this company lists in their jobs page is "In-Office Culture". Do people actually believe that having to commute is a benefit ?

In-office culture would be dope if there were actual benefits to an office like maybe Learning from smart people, making friends, free food and drinks, a DDR machine My last office job had none of that. Instead it was just sort of like a depressing scaled up version of my home office

My office has some nice perks!

1. It's extremely cold and dark! I must wear extra clothes when going inside and I get depressed at wasting a day of nice weather in what looks like a WW1 bunker.

2. Terrible accessibility for disabled people! (such as myself)

3. Filthy toilets!

4. Internet is slower than at home!

5. Half the team lives somewhere else so all meetings are on teams anyway!

6. They couldn't afford a decent headset so I get pain in my head after 5 minutes, but I don't have a laptop so I can't move to a meeting room.

The HR really can't understand why after all these great perks I insist on wanting to work from home. I am such an illogical person!

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#67
post #65

Earlier quoted context omitted.

In-office culture would be dope if there were actual benefits to an office like maybe Learning from smart people, making friends, free food and drinks, a DDR machine My last office job had none of that. Instead it was just sort of like a depressing scaled up version of my home office

My office has some nice perks! 1. It's extremely cold and dark! I must wear extra clothes when going inside and I get depressed at wasting a day of nice weather in what looks like a WW1 bunker. 2. Terrible accessibility for disabled people! (such as myself) 3. Filthy toilets! 4. Internet is slower than at home! 5. Half the team lives somewhere else so all meetings are on teams anyway! 6. They couldn't afford a decent…

Friend works at office that allows dogs. Her workplace is one big dog toilet! She is expected to clean it (she is not toilet cleaner). She get sexually assaulted, when her boss shoved his dog into her crotch!

There were some hospitalisations from work related injuries... Regular bullying, threats of violence....

Lovely office culture!

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#69

Bit thin on details and not looking like they’ll open source it, but if someone clicked the post because they’re looking for their “replace ES” thing: Both https://typesense.org/ and https://duckdb.org/ (with their spatial plugin) are excellent geo performance wise, the latter now seems really production ready, especially when the data doesn’t change that often. Both fully open source including clustered/sharded setu…

DuckDB does not have any kind of sharding or clustering? It doesn't even have a server (unless you count the HTTP Server Extension)?

Good point and was mostly re Typesense (can't edit the comment anymore).

But given that duckdb handles "take this n GB parquet file/shard from a random location, load it into memory and be ready in We use it for both the importer pipeline that processes the 2B row / 200GB compressed GBIF.org parquet dataset and queries like https://www.meso.cloud/plants/pinophyta/cupressales/pinopsid... and the sheer amount of functions[1] beyond simple stuff like "how close is a/b to x/y" or is "n within area x" is just a joy to work with.

[1] https://duckdb.org/docs/stable/core_extensions/spatial/funct...

Re: How we replaced Elasticsearch and MongoDB with Rust and RocksDB

#70
post #19

It’s interesting as someone in the search space how many companies are aiming to “replace Elasticsearch”

Author here! We were really motivated to turn a "distributed system" problem into a "monolithic system" from an operations perspective and felt this was achievable with current hardware, which is why we went with in-process, embedded storage systems like RocksDB and Tantivy. Memory-mapping lets us get pretty far, even with global coverage. We are always able to add more RAM, especially since we're running in the clou…

Why not just use a open source solution like paradedb ... .

Paradedb = postgres pg_search plugin (the base is tantivy). Need anything else like vectors or whatever, get the plugins for postgres.

The only thing your missing is a LSM solution like RocksDB. See Orioledb what is supposed to become a plugin storage engine for postgres but not yet out of beta.

Feels like people reinvent the wheel very often.

Post reply on HN