TerarkDB, ByteDance's RocksDB replacement • Hacker News Board

Re: TerarkDB, ByteDance's RocksDB replacement

#81

Post by ddorian43 » Thu, Dec 24, 2020, 8:05 AM UTC

royguo1988 wrote: post #13

royguo1988 wrote: post #11

Earlier quoted context omitted.

We are working on our `all-in-one docs` which will explain everything. I want to address that we are not meant to "get rid of" RocksDB (which lots of KV engine claimed). What we want to do is provide another solution for storage engine users with different road path (focusing on new hardware and heavy-write workloads). For simple use cases, there will be no difference no matter what engine you use. And for most cases…

The reasons we did a better job(from our own perspective) than RocksDB are: 1. We moved lots of code out side db_mutex (db mutex is convenient but costs too much) 2. We introduced a new KV separation implementation that we believe is better than RocksDB’s implementation (we didn't hear any production user are using RocksDB's KV separation yet) 3. We introduced a lazy compaction strategy that can delay compaction task…

There is https://pingcap.com/blog/titan-storage-engine-design-and-imp... that splits keys from values.

Re: TerarkDB, ByteDance's RocksDB replacement

#82

Post by that_guy_iain » Thu, Dec 24, 2020, 8:57 AM UTC

wenc wrote: post #67

that_guy_iain wrote: post #48

Earlier quoted context omitted.

But what if RocksDB or TerakDB would solve a specific problem I'm facing and I just don't know of this solution? Lots of us have problems but don't know the exact tech stack to solve our problems; this is true at nearly every tech company I've worked for. My favourite anecdote for this was a guy basically reinvented map/reduce in the form of hacky scripts from Hadoop round about the time Hadoop and map/reduce was sta…

There are two kinds of mindsets when it comes to learning: a consumer mindset and an autodidact mindset. In an organization, it's easy to recognize consumers -- they typically say things like: "I don't understand this. Is there a training course for this that I can sign up for?" and expect to be assigned to an internal training session or to some external course. An autodidact on the other hand goes: "I don't underst…

> An autodidact on the other hand goes: "I don't understand this. Let me do some research on my own and try to teach myself."

As someone who is very autodidact, I can tell you, that just because I don't understand something doesn't mean I go and learn it. There are far too many things to do and to learn just to go and learn things when I have a reason to. Just like consumers will ask for training sessions when they have a reason to.

This is marketing 101, you have a product and you want people to use it. Even in open source, you still want people to use it, you want there to be value in the thing that you built. Build it and they will come doesn't work.

> There's nothing wrong in asking for a clarifying blurb (good marketing aims to make things frictionless for potential customers). But RocksDB is its own universe and it's actually pretty well known. I don't work in this space, and even I know what RocksDB is because it has come up a lot in technical discussions about storage engines. When I first encountered it, I had no idea what it was, but I gathered from comments that people were excited about it, so I googled "wiki rocksdb". It took 2 seconds.

I have heard of RocksDB before, but I still don't know the use-case, why? Because there are so many other database systems. And even if I was using RocksDB looking at that paper I don't know why as a company I would invest in a rewrite to switch over to this new one, since the performance benefits don't seem massively clear.

> Truly curious people are autodidacts, not consumers.

What I think you think autodidacts are, are people with no focus and spend time researching every new thing that pops up. The sort of people that "Jack of all trades, master of none" is made up to describe.

Re: TerarkDB, ByteDance's RocksDB replacement

#83

Post by KingOfCoders » Thu, Dec 24, 2020, 9:58 AM UTC

reitzensteinm wrote: post #78

KingOfCoders wrote: post #59

Earlier quoted context omitted.

And how should I know this from the README? How should I decide between a project that is interesting and not explained from a project that is not intended for me?

You're shifting the goal posts. You wrote the initial post saying they're hurting themselves by not being more clear. I don't think that's true. Now you're saying they should be more clear to help beginners in the field. I completely agree.

I was rephrasing my point because some didn't understand it the first time. It appears you are cherry picking and missresenting my post again.

It had several points where the first point was about me as someone reading dozens of HN posts a day following links and needing to go on a googling spree to find out about a product I might or might not be interested in.

The other point was about the missed opportunity to attract users.

And even the second point used "projects" and talked about relase pages which made it clear that the it's about the generality of the problem (the linked page wasn't even a release page) Picking out the specifics you either haven't read the post or are misresenting it intentionally.

Re: TerarkDB, ByteDance's RocksDB replacement

#84

Post by KingOfCoders » Thu, Dec 24, 2020, 10:02 AM UTC

that_guy_iain wrote: post #82

wenc wrote: post #67

Earlier quoted context omitted.

There are two kinds of mindsets when it comes to learning: a consumer mindset and an autodidact mindset. In an organization, it's easy to recognize consumers -- they typically say things like: "I don't understand this. Is there a training course for this that I can sign up for?" and expect to be assigned to an internal training session or to some external course. An autodidact on the other hand goes: "I don't underst…

> An autodidact on the other hand goes: "I don't understand this. Let me do some research on my own and try to teach myself." As someone who is very autodidact, I can tell you, that just because I don't understand something doesn't mean I go and learn it. There are far too many things to do and to learn just to go and learn things when I have a reason to. Just like consumers will ask for training sessions when they h…

You've made that point much better than me. As someone who teached myself coding in a department store as a kid in 80 or 81 and learned 20+ languages in the last 40 years on my own, I exactly agree with your point about autodidacts.

I'm interested in a many different things, as you I have read about RocksDB before but my time is limited between my family, hobbies and work.