Live data from Hacker News

Database Review 2021

bytebase.com

41–50 of 68 posts

Re: Database Review 2021

#41
post #15

Earlier quoted context omitted.

Setting up Postgres is a PITA compared to SQLite. It comes bundled with python these days. Obviously it's going to be a trade off as to which one causes you more pain.

agreed, running pg locally is a pain. I use a cloud postgres instance (even for local dev). They're dirt cheap and it's not worth the hassle of working with a local pg.

What OS are you on? Running postgres on Linux is as trivial as it gets, and for Windows they have a nice installer. Or just use docker.

Re: Database Review 2021

#42

This is a review of companies making databases, not the databases themselves.

Correct. And Databricks/Spark is not really a database, it's a processing engine which can connect to many databases.

I think you could view SparkSQL with catalyst optimizer as a DB.

Re: Database Review 2021

#43
It's curious to see news about many database products that have come out only recently, since in my personal experience I've seen some companies and their products just evaporate (for example, Clusterpoint a number of years back).

Of course, if there's a sufficient amount of hype/mindshare from the industry towards a certain technology, it feels like the critical mass of attention might perhaps be there to sustain the projects and ensure adoption that's widespread enough for the companies behind them to profit and stay around.

It's also nice to see mentions of MySQL, albeit sadly there's nothing of MariaDB, which has become the "replacement" for MySQL in some projects out there due to the pretty good compatibility: https://mariadb.org/

They even seem to have their own cloud offering, though I cannot comment on it personally (it feels enterprisey and generally I run my own DBs, or let someone manage them in other projects): https://mariadb.com/products/skysql/

Though it's also nice to see PostgreSQL remain popular and it's generally one of the better options for most of the projects out there in my eyes. It has decent tooling, good driver support and lots of useful functionality and plugins (things like PostGIS, for example). I think the article puts it nicely, even if someone could nitpick about the wording:

> PS is batteries included. When a company chooses PG, it gets a database with OLTP, OLAP, Document (JSON-B), FTS (tsvector/tsquery), Time-series (TimescaleDB), Geospatial (PostGIS), Multi-tenancy capabilities (batteries included). To summarize it in a simple formula:

> PostgreSQL = MySQL + Poor man version of (ClickHouse + MongoDB* + Elasticsearch + InfluxDB) + Geospatial + Multi-tenancy

Then again, I come from Latvia and generally the technology choices that I've seen are a bit on the "boring" side (which isn't always a bad thing). Lots of PostgreSQL, some MySQL (sometimes MariaDB), though also proprietary offerings like Oracle DB or MS SQL in enterprise projects. Here, certain solutions are comparatively rare, like MongoDB, Redis or RabbitMQ/Kafka.

CockroachDB, ClickHouse, Yugabyte, CouchDB and a bunch of others are basically unheard of. I suspect that in certain capacity it's "Nobody ever got fired for choosing IBM", in part the fact that the tech scene here is lukewarm at best and people aren't interested in experimenting that much, or maybe just choose whatever has worked decently in the past.

I wonder whether one can read more into this trend, with countries like mine being a few years behind in adoption of certain new things. Only recently (the past few years) seeing Kubernetes at scale comes to mind.

Oh, I also have to support the argument that SQLite is pretty nice for simpler setups, or as an application format. My homepage actually runs on SQLite, since it doesn't need to scale much.

Re: Database Review 2021

#44
I recently started to play with BigQuery for the first time and it is kind of an unreal piece of software. I've been able to ingest and query hundreds of GB of data in a matter of seconds.

Re: Database Review 2021

#45

> It already seems to be an afterthought after Gigapipe and Firebolt debuts. This is referring to Clickhouse Inc. and is way off the mark. If you're looking for hosted Clickhouse, why would a company whose CTO is the creator of Clickhouse be an "afterthought"? Their cloud product offers separation of storage and compute, but you still have an escape hatch to self-hosting if you need it (which you wouldn't have with F…

I hadn't heard of Clickhouse, so had a look. It seems to be an OLAP backing store and not what one would call an OLAP database - because it doesn't support MDX. You have to use Mondrian as the OLAP server middleware. Or am I misunderstanding? And why would one use Clickhouse vs Sql Server as the backing store for OLAP? Is it that much cheaper?

ClickHouse is ridiculously fast, and its architecture is versatile while still being easy to setup. Its just a very well made product.

Re: Database Review 2021

#46

If pg was olap there wouldn’t exist any competition out there. IMO hybrid oltp and olap is what most companies need for their user facing apps and whoever nails it will become #1 in zero time. Ps. I would like to test firebolt, sounds promising.

there's a few projects that are creating something like that, most include some type of column based storage extension for OLAP stuff

Re: Database Review 2021

#47

Considered posting an Ask HN, and this is a bit of a segue, but seems a relevant place to ask... What do people think of DynamoDB? I think it fits my use case, and apparently, there's enough overlap with Cassandra to support a migration, if it turns out to be a bad idea. But I rarely see much writing about teams using it, so wondered about support / popularity / resources?

The hard 400kb per item limit does sort of force a pause to make sure it will still work well over the lifetime of the app. Not that huge "rows" are a good idea, but that's roughly 2 pages of text.

Re: Database Review 2021

#48
post #47

Considered posting an Ask HN, and this is a bit of a segue, but seems a relevant place to ask... What do people think of DynamoDB? I think it fits my use case, and apparently, there's enough overlap with Cassandra to support a migration, if it turns out to be a bad idea. But I rarely see much writing about teams using it, so wondered about support / popularity / resources?

The hard 400kb per item limit does sort of force a pause to make sure it will still work well over the lifetime of the app. Not that huge "rows" are a good idea, but that's roughly 2 pages of text.

I think it's more like 200 pages of text -- 2kB/page is what I've always worked off.

Re: Database Review 2021

#49
post #2

My prediction: SQLite will keep gaining popularity. Especially among pragmatic software builders who run their own business and do not work for the man. A demographic that I expect to grow. Talking about SQLite: Is there any downside to partitioning an SQLite db into multiple files? For example one of my systems has a table 'details' which is not vital for the system to work. It's just a nice to have, to have data in…

SQLite has seen runaway popularity on HN lately and I bought into the hype too for a while but when I look under the hood, the 3rd party backup and replication stories just seem janky, tedious and not yet mature. It's the kind of thing where a misconfiguration could wipe out everything and/or waste you hours of time. >Especially among pragmatic software builders who run their own business and do not work for the man.…

I also find SQLite to be a poor solution for a backend database.

In particular - it makes it incredibly frustrating to manage multiple instances accessing it, and has some very strict limitations around how the underlying FS is mounted.

SQLite is an incredible tool - but the right place for it is in a deployed client application (where - seriously - it's a first class project and is an incredible joy). It's not really designed to be your web db.

Re: Database Review 2021

#50

> It already seems to be an afterthought after Gigapipe and Firebolt debuts. This is referring to Clickhouse Inc. and is way off the mark. If you're looking for hosted Clickhouse, why would a company whose CTO is the creator of Clickhouse be an "afterthought"? Their cloud product offers separation of storage and compute, but you still have an escape hatch to self-hosting if you need it (which you wouldn't have with F…

Couldn't agree more with the join statement. Working on a data heavy SaaS, we've looked to overhaul our database for almost a year now, and Clickhouse hits every nail on the head for our needs except this. Most OLAP databases, and especially Clickhouse, seem to suffer from the syndrome of being structurally incompatible with relational data - understandably enough for mostly columnar databases, but still have rather weak solutions to bridge the gap. The author talks about an AP/TP hybrid being the industry killer, and it seems to ring true if we measure success on the profitability of the products that will be built on it. Data is where the money is, but not in end result data like IoT performance metrics / generally 1-dimensional data etc but in the intersections of high volumes of data. This ends up giving you two options right now if you're building in this space; A. very expensive and slow/tedious feature building on OLTP databases with lots of preprocessing or B. low cost, high performance but limited in features on OLAP.
Post reply on HN