Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Red Hat Satellite to standardize on PostgreSQL backend

redhat.com

Post reply on HN
Print view View original

121–130 of 298 posts

Re: Red Hat Satellite to standardize on PostgreSQL backend

#121

Post by bschwindHN »

nouveaux wrote: post #109
paxys wrote: post #102

Earlier quoted context omitted.

In-memory or local disk databases aren't going to work for any real world use case, regardless of how small or simple your application is. Using Postgres or MySQL is barely more complex than SQLite, and it makes more sense to start with one of them than move everything over once you realize you aren't going to be running on your dev machine forever.

After developing with Postgres for several years and then moving to sqlite for dev, sqlite is a breath of fresh air. Now our policy is that everything goes into sqlite db unless the data requires multiple writers. The fact that you have to keep a process running and updated is reason enough to not use Postgres if you can help it. For instance, we have processes that fetches and stores data from APIs. It's trivial to…

I'm using sqlite for a small project but I was kind of disappointed to learn how annoying it is to drop or rename columns. Especially when you write migration scripts with up/down statements, it's kind of a pain compared to postgres.

But otherwise it's been fine to use, a good starting point for internal projects.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#122

Post by shanemhansen »

fyfy18 wrote: post #97
mikekchar wrote: post #74

Earlier quoted context omitted.

When you want to do validation depends on when you can do something about it. I work with a NO-SQL DB at work and while it wouldn't be my choice for most things I would use a DB for, the lack of validation has some benefits. A good example is where you have no ability to validate input from a user, but where you need to store the data anyway. The last thing you want is your noisy data being kicked out by the DB becau…

It sounds more like an edge case though. I can't imagine all the data you need to store may or may not be the right format, so I wouldn't switch my database just because one or two entities need this. Anyway this is 2019 so PostgreSQL JSONB fields have got you covered. You can even efficiently query the JSON objects within them.

Unlogged jsonb tables in postgres have generally made nosql systems look pretty bad. I'm really happy the industry finally came up with vitesse so we could have a middle option between "My ACID database needs to scale writes so I'll roll my own fragile sharding layer" and "give up all attempts at schema and consistency and transactions".

Vitess is a really comfortable middle ground of fairly familiar database semantics within a partition.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#123

Post by almostdeadguy »

thomaslangston wrote: post #100
int_19h wrote: post #77

Earlier quoted context omitted.

First of all, we were talking about validation of data in the database, specifically. > 'validation' generally implies aspects which are inherently application specific Not at all. Taking this at face value implies that some app can write data to the database that is valid according to that app, and then another app can read data that is invalid from its perspective, and have to deal with it. That doesn't make sense…

The general idea I've seen (for both SQL and NoSQL databases) is two apps never should write to the same database to ensure separation of concerns. Some API layer instead handles all write operations. Disclaimer: MongoDB employee. All opinions are my own.

That's generally a good practice (though not always, many people do blue-green deployments, for instance), but it's rarely an assumption you want to make. Lots of deployment blunders can happen that render assumptions like that incorrect.

Even if your team executes perfectly and never runs into this, the biggest problem IMO is that you can't really enforce most of your guarantees w/ any degree of confidence w/o a typed schema. Even if you work within a typed language that perfectly validates all the invariants of your application before storing anything, the second you need to perform work that does not strictly funnel data through your application (i.e. an update query), you are effectively gambling on whether or not those invariants will hold. This kind of "read-modify-write" flow of data doesn't really perform well (or even hold validity) for a lot of common use cases, so in reality you need your database to ensure these things for you.

Also the two deployed apps problem is just a special case of two people interacting w/ a database who aren't working under the same assumptions as to what invariants should hold. That can happen in single code bases, even with a lot of care taken.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#124

Post by inopinatus »

int_19h wrote: post #43
bane wrote: post #29

Rule #1: Always start with PostgreSQL unless you have a very compelling reason not to.

Always start with in-memory data structure with straightforward persistence (i.e. load and save it all at once, using some popular format). If you need ACID, then start with SQLite. If you need scalability as well, then PostgreSQL. Most other features aren't worth the hassle of configuration compared to a single file on disk and a library to link to. (But I suspect that most NoSQL apps these days would do just fine w…

There was a time when it was a shoo-in to also say "if you only need a key-value store, use Berkeley DB".

Re: Red Hat Satellite to standardize on PostgreSQL backend

#125

Post by rtpg »

kevinmgranger wrote: post #98
int_19h wrote: post #43

Earlier quoted context omitted.

Always start with in-memory data structure with straightforward persistence (i.e. load and save it all at once, using some popular format). If you need ACID, then start with SQLite. If you need scalability as well, then PostgreSQL. Most other features aren't worth the hassle of configuration compared to a single file on disk and a library to link to. (But I suspect that most NoSQL apps these days would do just fine w…

For most languages and frameworks, using Postgres as opposed to sqlite is barely a higher cost if you’ve got the ops skills. I’ve yet to see an app that straddled this local/web-scale line where SQLite was a better choice or Postgres wasn’t the obvious choice. There isn’t some massive cost to using Postgres instead, is there some desktop divide I’m missing here?

sqlite doesn't require a server, right? It's just "read the database file", whereas Postgres requires a running server as well as the client processes.

Granted, if your ops are at the stage where "run postgres myself" is a pain, using some managed DB (heroku?) probably makes more sense than sticking to the sqlite file, if only because you'll be paying for "real backups" and availability.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#126

Post by atombender »

mlevental wrote: post #28
foobiekr wrote: post #14

Earlier quoted context omitted.

A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things…

how do you do this? tie it all the way from the DB to the client? incidentally I believe this is the difference between being an engineer and being a product person. I straddle the line. in theory I want a strong type system so I can be confident about the future. in practice I don't want to pay the upfront cost and I'm more interested in shipping now and taking on debt.

Look at GraphQL. It lets you enforce a single schema from the front end down to the data layer.

In my company's product, we specify the data models as JSON Schema, and then generate the necessary language code -- we generate the GraphQL schema from it as well as Go data types, with some database glue. Our front end code is currently JavaScript, but we hope to migrate to TypeScript, which will make everything statically types all the way through.

(gRPC fills a similar role, though the web story is lacking. gRPC is brilliant for APIs between backend services, but browser support is not there yet. GraphQL is more convenient for the app-facing layers.)

Re: Red Hat Satellite to standardize on PostgreSQL backend

#127

Post by dralley »

wcchandler wrote: post #72

Any guesses on a timeline? Satellite 6.6? Or will it be in the next major version? 7.0?

Pulp 2 will continue using MongoDB until EOL. Pulp 3, which is still in development, is using Postgres -- but while the platform functionality is nearly done the plugins for e.g. RPM and Docker are not yet near feature parity.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#128

Post by stephenr »

staticassertion wrote: post #49
djrobstep wrote: post #41

Earlier quoted context omitted.

My pet theory is that NoSQL took off purely because people were sick of having to manage schema changes. Unfortunately, people reacted to being (justifiably) frustrated with schemas by throwing strict schemas out entirely, instead of making better schema management/migration tools.

Tell that to AWS. They've banned relational databases for specific workloads because Dynamo (nosql) provides more consistent performance, and is easier to operate. Tons of conflation of Mongo's problems with those of nosql in this thread.

It couldn't possibly because they're trying to push use of their own technology to force dogfooding.

Re: Red Hat Satellite to standardize on PostgreSQL backend

#129

Post by [deleted] »

int_19h wrote: post #43
bane wrote: post #29

Rule #1: Always start with PostgreSQL unless you have a very compelling reason not to.

Always start with in-memory data structure with straightforward persistence (i.e. load and save it all at once, using some popular format). If you need ACID, then start with SQLite. If you need scalability as well, then PostgreSQL. Most other features aren't worth the hassle of configuration compared to a single file on disk and a library to link to. (But I suspect that most NoSQL apps these days would do just fine w…

[deleted]

Re: Red Hat Satellite to standardize on PostgreSQL backend

#130

Post by yjftsjthsd-h »

atm0sphere wrote: post #86
baroffoos wrote: post #54

Earlier quoted context omitted.

Setting up graylog was one of the worst mistakes I made. It took forever to get all the required software installed and configured and then it was taking up all the ram on the server doing fuck all.

There were single script 1-click installs for it in bash all over for me..but yeah jvm is a hog.

My experience with those one-liner installs is that they usually work... strictly speaking. They don't scale, they don't deal with edge cases, they know nothing of your environment. They install one piece of software (in an "interesting" way that won't upgrade), and that's it.
Post reply on HN