Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database schema changes are hard (2017)

djrobstep.com

Post reply on HN
Print view View original

81–90 of 93 posts

Re: Database schema changes are hard (2017)

#81

Post by pytester »

ken wrote: post #76
pytester wrote: post #58

Earlier quoted context omitted.

ORM are frameworks that exist mainly as a way to avoid writing and maintaining thousands of lines of boilerplate. Like most frameworks (i.e. packages that force you to write your code to conform to their patterns) a really good one confers a massive boost in productivity while a really bad one is horrifying to work - it fails in bizarre unexplainable ways and straitjackets your development. IME, passionate rejection…

Blaming it all in “really bad ORMs” without naming any sounds suspect to me. Can you name some good ones?

For example JPOX was an ORM that almost put me off the entire idea. It was horrendous.

Recently I've used Django ORM and SQLAlchemy - both are warty but generally decent and better than not using any ORM.

This is somewhat language dependent - certain language qualities inhibit the creation of decent ORMs (e.g. it's not really possible to create a decent ORM in golang due to its design).

Re: Database schema changes are hard (2017)

#82

Post by cheez »

lenticular wrote: post #70
sametmax wrote: post #52

Earlier quoted context omitted.

That's assuming changing the schema is just about changing the layout of the database, which is definitly not true. You have your code depending on it, documentation, constrainsts, replication, caching, db load, deployement systems, different envs and versions... The SQL is the easy part and barely registers as an issue. I see the same problem with SQL lovers rejecting ORMs as a dumb way to avoid using the right tool…

I've had bad luck with ORMs in general, mainly because they fall apart whenever I need to do something complicated. The abstraction is just too leaky. However, I've had really good luck with Doobie, a purely functional SQL library in Scala. The big difference is that queries can be manipulated as ordinary pure values, which really lets you do some cool stuff. I actually used these features to significantly speed up a…

Given your experience (which is replicated all over the world), why not just use ORMs for what they're good at and skip the rest?

Re: Database schema changes are hard (2017)

#83

Post by djrobstep »

simonw wrote: post #77
djrobstep wrote: post #14

Earlier quoted context omitted.

In many (most?) cases people have a single production environment. If you have multiple versions in the wild to support, there's nothing stopping you from supporting multiple upgrade paths, or even continuing to chain migration files, if you want to. Moving data about is always going to be a manual process. This approach just helps you test it better.

I don't buy this at all. It may be true for projects with only a single developer, but consider environments with a larger team. First, every developer has a development environment. If someone goes on holiday for a few weeks they will come back to a personal environment that's likely several steps behind everyone else. Smart engineering orgs have one or (hopefully) more QA environments which differ from production b…

? None of those are production environments.

Re: Database schema changes are hard (2017)

#84

Post by simonw »

djrobstep wrote: post #83
simonw wrote: post #77

Earlier quoted context omitted.

I don't buy this at all. It may be true for projects with only a single developer, but consider environments with a larger team. First, every developer has a development environment. If someone goes on holiday for a few weeks they will come back to a personal environment that's likely several steps behind everyone else. Smart engineering orgs have one or (hopefully) more QA environments which differ from production b…

? None of those are production environments.

They're "production" in as much as they need to work. They still need to have migrations applied to them in an automated and repeatable way.

Re: Database schema changes are hard (2017)

#85

Post by djrobstep »

simonw wrote: post #84
djrobstep wrote: post #83

Earlier quoted context omitted.

? None of those are production environments.

They're "production" in as much as they need to work. They still need to have migrations applied to them in an automated and repeatable way.

"An environment that needs to work" is not what "production environment" means.

Re: Database schema changes are hard (2017)

#86

Post by paulddraper »

recursive wrote: post #23
paulddraper wrote: post #15

Earlier quoted context omitted.

Program: code, dockerfile, Kubernetees config Persistent data: databases tables, disk snapshots program != persistent data

But a particular version of a program expects a certain schema in the persistent data store. So when the program gets upgraded, the persistent data store must be migrated, unless you're able to lose all your data. In that case, you don't need to care about migrations, and it's much easier.

Exactly.

Re: Database schema changes are hard (2017)

#87

Post by speedplane »

chousuke wrote: post #46
speedplane wrote: post #39

Earlier quoted context omitted.

In an RDMS, a change in schema is hard because changing one table can affect many others. In a NOSQL database, there are far fewer connections between tables (or "objects"). There is less to wrap your head around.

That's just up to how you model your data. there's nothing that prevents you from using a single table with a JSON blob in an RDBMS either, but it's throwing away most of the benefits. A NoSQL database might seem to make migrations easier, but I'm not convinced that it actually does.

> That's just up to how you model your data.

Well, a NoSQL database forces you to reduce or eliminate relations, whereas an RDMS allows and encourages them. Sure, theoretically, you can make an RDMS database without relations, but it goes against the standard design methodology. You can be sure that if you implement an RDMS in a large organization, relationships between tables will seep into your schema. With a NoSQL database, that's pretty much impossible.

Re: Database schema changes are hard (2017)

#88

Post by sametmax »

cheez wrote: post #82
lenticular wrote: post #70

Earlier quoted context omitted.

I've had bad luck with ORMs in general, mainly because they fall apart whenever I need to do something complicated. The abstraction is just too leaky. However, I've had really good luck with Doobie, a purely functional SQL library in Scala. The big difference is that queries can be manipulated as ordinary pure values, which really lets you do some cool stuff. I actually used these features to significantly speed up a…

Given your experience (which is replicated all over the world), why not just use ORMs for what they're good at and skip the rest?

That's something I don't understand as well. There is no need to go 100% SQL or 100% ORM. You can use raw SQL with the ORM, and the good ones have actually features to make it easy and productive. You don't even need to use the OOP nature of ORM for querying, good ones will have a functional layer for those as well.

Re: Database schema changes are hard (2017)

#89

Post by cheez »

sametmax wrote: post #88
cheez wrote: post #82

Earlier quoted context omitted.

Given your experience (which is replicated all over the world), why not just use ORMs for what they're good at and skip the rest?

That's something I don't understand as well. There is no need to go 100% SQL or 100% ORM. You can use raw SQL with the ORM, and the good ones have actually features to make it easy and productive. You don't even need to use the OOP nature of ORM for querying, good ones will have a functional layer for those as well.

Indeed, I often use all 3 in the same file, let alone the same app.

Re: Database schema changes are hard (2017)

#90

Post by jjeaff »

zepolen wrote: post #50
jjeaff wrote: post #13

So how do you deal with it when you actually need to "migrate" data? Getting a correct schema is only part of the migration process. Many times a refactoring requires a new table or field to be populated with data from the old table or field. For example, a single table has a one to one relationship that we must select from to insert data into a newly created table so we can have a one to many relationship.

If only there was some way to query the data from the old tables and insert them into the new ones...

On, dev, local, staging, live, and where else?

If you have to manually access all of the databases and run queries on them, then that defeats the purpose of a ci/cd migration tool.

Of course, the migrations that this method is purporting to replace handles that.

Post reply on HN