Slicing columns into multiple tables is fairly common type of sharding. Sort of SQL way to do columnary store.
"We ran out of columns"
21–30 of 588 posts
Re: "We ran out of columns"
#22> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…
This is perfectly fine when you are driving some app that has a per-user experience that allows you to wrap up most of their experience in some blobs. However I would still advise people to use a third normal form - they help you, constraints help you, and often other sets of tooling have poor support for constraints on JSON. Scanning and updating every value because you need to update some subset sucks. You first po…
> Scanning and updating every value because you need to update some subset sucks.
Mirrors my experience exactly. Querying json can get complex to get info from the db. SQLite is kind of forgiving because sequences of queries (I mean query, modify in appliation code that fully supports json ie js, then query again) are less painful meaning it's less moprtant to do everytning in the database for performance reasons. But if you're trying to do everything in 1 query, I think you pay for it at application-writing time over and over.
Re: "We ran out of columns"
#23> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…
I actually love your approach and haven’t thought of that before.
My problem with relational databases often stems from the fact that remodeling data types and schemas (which you often do as you build an application, whether or not you thought of a great schema beforehand) often comes with a lot of migration effort.
Pairing your approach with a „version“ field where you can check which version of a schema this rows data is saved with would actually allow you to be incredibly flexible with saving your data while also being able to be (somewhat) sure that your fields schema matches what you’re expecting.
Re: "We ran out of columns"
#24Honestly, I like to work on such systems because:
- there is so much stuff to improve. There’s nothing better than the feeling of improving things with code (or by removing it)
- it’s back to school again and everything goes. You can implement features in any way you want because the constraints the system imposes. Granted, sometimes it’s painful to add functionality
- there’s usually no room to subjective topics like clean code and architecture. The most important thing with these systems is correctness (and this is usually an objective topic)
- nobody can blame you for something that doesn’t work. It’s always the fault of the legacy system
I wouldn’t recommend working on such systems to junior engineers, though.
I don’t really like to work on “perfect” codebases where everyone follows the same pattern, with linters, where if something breaks is because of your shitty code (because the codebase is “clean”). It’s very frustrating and limiting.
Re: "We ran out of columns"
#25> All that remained were ragtag interns and junior developers.
For many people, their first job in software engineering is the worst codebase they will deal with professionally for this reason. The first job hires lot of people with little/no experience. As soon as someone gains some experience than can move on to better paying jobs, where there are better developers with better standards.
Re: "We ran out of columns"
#26> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…
I actually love your approach and haven’t thought of that before. My problem with relational databases often stems from the fact that remodeling data types and schemas (which you often do as you build an application, whether or not you thought of a great schema beforehand) often comes with a lot of migration effort. Pairing your approach with a „version“ field where you can check which version of a schema this rows d…
Having to write and perform migrations for every small schema change is a bore, but it means your software doesn't have to worry about handling different versions of data. Going "schemaless" with version numbers means moving code from "write-and-forget" migrations to the main codebase, where it will live forever.
I think not doing database migrations only makes sense when you can make do without version numbers (or if you can't do atomic migrations due to performance constraints, but that's only a problem for a very small number of projects).
Re: "We ran out of columns"
#27I had dubious pleasure of working with similar codebases and devs. I'll remember one of those guys forever, because whenever he wanted to work on a new branch he would clone the repo, make changes to the master branch, and push code to a new repo numbered repo0001, repo002, ... He refused to change his ways, because "I have a PhD so you are wrong".
Another WTF moment was realisation that MS SQL Server does not support BOOLEAN type. That made porting code fun.
Re: "We ran out of columns"
#28> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…
I’m still in the think hard about the schema camp. I like to rely on the database to enforce constraints.
Yeah, a good database is pretty damn handy.
Have you had the pleasure of blowing young minds by revealing that production-grade databases come with fully fledged authnz systems that you can just...use right out of the box?
Re: "We ran out of columns"
#29Earlier quoted context omitted.
I actually love your approach and haven’t thought of that before. My problem with relational databases often stems from the fact that remodeling data types and schemas (which you often do as you build an application, whether or not you thought of a great schema beforehand) often comes with a lot of migration effort. Pairing your approach with a „version“ field where you can check which version of a schema this rows d…
Having to write and perform migrations for every small schema change is a bore, but it means your software doesn't have to worry about handling different versions of data. Going "schemaless" with version numbers means moving code from "write-and-forget" migrations to the main codebase, where it will live forever. I think not doing database migrations only makes sense when you can make do without version numbers (or i…
> Not having to write and perform migrations for every small schema change is a bore, but it means your software doesn't have to worry about handling different versions of data.
Is that "not" at the front supposed to be there?
Re: "We ran out of columns"
#30> I miss that direct connection. The fast feedback. The lack of making grand plans. There's no date on this article, but it feels "prior to the MongoDB-is-webscale memes" and thus slightly outdated? But, hey, I get where they're coming from. Personally, I used to be very much schema-first, make sure the data makes sense before even thinking about coding. Carefully deciding whether to use an INT data type where a BYTE…
I actually love your approach and haven’t thought of that before. My problem with relational databases often stems from the fact that remodeling data types and schemas (which you often do as you build an application, whether or not you thought of a great schema beforehand) often comes with a lot of migration effort. Pairing your approach with a „version“ field where you can check which version of a schema this rows d…
> remodeling data types and schemas (which you often do as you build an application, whether or not you thought of a great schema beforehand)
This is not my experience, it only happens rarely. I’d like to see an analysis of what causes schema changes that require nontrivial migrations.