51–60 of 135 posts
After all these years, I thought the advantage and trade-off of different database should be well-understood. But the fact is, there are still lots of mis-infomation floating around.
It looks like the lesson we have learnt are not communicated to border groups of engineers.
Another consideration is that, at scale, no sql is way cheaper. I run a service with approx. 900k daily users, each generating about 210 object writes and reads that need to execute within 50ms, and I am running this on firestore for about $350 a month, incl. Elb, waf, regionally replicated compute, managed NLP and translate. I sync the no sql stuff to bigquery for analytical usage. Cheap, and scales without any prob…
Would it be 210 writes in a relational db though? Our product at work started on Firestone and one of the main problems was it was causing us to have to do far more operations than we would otherwise have needed. E.g in a relational db you can do UPDATE WHERE or DELETE WHERE in a single operation. I’m Firestone that’s a get followed by an update/delete for each record!
I guess it depends on your workload, but we’re the opposite: never doing Firestone again!
Earlier quoted context omitted.
Or you rewrite you queries so that you can use prepared statements. You’re right that we should avoid gluing strings together, but in most cases we can use prepared statements. There’s also a middleground where developers learn to use the ORM better. I’ve seem people get terrible performance using the Django ORM, but after a rewrite, redesigning the queries and using the more advanced features performance would impro…
Prepared statements are a good tool to have in your arsenal but for the majority of the queries you need an ORM will be fine. It doesn't absolve you of the need to understand SQL but it does save a huge amount of time and generally results in easier to follow code.
Great article, and I this particularly brought back memories for me: > [...] in some instances you might work with vast quantities of data, or deal with transactional systems that just don’t easily fit the operational limitations of relational databases. And in those cases, you should consider moving some, or all, of your data into a non-relational database. I've worked with a big application that utilised this appro…
> Our solution was to move the typing of this information to the application layer, and just store the arbitrary data in a NoSQL solution. This worked perfectly, and to my knowledge it's still working without a glitch. > This was before RDBMS solutions supported JSON, and if I were to do this again, I'd probably just continue to use MySQL, PostgreSQL or whatever, and store it as JSON in the database. Both solutions s…
YAGNI
Great article, and I this particularly brought back memories for me: > [...] in some instances you might work with vast quantities of data, or deal with transactional systems that just don’t easily fit the operational limitations of relational databases. And in those cases, you should consider moving some, or all, of your data into a non-relational database. I've worked with a big application that utilised this appro…
> Our solution was to move the typing of this information to the application layer, and just store the arbitrary data in a NoSQL solution. This worked perfectly, and to my knowledge it's still working without a glitch. > This was before RDBMS solutions supported JSON, and if I were to do this again, I'd probably just continue to use MySQL, PostgreSQL or whatever, and store it as JSON in the database. Both solutions s…
We were using MySQL, and MySQL didn't (doesn't?) have user defined types, so it was really not an option.
(This was back in 2013, btw.)
Schema, stability, performance, fault tolerance, and the full laundry list of things is available in either form.
Great article, and I this particularly brought back memories for me: > [...] in some instances you might work with vast quantities of data, or deal with transactional systems that just don’t easily fit the operational limitations of relational databases. And in those cases, you should consider moving some, or all, of your data into a non-relational database. I've worked with a big application that utilised this appro…
> Our solution was to move the typing of this information to the application layer, and just store the arbitrary data in a NoSQL solution. This worked perfectly, and to my knowledge it's still working without a glitch. > This was before RDBMS solutions supported JSON, and if I were to do this again, I'd probably just continue to use MySQL, PostgreSQL or whatever, and store it as JSON in the database. Both solutions s…
One of our addresses that caused trouble is literally: "The yellow sign across the street from the Seven-Eleven at ".
We have one address that's legally in two countries at the same time.
One address is just a whole city. Like the entirety of the city, but also it still needs to be considered a separate place from the city.
It of course depends on your use cases and etc. But I find addresses can be like storing names in a lot of contexts. i.e. Just take the bytes the user gives you and alert them if some service downstream complains, but don't require they change it to meet your requirements.
There are systems to help offer standardized addresses and you can display them as suggestions to the user. But sometimes you get a multi billion dollar company telling you "Maybe the address is legally X, but the bus stops at the yellow sign across the street, and we get 30 customers calling for refunds every week because they didn't get on the bus. So either accommodate this change or we'll need to find a new partner."
Earlier quoted context omitted.
> Our solution was to move the typing of this information to the application layer, and just store the arbitrary data in a NoSQL solution. This worked perfectly, and to my knowledge it's still working without a glitch. > This was before RDBMS solutions supported JSON, and if I were to do this again, I'd probably just continue to use MySQL, PostgreSQL or whatever, and store it as JSON in the database. Both solutions s…
I work with locations and addresses on an international system. They are not anywhere near as standardized as to allow this. And also at scale, you'll get things that seem like they shouldn't be addresses but are. One of our addresses that caused trouble is literally: "The yellow sign across the street from the Seven-Eleven at ". We have one address that's legally in two countries at the same time. One address is jus…