Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I love databases

medium.com

Post reply on HN
Print view View original

91–100 of 172 posts

Re: Why I love databases

#91

Post by boyter »

Pxtl wrote: post #83

I love databases, but I loathe SQL. And no, I don't mean NoSQL is better - that's throwing out the baby with the bathwater. To me, SQL is the Common Lisp of relational languages - a brilliant invention of its time that has since long-overstayed its welcome and should be replaced by modern considerations of the problem it solves. The difference is that there are a million rethinks and descendents and redesigns of LISP…

The language issue you raise is in part solved by C# and Entity framework.

Because of the lazy way it works you can define a single select over a table, then apply extension methods, joins etc.. over the return of this select and you end up with a highly reusable clean easily tested data layer. Done right you get code like the following,

  var url = _urlRepo.GetUrl().OrderByCreateDate().ByCreatedUser("boyter");
  var url2 = _urlRepo.GetUrl().OrderByCreatedUser().ByDomain("geocities");
Or with joins,

  var users = _userRepo.GetUser();
  var locations = _locationRepo.GetLocation();

  var result =  from user in users
              join location in locations on user.locationid equals location.id && location.name = "Parramatta"
              select user;
I wrote about this a bit here http://www.boyter.org/2013/07/clean-repository-data-access-c... and it is to date the cleanest way I have come across in any language to deal with SQL. Compile time checked, reusable and testable.

Its the best ORM I have found so far across any language. I would dearly love to have it ported to other languages and frameworks.

Re: Why I love databases

#92

Post by marknadal »

zzzcpan wrote: post #81
marknadal wrote: post #32

I'm glad he loves databases, databases have been the bane of my existence. However, the torment they have given me has also lead to a similar fascination - and now I'm writing my own database! So I've become very familiar with the topics he writes on, and they are very good points for anybody interested in the subject. Why would I write my own database? Because databases are hard, and I am determined to make them eas…

It's always fun the first time. Good luck. Some notes from my experience implementing AP: - Easiest way to support scaling out is pre-sharding an entire database at the beginning, so there will be no need to ever reshard it. Like have 16 shards spread manually across all nodes and put into configs which nodes synchronize with which. Once you add another node simply move some shards to that node and be done with it. -…

Thanks! Although I would like to disagree with some of your points:

- Sharding is only necessary for centralized systems, where there exists some "Single Source of Truth" of the record. GUN can and will store things in multiple places, so it behaves more like BitTorrent. This reduces the complexity and thus makes the database significantly easier to use and scale, however I had to solve some hard conflict resolution problems to get that to work.

- Yes, Merkle trees are awesome. But deletion is a very dangerous action in a database and should be avoided (as noted in the original article). It causes all sorts of potential data-corruption/divergence problems.

- All data syncs in realtime, cause GUN sports push notifications. However, if the there is a network partition, the data will sync (without problems) after it is restored. So yes, agreed, don't do things periodically - do them immediately!

- Disk access is slow, correct. That is why you should keep as much of the working set in memory. Doing so minimizes disk access, so the majority of your write/read ops are lightning fast.

Do you have any links to your stuff?

Re: Why I love databases

#93

Post by jeffdavis »

pavlov wrote: post #28

I hate databases. People tend to have way too much faith in them (or their surrounding marketing), and thus make poor database choices that don't actually fit the shape of their data. Persistence is fundamentally the programmer's responsibility; a magic box behind a socket can't design it for you. Most applications I've seen wouldn't even need a database, but apparently a lot of programmers are conditioned into belie…

RDBMS have completely dominated in a way that few other technologies have. Especially for anything that might be described as business data (which is the prime value market for databases). Everything else is relegated to share a small niche. If you count SQLite as an RDBMS, the domination is even more complete.

So, my question to you is: why?

It can't be some kind of vendor lock-in, because there is no one vendor; there are several commercial vendors and open source projects available to choose from (each individual vendor might lock in their users, but that doesn't explain why they are still all RDBMSs and not something "better"). It can't be legacy, because SQL is as popular as ever; perhaps more so, as it seems that using a normal filesystem to store data is not very common any more.

So you need to explain why they are so popular yet so wrong; otherwise it just sounds like a case of "everyone is dumb except me".

Re: Why I love databases

#94

Post by marknadal »

sontek wrote: post #90
marknadal wrote: post #32

I'm glad he loves databases, databases have been the bane of my existence. However, the torment they have given me has also lead to a similar fascination - and now I'm writing my own database! So I've become very familiar with the topics he writes on, and they are very good points for anybody interested in the subject. Why would I write my own database? Because databases are hard, and I am determined to make them eas…

The commit messages on that repo have me worried!

Haha, sorry about that. I'm in the middle of cleaning up the directory structure to make things more coherent.

Try out some of the examples and the readme tutorial and let me know your thoughts. It is MVP status, and I'm trying to add more docs and get things ready to be stable.

Re: Why I love databases

#95

Post by NDizzle »

boyter wrote: post #91
Pxtl wrote: post #83

I love databases, but I loathe SQL. And no, I don't mean NoSQL is better - that's throwing out the baby with the bathwater. To me, SQL is the Common Lisp of relational languages - a brilliant invention of its time that has since long-overstayed its welcome and should be replaced by modern considerations of the problem it solves. The difference is that there are a million rethinks and descendents and redesigns of LISP…

The language issue you raise is in part solved by C# and Entity framework. Because of the lazy way it works you can define a single select over a table, then apply extension methods, joins etc.. over the return of this select and you end up with a highly reusable clean easily tested data layer. Done right you get code like the following, var url = _urlRepo.GetUrl().OrderByCreateDate().ByCreatedUser("boyter"); var url…

It's funny you mention entity framework. That's what blows up tempdb on one of my many environments with some of the stupidest queries I've ever seen hit the server.

Maybe your experience with entity is better than mine, but I personally think it and most other ORMs are complete garbage.

Re: Why I love databases

#96

Post by threeseed »

takeda wrote: post #59
daigoba66 wrote: post #27

Earlier quoted context omitted.

Postgres, and MSSQL if you can afford it, are the Swiss Army Knives of database systems. And I mean that in a good way.

For a second I thought you said MySQL and was a bit puzzled. I know indirectly that MSSQL is quite good database, but since you are putting Open Source and proprietary database next to each other it makes me wonder why you picked MSSQL and for example skipped its competitor such as Oracle. This is an honest question, I know very little about MSSQL or Oracle so I'm curious why you picked this one. I know that at least…

Everyone avoids MSSQL, Oracle, DB2, Teradata etc these days not for any technical reasons.

It purely comes down to cost. Both in the short term and the long term. Even in enterprises its becoming hard to justify the vendor, consulting and hardware costs.

Re: Why I love databases

#97

Post by Pxtl »

boyter wrote: post #91
Pxtl wrote: post #83

I love databases, but I loathe SQL. And no, I don't mean NoSQL is better - that's throwing out the baby with the bathwater. To me, SQL is the Common Lisp of relational languages - a brilliant invention of its time that has since long-overstayed its welcome and should be replaced by modern considerations of the problem it solves. The difference is that there are a million rethinks and descendents and redesigns of LISP…

The language issue you raise is in part solved by C# and Entity framework. Because of the lazy way it works you can define a single select over a table, then apply extension methods, joins etc.. over the return of this select and you end up with a highly reusable clean easily tested data layer. Done right you get code like the following, var url = _urlRepo.GetUrl().OrderByCreateDate().ByCreatedUser("boyter"); var url…

I use entity framework. Like any ORM it has some abstraction leaks and a few warts, but all in all it only cemented my opinion above that SQL is holding RDBMS systems back.

Re: Why I love databases

#98

Post by jeeyoungk »

read wrote: post #7

> You cannot give up partition tolerance. It'd be more accurate to say you don't want to give it up all the time. You don't want CAP; you want PACELC. http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-an...

This is an interesting categorization - somewhat like adding an axis to the political compass. Thanks.

Re: Why I love databases

#100

Post by pjmlp »

threeseed wrote: post #46
Slackwise wrote: post #12

I've always loved databases, but after having discovered write-only timestamped databases like Datomic, I can't imagine going backwards. It's a real shame that Datomic isn't fully open source. (Aren't BigTable and Spanner also write-only and timestamped?)

> Datomic isn't fully open source Couldn't agree more. I've worked at two Clojure enterprises recently and both really wanted to use Datomic. But in this day and age nobody wants to commit their data to another proprietary database.

Maybe it is more about paying and less about being open source?
Post reply on HN