Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I love databases

medium.com

Post reply on HN
Print view View original

121–130 of 172 posts

Re: Why I love databases

#121

Post by thorin »

matwood wrote: post #68
takeda wrote: post #59

Earlier quoted context omitted.

For a second I thought you said MySQL and was a bit puzzled. I know indirectly that MSSQL is quite good database, but since you are putting Open Source and proprietary database next to each other it makes me wonder why you picked MSSQL and for example skipped its competitor such as Oracle. This is an honest question, I know very little about MSSQL or Oracle so I'm curious why you picked this one. I know that at least…

MSSQL can be a great value for the money if you need all of the additional tools (SSIS, MSAS), and you are already in a Windows environment. For a straight rdbms, pg is a great solution. Oracle is something you only use when you have reached some problem that only Oracle can solve. Even then, I would still try to work around Oracle. It is expensive and requires a large amount of tuning to make work well. Oh, and pret…

I've used Oracle since 1998 and SQLServer since early 2000s. Oracle has great defaults now and the free version is very good. Licenses for MSSQL and Oracle can be competitive - obviously if you are a MS shop SQLServer will become cheap(er) in comparison. I prefer Oracle because I know it better and I sometimes work with *nix. PL/SQL packages are far better than the comparable stored procedures in MSSQL. APEX is pretty unique too.

If I was starting on an open source or startup project with a modern web framework I definitely be looking at using Postgres - it looks great.

Re: Why I love databases

#122

Post by sklogic »

Pxtl wrote: post #83

I love databases, but I loathe SQL. And no, I don't mean NoSQL is better - that's throwing out the baby with the bathwater. To me, SQL is the Common Lisp of relational languages - a brilliant invention of its time that has since long-overstayed its welcome and should be replaced by modern considerations of the problem it solves. The difference is that there are a million rethinks and descendents and redesigns of LISP…

Use hierarchical DBMS is you need trees. Relational does not fit here.

Re: Why I love databases

#123

Post by al2o3cr »

marknadal wrote: post #32

I'm glad he loves databases, databases have been the bane of my existence. However, the torment they have given me has also lead to a similar fascination - and now I'm writing my own database! So I've become very familiar with the topics he writes on, and they are very good points for anybody interested in the subject. Why would I write my own database? Because databases are hard, and I am determined to make them eas…

"The beauty of this, though, is that you can always build strong consistency out of eventual consistency (it just requires knowing X amount of peers in advance, and doing a trivial lock until you've heard back from all of them)"

Sounds like a way to trade A for maybe-C, there - what happens if Godzilla stomps one of the servers you're waiting on?

Re: Why I love databases

#124

Post by zzzcpan »

marknadal wrote: post #92
zzzcpan wrote: post #81

Earlier quoted context omitted.

It's always fun the first time. Good luck. Some notes from my experience implementing AP: - Easiest way to support scaling out is pre-sharding an entire database at the beginning, so there will be no need to ever reshard it. Like have 16 shards spread manually across all nodes and put into configs which nodes synchronize with which. Once you add another node simply move some shards to that node and be done with it. -…

Thanks! Although I would like to disagree with some of your points: - Sharding is only necessary for centralized systems, where there exists some "Single Source of Truth" of the record. GUN can and will store things in multiple places, so it behaves more like BitTorrent. This reduces the complexity and thus makes the database significantly easier to use and scale, however I had to solve some hard conflict resolution…

I'm not gonna argue with your points, just gonna warn you, that you have too much confidence in your algorithms, be careful.

> Do you have any links to your stuff?

No, it's not open source. Maybe with next implementation I will finally be ready to share it.

Re: Why I love databases

#125

Post by sequence7 »

NDizzle wrote: post #95
boyter wrote: post #91

Earlier quoted context omitted.

The language issue you raise is in part solved by C# and Entity framework. Because of the lazy way it works you can define a single select over a table, then apply extension methods, joins etc.. over the return of this select and you end up with a highly reusable clean easily tested data layer. Done right you get code like the following, var url = _urlRepo.GetUrl().OrderByCreateDate().ByCreatedUser("boyter"); var url…

It's funny you mention entity framework. That's what blows up tempdb on one of my many environments with some of the stupidest queries I've ever seen hit the server. Maybe your experience with entity is better than mine, but I personally think it and most other ORMs are complete garbage.

Although I understand your frustration if SQL queries were blowing up tempdb would you blame SQL or would you blame the developers writing those stupid queries?

ORMs are not a magic bullet and developers can do stupid things with any tool.

Re: Why I love databases

#126

Post by bballard »

jbergens wrote: post #117
wowmsi wrote: post #56

Earlier quoted context omitted.

I would recommend "Architecture of a Database System" by Joe Hellerstein, Micheal Stonebraker, and James Hamilton [0] as a start if you want formal perspective without getting too much into theory. [0] http://db.cs.berkeley.edu/papers/fntdb07-architecture.pdf

As a sidenote, Michael Stonebreaker himself is nowdays building VoltDb which is a memory-db that he thinks is much more evolved than relational systems. I havn't used it but it looks interesting. I did try OrigoDb which is an open source .NET based memory db and it worked great for my needs and the queries are really fast.

(I work at VoltDB)

VoltDB is easy to try. There is a free 30-day trial available on voltdb.com, and a docker-based demo (http://voltdb.com/voltdb-demo).

Re: Why I love databases

#127

Post by edraferi »

sklogic wrote: post #38
bbcbasic wrote: post #34

Earlier quoted context omitted.

Hmm. You are not thinking in business terms. You run a software house: do you want your developers reinventing data storage on each application? Or using a fairly decent data storage that is RDBMS. Most of the time RDBMS is a very good choice. Think about the tooling, support, hiring knowledgeable people etc. Lets face it RDBMS are good at the very small (single table, replacing a text file) up to the very large. In…

Most of the time a plain text file is good enough. Databases are overrated.

JWZ wrote a great article about choosing between flat files and a database for Netscape Mail:

http://www.jwz.org/doc/mailsum.html

In Netscape 4.0, the new team went both C++ and Database happy, threw away the tightly-tuned mail summary files I had designed, and generally screwed the pooch raw.

My code had summary files that were on the order of 2% of the size of the folder they were summarizing, and was blazingly fast in all respects. The 4.0 code had an overhead closer to 30% (last time I checked) and was insanely slow, not to mention extremely fragile: their summary files got corrupted all the time.

Re: Why I love databases

#128

Post by arielby »

bbcbasic wrote: post #34
pavlov wrote: post #28

I hate databases. People tend to have way too much faith in them (or their surrounding marketing), and thus make poor database choices that don't actually fit the shape of their data. Persistence is fundamentally the programmer's responsibility; a magic box behind a socket can't design it for you. Most applications I've seen wouldn't even need a database, but apparently a lot of programmers are conditioned into belie…

Hmm. You are not thinking in business terms. You run a software house: do you want your developers reinventing data storage on each application? Or using a fairly decent data storage that is RDBMS. Most of the time RDBMS is a very good choice. Think about the tooling, support, hiring knowledgeable people etc. Lets face it RDBMS are good at the very small (single table, replacing a text file) up to the very large. In…

First-Order Logic (and therefore SQL, which is based on it) sucks at describing graphs and highly heterogeneous structures. If you're working with these, you should have a better representation.

Re: Why I love databases

#129

Post by SamReidHughes »

sklogic wrote: post #104
superuser2 wrote: post #71

Earlier quoted context omitted.

It depends on the problem domain you are working in. I can't think of an application I've done where this wouldn't have required complex serialization and parsing. Those tools are already built for me in the form of ORMs and RDMSes. Why should I write my own?

> It depends on the problem domain you are working in. Unix is a pretty general purpose thing. And yet, you won't find anything in it that needs a database, nothing at all among hundreds of applications, including some fairly complex ones, like CAD/CAE tools, IDEs, compilers, etc. > I can't think of an application I've done where this wouldn't have required complex serialization and parsing. Are you doing CRUD mostly…

> Parsing?!? In ORMs and DBMSes?!?

Yes. Stuff stored in files has to be parsed when you read it out of files.

Re: Why I love databases

#130

Post by SamReidHughes »

sklogic wrote: post #106
SamReidHughes wrote: post #82

Earlier quoted context omitted.

How do you randomly access variable length fields in a text file and modify them and add new ones?

Why would I want to randomly access variable length fields in a file, to start with? It's a very niche thing to do. Filesystem driver may want to do something like this, some large scale caching proxy, probably. In most cases it's just an optimisation (like a berkeley db records backing text files such as /etc/passwd).

> Why would I want to randomly access variable length fields in a file, to start with? It's a very niche thing to do.

If by "niche" you mean every interactive website does it.

Post reply on HN