Goodbye MongoDB, Hello PostgreSQL • Hacker News Board

Re: Goodbye MongoDB, Hello PostgreSQL

#131

Post by bau5 » Tue, Mar 10, 2015, 7:25 PM UTC

wainstead wrote: post #126

bau5 wrote: post #57

Earlier quoted context omitted.

No, this is bad. You shouldn't design a system that won't work from the outset. "Plan one to throw away" is about budgeting time, not about knowingly making big technical compromises when you write the first version.

I had to look the quote up, because it didn't sound quite right. "Plan to throw one away; you will anyway." The quote is Fred Brooks from his book "Mythical Man Month." It seems to me the author's post reflects that quote. Granted they didn't throw the entirety of every application out, but arguably it's an entirely different application now. I would even wager there's months of refactorings ahead of them as they tak…

I think the author of the post has the right approach. They tried hard to build something that would work right, and since it didn't, they rewrote most of it. They had enough budget and willingness to rewrite it. The author learned from their mistakes, and probably won't be using MongoDB on a future project.

The author of the comment I'm replying to seems to advocate a different approach where you choose tech and techniques that are good for short-term but maybe not long-term. This way you have planned technical debt plus unplanned technical debt. Also the comment author seems to be missing one of the post author's main points, which is that MongoDB's schemalessness is awkward even in the early stages of the app, and this is worked around by a less-than-ideal technique of moving the schema from the database layer to the application layer.

Re: Goodbye MongoDB, Hello PostgreSQL

#132

Post by tannerj » Tue, Mar 10, 2015, 7:26 PM UTC

pikzen wrote: post #122

tannerj wrote: post #102

So let me ask a question. What should I use when I do need a schemaless database? Is NoSQL never the answer? I've got a project that needs to allow clients to create registration forms for different events that my company hosts. A lot of the registration data will have a defined shema ex: name, email, address. I feel like that stuff should go in a RDMS, but all the event specific stuff needs to be schemaless. I know…

> Is MongoDB useless as a database, or are people being bitten for thinking it's a silver bullet and throwing it at every problem? A bit of both really. NoSQL databases allow for rapid prototyping, as do weakly and dynamically typed languages. It's amazing if you want to just get a product out of the door. NoSQL is the short term answer. And MongoDB is the answer if writing your data to /dev/null feels like a good id…

Yea, I'm going to check out PostgreSQL's json storage.

>But at the end of the day, what matters is that your product works.

With my requirements, I could just write to a flat file and be fine... I seem to like complicating things just enough that I no longer understand how what I'm building works. LOL.

Re: Goodbye MongoDB, Hello PostgreSQL

#133

Post by goshx » Tue, Mar 10, 2015, 7:26 PM UTC

Am I alone in thinking that if a programmer writes a code that allows a string to be sent to an integer field in the database the issue is not with the DBMS?

Re: Goodbye MongoDB, Hello PostgreSQL

#134

Post by joslin01 » Tue, Mar 10, 2015, 7:27 PM UTC

xyzzy123 wrote: post #77

distantsounds wrote: post #60

Earlier quoted context omitted.

Really, you can't spend the 10 minutes designing a table structure in a SQL database? And now you have to spend months re-inventing the wheel because you wanted an easy out? This post reflects on developers being lazy, instead of doing it right the first time around. Oh no, you have to log in to the db and run a CREATE TABLE statement every few months when you need to scale. Cry some more. And even then, 'lazy' is su…

Actually, I think that's a great way to think about it: NoSQL is the "dynamic typing" of the database world. Put another way, it's like "what? you couldn't spend 10 minutes declaring types everywhere?" - yeah, it's less robust, yet dynamically typed languages remain popular. My excuse: When I'm just past the mock stage, and still playing with what UI functionality should be, sometimes I just want to get some JSON per…

I read this article and was just like "ya. types are important" and reflected on how I used to love Ruby, but now swear against it for serious projects. Glad to see I'm not the only one who made the analogy in their heads.

Right now though, I have to write a script to pull some crap out of an Excel file and into my database. I use Scala with the Play framework, but what a nightmare getting typed information out of an Excel sheet is. No thanks, I'll just pull it out magically with Ruby and construct the JSON requests to my real server to create the data. It's a smart business decision and I would defend it all day. I'm sure there's corollaries for "dynamic databases" such as MongoDB.

To be clear though, I'm with the grandparent. I really think decisions for MongoDB are made out of a combination of laziness and eagerness most of the time. I mean honestly, if your data is going to be relatable, why would you employ a scheme that forces you to make those relationships manually? So you go to a relational databases which ironically lacks relationships, but hey there's plenty of support and libraries that will do it for you. But really if you had to ask me my personal opinion? Graph databases where relationships are FIRST CLASS CITIZENS are typically the best for serious projects containing lots of relationships.

Re: Goodbye MongoDB, Hello PostgreSQL

#135

Post by ad_hominem » Tue, Mar 10, 2015, 7:30 PM UTC

bigpeopleareold wrote: post #124

YorickPeterse wrote: post #85

Earlier quoted context omitted.

It's not so much about not wanting to write/understand SQL (both are still very much required), but about composability. If you want to re-use bits of a SQL query written as a string literal your only option is string concatention or using some kind of string builder/template system. In both cases there's little validation of the query's correctness (syntax wise) until you actually run it. While I agree that many ORM…

I once agreed with this, but now I don't. I just want to write SQL (dammit!). I can never, ever remember the intricacies of the Sequel API or any one of these query builder APIs. I am always looking up something that is rather trivial because I am thinking in SQL, the language, and always have to convert back to Ruby or whatever language I am working in. CTEs and SQL functions in PostgreSQL strike a good balance in t…

Let's say you have a product search screen in your application. There's a text field for filtering on product title (WHERE title LIKE), one for filtering on UPC (WHERE upc LIKE), a couple range filters for min/max prices (WHERE price =), and then on the results screen the user sort on a few different columns (ORDER BY) as well as paginate and set number of results per-page (LIMIT + OFFSET).

How exactly are you going to just write SQL if the actual query statement needs to change based on the user input? Are you going to hardcode every possible permutation, or are you going to use a query builder? Or are there some advanced Postgres features that I'm not aware of that would allow all these combinations in some kind of prepared statement?

edit: The question I'm asking is pointed towards the people who are implying that you can "just use SQL" as static statements that are not dynamically assembled. Like a static function or prepared statement that takes some parameters, and at runtime you only pass in those parameters - not rejigger the actual SQL statement fragments. Your ad-hoc query builder implementations with SQL injection vulnerabilities are not relevant to what I'm asking.

Re: Goodbye MongoDB, Hello PostgreSQL

#136

Post by kstirman » Tue, Mar 10, 2015, 7:32 PM UTC

super_sloth wrote: post #2

Is there anybody here who has run MongoDB at moderate scale with good results? As in a few terabytes of data, >10k ops/second territory. I've been really disappointed with its reliability and performance in situations where I've been around that.

High scale public references and links to their presentations are maintained here: http://www.mongodb.com/mongodb-scale

I think scaling any system takes careful planning.

IMHO, this is an area where MongoDB has improved a lot in the past two years, especially with 3.0, but there is still a lot of work to do.

Re: Goodbye MongoDB, Hello PostgreSQL

#137

Post by mbreese » Tue, Mar 10, 2015, 7:34 PM UTC

YorickPeterse wrote: post #85

jgrahamc wrote: post #13

As a greying developer I am most amused by people discovering that 'old' technologies like SQL databases work really well. The only useful piece of advice I can give a younger developer is... be careful when drinking the newtech koolaid. And one more thing: star = Sequel.lit('*') User.select(:locale) .select_append { count(star).as(:amount) } .select_append { ((count(star) / sum(count(star)).over) * 100.0).as(:percen…

It's not so much about not wanting to write/understand SQL (both are still very much required), but about composability. If you want to re-use bits of a SQL query written as a string literal your only option is string concatention or using some kind of string builder/template system. In both cases there's little validation of the query's correctness (syntax wise) until you actually run it. While I agree that many ORM…

How often do you really reuse bits of a SQL query? SQL is the language that an ORM will generate. The question is, how much work is done to avoid using SQL and is it worth it?

The only time that I've found ORMs useful for composing queries when I have to dynamically create a query at runtime based upon user input. And even in that case, today, I'd probably still just concatenate strings for a proper SQL statement.

Re: Goodbye MongoDB, Hello PostgreSQL

#138

Post by DonnyV » Tue, Mar 10, 2015, 7:35 PM UTC

As soon as you have more then 1 app talking to your database its time to either create a common library that talks to your database that can be shared or wrap it in a service. This has nothing to do with NoSQL vs SQL. It has to do with architecture composition.

Plus your problem with checking if a field exists is because Ruby doesn't support property attributes. This is easily solved in C# using attributes. Which you can fake in Ruby. http://stackoverflow.com/questions/1085070/how-do-i-fake-c-s...

Data consistency is what your models are for. No data should be inserted before being assigned to a model.

Again it sounds like you needed a shared library or just a service wrapped around your database.

Re: Goodbye MongoDB, Hello PostgreSQL

#139

Post by digsmahler » Tue, Mar 10, 2015, 7:36 PM UTC

tannerj wrote: post #127

digsmahler wrote: post #114

Earlier quoted context omitted.

MongoDB is a great database and has a really good set of client tools. It has a learning curve and it has not been without problems but I have loved it at a past startup and would absolutely use it again. If you try it out and find you like it, I'd really suggest getting to one of their MongoDB seminar days. They tend to have good speakers and for sure you'll learn something new about databases and MongoDB.

Yea, from the good stuff I've read about MongoDB I really want to like it. There's just been so much more negative that I've read.

Every choice you make in software has pros/cons. This is true of all database technologies as well. You can probably solve your problem using any of the choices before you. If you decide to use something new (e.g. MongoDB, Riak, whatever) first make sure it lines up with your requirements then see how it goes. Always keep backups. In the worst case, you'll restore and migrate to something else. But that's kind of what we do as an industry anyways.

Re: Goodbye MongoDB, Hello PostgreSQL

#140

Post by YorickPeterse » Tue, Mar 10, 2015, 7:40 PM UTC

bgentry wrote: post #7

In the future we might also move our Rails applications over to Sequel, but considering Rails is so tightly coupled to ActiveRecord we’re not entirely sure yet if this is worth the time and effort. Actually, with modern versions of Rails, using Sequel in place of ActiveRecord isn't bad at all. Nothing in Rails is really tied to ActiveRecord anymore. There are dependencies on ActiveModel, but you can easily make Seque…

While certainly possible we simply haven't really evaluated it yet in depth. In between releasing a bunch of upcoming features and upgrading Rails from 3.2 to 4.2 I'd rather wait with _also_ moving from ActiveRecord to Sequel for the time being.