They migrated to MongoDB from Postgres. Specifically because they needed distributed sharding which Postgres didn't have.
Jorge Ortiz, one of the developers at Foursquare said it best on twitter yesterday:
"Baffled by the Mongo haters. If foursquare had stayed on Postgres, we would've had to write our own distributed sharding/balancing/indexing."
"Two problems: 1) Our job is to build foursquare, not a database. 2) These things are hard. Odds are we would have had even more downtime."
http://twitter.com/#!/jorgeortiz85/status/26563381834
http://twitter.com/#!/jorgeortiz85/status/26563387808
Which sums it up incredibly well.
Your comment shows a scary lack of depth - this is the "dirty little secret" of ANY application that needs serious scalable data performance. You need to put your data in RAM where it is quickly accessible.
As soon as you go to disk especially on a low I/O cloud box you are going to be in 'extremely slow' territory. This is why, even before the current NoSQL movement, everyone was using Memcached all over the place. Not because it was a fad but because they needed speed.
SQL Databases have RAM caches as well which give you a pretty similar behavior. And big surprise - if you throw more memory at them - they run faster! Because they cache things in memory! In this case I believe that what Eliot was highlighting was that Foursquare's performance tolerance requirements were such that going to disk was not an optimal situation for them.
There are plenty of applications out there using MongoDB without keeping it all in RAM; I've deployed and maintain several. The more memory you have certainly gives you better performance but it tends to be about a most frequently used cache.
It never "Shit the bed" here to use your sophomoric vernacular. As the post states, it started having to go to the disk after it surpassed the memory threshold which slowed things down... nothing "crashed" as far as the description states. But a read/write queue backlog on any database is likely to exhibit the same behavior.
Your postgres + memcached solution fixes what exactly? They would still need the same slabs of RAM to solve the problem, lest they go to disk on postgres and slow to a crawl the same way they did with MongoDB.