Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"How hard can it be to implement?"

answers.37signals.com

Post reply on HN
Print view View original

21–30 of 102 posts

Re: "How hard can it be to implement?"

#21

Post by texel »

xentronium wrote: post #4
SMrF wrote: post #3

Earlier quoted context omitted.

From the response; "We can't use database transactions because performing a big move would slow Basecamp down for everyone. So we have to log the process of each step of the move, and make it so any failure in the move can be rolled back gracefully. That means a move is actually a series of copies and deletions instead of just changing a field for each moved item"

What's the point of having ACID when you don't use its advantages, one might wonder.

Well you're ignoring possible disadvantages, like locking an inordinate number of rows in several tables while a long-running copy takes place. Transactions are all well and good, until they run counter to the usability and stability of your app.

Re: "How hard can it be to implement?"

#22

Post by Groxx »

xentronium wrote: post #4
SMrF wrote: post #3

Earlier quoted context omitted.

From the response; "We can't use database transactions because performing a big move would slow Basecamp down for everyone. So we have to log the process of each step of the move, and make it so any failure in the move can be rolled back gracefully. That means a move is actually a series of copies and deletions instead of just changing a field for each moved item"

What's the point of having ACID when you don't use its advantages, one might wonder.

(in reply to all threads stemming from your comment when I posted this)

What's all this talk about "re-implementing" and "if" they have transactions in their DB? They explicitly state that they can't use transactions because it would slow things down, not because they don't have them. They also state that Since moving one milestone could potentially result in hundreds of database operations...

This means they have a transaction supporting database. There's no re-implementing onto a different system. There's an apparently-broken implementation.

Re: "How hard can it be to implement?"

#23

Post by mkramlich »

Great post but note that the total complexity/LOE of adding that feature would be significantly reduced if (1) it was delivered earlier in the app's lifecycle (less users, simpler code, lower expectations, less/no customers, etc), and/or (2) quality/UX standards were relaxed somewhat.

I agree that even a seemingly simple feature can be hard to deliver, but on the flip side there are dials a developer can turn to adjust his LOE up or down as desired. Tradeoffs as always.

Re: "How hard can it be to implement?"

#24

Post by smiler »

weaksauce wrote: post #9
thisisnotmyname wrote: post #2

Is there really no simpler way to solve this problem? "Moving a message needs to move all of the message's comments, and all of the comments' files, and all of the comments' files' versions." Why can't you just change some top level reference in the database? I'm imagining a Projects table and a TodoLists table. Each TodoList has something like a projectID foreign key right? Why can't you just change that and automat…

Might be trickier to do this if the foreign key is across a shard or if the db record for the comment wasn't structured like this. One situation could be because they store the threaded comments with the thread key in each record and also the parent comment key. This would be useful to pull down all the comments associated with a thread in one db query but also allow you to have the nested comments.

37signals don't use sharding. They use one database server with enough RAM to load the db into memory

Re: "How hard can it be to implement?"

#25

Post by edanm »

In the same vein, I highly recommend the article "How many Microsoft employees does it take to change a lightbulb?". This was features in Joel Spolsky's "The Best Software Writing 1" (http://www.amazon.com/gp/product/1590595009?ie=UTF8&tag=...).

Link to article: http://blogs.msdn.com/b/ericlippert/archive/2003/10/28/53298...

Re: "How hard can it be to implement?"

#26

Post by [deleted] »

Groxx wrote: post #22
xentronium wrote: post #4

Earlier quoted context omitted.

What's the point of having ACID when you don't use its advantages, one might wonder.

(in reply to all threads stemming from your comment when I posted this) What's all this talk about "re-implementing" and "if" they have transactions in their DB? They explicitly state that they can't use transactions because it would slow things down, not because they don't have them . They also state that Since moving one milestone could potentially result in hundreds of database operations ... This means they have…

[deleted]

Re: "How hard can it be to implement?"

#27

Post by amix »

thisisnotmyname wrote: post #2

Is there really no simpler way to solve this problem? "Moving a message needs to move all of the message's comments, and all of the comments' files, and all of the comments' files' versions." Why can't you just change some top level reference in the database? I'm imagining a Projects table and a TodoLists table. Each TodoList has something like a projectID foreign key right? Why can't you just change that and automat…

The strategy you usually take with scaling a relational database is sharding, partitioning and de-normalization. And given that Basecamp has millions of users I don't imagine their database structure to be pretty or normalized.

This said, if they have their database sharded by project_id then it would not be that big an issue, but it can be that their database structure is very complex or messed up...

Re: "How hard can it be to implement?"

#28

Post by josephholsten »

richcollins wrote: post #10

This would be easy in a graph db. Just move the node and its children come along for the ride.

Not exactly. If you look a bit closer, you'll see that this isn't just a strict child, otherwise they might have managed by swapping a foreign key or something. In a graph db, you'd still have to reconnect dangling edges that no longer make sense, like for comments and users. Also, the transaction rollback issue. Which is to say: tanstaafl.

Re: "How hard can it be to implement?"

#29

Post by josephholsten »

mkramlich wrote: post #23

Great post but note that the total complexity/LOE of adding that feature would be significantly reduced if (1) it was delivered earlier in the app's lifecycle (less users, simpler code, lower expectations, less/no customers, etc), and/or (2) quality/UX standards were relaxed somewhat. I agree that even a seemingly simple feature can be hard to deliver, but on the flip side there are dials a developer can turn to adju…

> quality/UX standards were relaxed somewhat

Yow.

Re: "How hard can it be to implement?"

#30

Post by jules »

smiler wrote: post #24
weaksauce wrote: post #9

Earlier quoted context omitted.

Might be trickier to do this if the foreign key is across a shard or if the db record for the comment wasn't structured like this. One situation could be because they store the threaded comments with the thread key in each record and also the parent comment key. This would be useful to pull down all the comments associated with a thread in one db query but also allow you to have the nested comments.

37signals don't use sharding. They use one database server with enough RAM to load the db into memory

One database server for the whole of basecamp?

Edit: I found this:

> With that in mind, we went looking for an option to host the Basecamp database, which is becoming a monster. As of this writing, the database is 325GB and handles several thousand queries per second at peak times.

325GB RAM?! But now they have multiple servers. Read more: http://37signals.com/svn/posts/2479-nuts-bolts-database-serv...

Post reply on HN