Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I learned from building a production database

maheshba.bitbucket.io

Post reply on HN
Print view View original

71–80 of 112 posts

Re: Things I learned from building a production database

#71

Post by notapenny »

_pastel wrote: post #47

> [16] Design as a team; implement as individuals. This is pithy. I'm curious - does anyone here follow this philosophy? How do you actually design as a team? A meeting for every single design decision? I'm used to the team finding general alignment on the problem in meetings, but then a single person coming up with the design, writing a doc, and circulating it for comments.

I think your example could be seen as designing as a team, its just one person who does the writing, but everyone who needed to be involved was involved.

On a smaller scale I've worked in teams where when someone picked up a story, for certain cases we would first do a rough outline of the implementation and then take that to one or two devs from the team, just to talk about how to tackle the problem. Not a meeting, just five minutes of "here's what I think would work". Its hard but good alignment before writing code really helps with finding the errors in a design.

Re: Things I learned from building a production database

#72

Post by bad_username »

_pastel wrote: post #47

> [16] Design as a team; implement as individuals. This is pithy. I'm curious - does anyone here follow this philosophy? How do you actually design as a team? A meeting for every single design decision? I'm used to the team finding general alignment on the problem in meetings, but then a single person coming up with the design, writing a doc, and circulating it for comments.

I understood this advice as "avoid doing parallelized design work", where "team" means a single coordinated actor rather than a bunch of individual streams. I don't think the emphasis here is collective design.

Re: Things I learned from building a production database

#73

Post by Izkata »

Avalaxy wrote: post #17
otras wrote: post #9

Earlier quoted context omitted.

IC is a commonly used acronym (at least at some of the big tech companies) for Individual Contributor. For instance, when the author says "customer ICs", I imagine that means the internal engineers that would be using the database they created (individual contributors on other teams who would be customers/clients of this service). (Vouched for this flagged comment after seeing the other comment)

> IC is a commonly used acronym for Individual Contributor Its not though. Outside of hacker news I have literally never heard this term before, nor does it make any sense. If I'd start talking about "Individual Contributors" to my colleagues they would all be confused wth I'm talking about and wondering what's wrong with me. Who even coined this stupid term, and why do we pretend we need a new name for "engineers"?

Even for someone like me who is on that track, it wasn't coming to mind when I saw "IC". I had no idea what it meant.

Re: Things I learned from building a production database

#74

Post by vincnetas »

bbulkow wrote: post #59

There is a critical element the post doesn't state: a successful small project has (has) very limited scope of customers and use patterns. That is where complexity in writing databases arises: limited scope databases are far easier.

Its on the list :

  [2] Be careful to have the right number of customers (in the beginning, just one) and the right customers (whose requirements allow you to build out key technology); and grow that number carefully.

Re: Things I learned from building a production database

#75

Post by vincnetas »

This list is amazing for most software that i worked on. Dropping "database" from list name would only remove couple of items from the list. Most of the items are applicable much broader than just for database projects. Favourited for sure for future references.

Re: Things I learned from building a production database

#76

Post by alex64 »

This is a good list, but I'm trying to understand the following in better detail:

> [12] Be conservative on APIs and liberal with implementations.

> [18] Maintain multiple implementations in test for APIs; compare results between them. The cost is worth it (it will help with correctness, and also prevent leakage of implementation detail).

The word implementation is thrown out a lot. What does implementation mean in these quotes? The databases used (ie. dependency injection), the endpoints (ie. /user, /user/123), or something else?

I'm trying to visualize this in my head and am having trouble.

Re: Things I learned from building a production database

#78

Post by buro9 »

alex64 wrote: post #76

This is a good list, but I'm trying to understand the following in better detail: > [12] Be conservative on APIs and liberal with implementations. > [18] Maintain multiple implementations in test for APIs; compare results between them. The cost is worth it (it will help with correctness, and also prevent leakage of implementation detail). The word implementation is thrown out a lot. What does implementation mean in t…

Once people use your APIs you cannot trivially modify the bits being used or take things away.

A large API likely has complexity, security implications, messy transaction boundaries, etc... keep it small and simple, think of UX of an API, make it intuitive and capable.

That's the conservative bit... now the liberal bit.

Make clients, make one in Python and look at date parsing, make one in Go and consider struct re-use and how to deal with sets/arrays of things, make one in JavaScript and consider how easy it it to re-use parts of the code. If you have an endpoint that returns a list of things... is it supposed to be paginated over? How easy is that? Is it an infinite list? How easy is that? What would a client want to achieve? Is it possible?

When you produce prototype applications in a variety of languages the majority of the small inconsistencies and issues are super obvious.

And remember, once the API is out there it's hard to change it and you can't remove things easily... so discovering all of this early helps you get it more right than wrong the first time round.

Re: Things I learned from building a production database

#79

Post by vishnugupta »

scrubs wrote: post #29

Wow! One hell of a good list. And well explained. You bet I book marked it. Heck even the 1-connected comment was worth the read. Because it's true. I'm dealing with that right now. +5 and another reason I love HN. Now, if it you would have made it 43 items, a nice prime number, then I would be jealous to the point of angry (that's a riff on John Daily).

> if it you would have made it 43 items, a nice prime number,

Something tells me the author deliberately stopped at 42nd item.

https://hitchhikers.fandom.com/wiki/42

Re: Things I learned from building a production database

#80

Post by bottled_poe »

dalanmiller wrote: post #77

> [12] Be conservative on APIs and liberal with implementations. Curious how people interpret this? Is it: * Attempt to keep our API small while also... * thinking widely about the way it's used? Or how does this work in practice?

I’d interpret this to mean keeping you API specification basic to minimise backwards compatibility problems and reduce future refactoring difficulty. Don’t have special methods for unique cases, stick to restful principles where possible, keep data structures simple, etc.
Post reply on HN