Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling lessons learned at Dropbox (2012)

eranki.tumblr.com

Post reply on HN
Print view View original

1–10 of 32 posts

Re: Scaling lessons learned at Dropbox (2012)

#4

Post by yaseer »

This may be from 2012, but it's surprising just how much of the specific advice still applies.

Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades.

Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

Re: Scaling lessons learned at Dropbox (2012)

#5

Post by gjkood »

yaseer wrote: post #4

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades. Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

Couldn't agree more!

When I am forced to use a Windows environment I find myself reaching out to these evergreen tools in many situations and is frustrated by having to find workarounds to these tools that should be ubiquitous tools in any OS.

Everything I learned almost 30 years ago still holds true today when it comes to the basics.

And yes I know of Cygwin, MKS toolkit etc, but trying to convince corporate IT in a Microsoft Centric world to install these usually hits a brick wall.

Re: Scaling lessons learned at Dropbox (2012)

#6

Post by charlieflowers »

yaseer wrote: post #4

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades. Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

What are some outstanding resources to learn from?

Re: Scaling lessons learned at Dropbox (2012)

#7

Post by ______-_-______ »

gjkood wrote: post #5
yaseer wrote: post #4

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades. Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

Couldn't agree more! When I am forced to use a Windows environment I find myself reaching out to these evergreen tools in many situations and is frustrated by having to find workarounds to these tools that should be ubiquitous tools in any OS. Everything I learned almost 30 years ago still holds true today when it comes to the basics. And yes I know of Cygwin, MKS toolkit etc, but trying to convince corporate IT in a…

Thankfully git is everywhere these days, and git on windows comes with most of the coreutils. I hope IT isn't telling you you're not allowed to use source control. Microsoft owns GitHub after all

Re: Scaling lessons learned at Dropbox (2012)

#8

Post by maxclark »

"One technique we repeatedly used was creating artificial extra load in the live site. For example, we would do a lot more memcached reads than necessary. Then when memcached broke, we could quickly switch off the duplicate queries and have time to come up with a solution."

As much as I want to hate this, it's actually genius and elegant in its simplicity.

Re: Scaling lessons learned at Dropbox (2012)

#9

Post by PeterWhittaker »

charlieflowers wrote: post #6
yaseer wrote: post #4

This may be from 2012, but it's surprising just how much of the specific advice still applies. Grep, sed, awk, xargs etc are still the canonical list of shell tools for debugging, and have been for decades. Makes you realise, learning Unix tools will likely be a better long term time investment than the latest trendy framework. I wish I could go back in time and tell that to my younger self...

What are some outstanding resources to learn from?

Honestly, I think the best way to learn these tools is to use them. After starting at BNR in 1990, I sat for a couple of days with Sobell's Practical UNIX on my lap, doing every example and every exercise; since them, I've stayed at the command line as much as possible. I've looked periodically for other, more recent resources, at least in part to answer questions such as yours, but I've never found anything that packed the punch of that early Sobell, not even later Sobell's.

If you want to direct the learning, though, I'd recommend figuring out how to do something with grep, e.g., then do the same with sed, awk, cut, etc.

For example:

grep myuserid /etc/passwd

sed -n '/myuserid/p' /etc/passwd

awk '/myuserid/ { print }

etc.

Then make it more interesting: repeat those with 'grep -I MYUSERID /etc/passwd'.

Then make it even more interesting: print the 4th field from each matching line. (Hint: man cut.) Come up with at least two or three different ways of doing this with regex's, then make those regex's as compact as possible.

Then make it a lot more interesting: print the two lines before and the three lines after each pattern.

Once you do this for a while, you will get used to thinking in terms of filters and regex and things will start to come more naturally for you.

Also consider doing all of those exercises from within vim, use :g... commands (or any other alternative you can think of).

Post reply on HN