Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sysadmin mistakes start-ups make

cloudkick.com

Post reply on HN
Print view View original

61–70 of 95 posts

Re: Sysadmin mistakes start-ups make

#61

Post by teej »

jrockway wrote: post #49

Fork is actually a very fast system call. It never blocks, and (on Linux), only involves copying a very small amount of bookkeeping information. If you exec right after the fork, there is basically no overhead. However, forking a new shell to parse "mv foo bar" is more expensive than just using the rename system call. And it's easier to check for errors, and so on. SQLite is also not as slow as people think it is; yo…

> SQLite is also not as slow as people think it is; you can easily handle 10s of millions of requests per day with it

Scaling relational databases to the point of 10s of millions of requests is extremely non-trivial. Unless you can show me personally or can show me evidence otherwise, don't make this claim. You're doing a disservice to the people that have worked countless hours to eke every last millisecond of performance out of MySQL and Postgres.

Re: Sysadmin mistakes start-ups make

#62

Post by joshOiknine »

Personally I don't think we would ever run into those issues. A. we don't have other servers to switch over to B. We are using MySQL for testing and development and C. we don't like what happens when we make system calls for within a web app. forget about forking.

Re: Sysadmin mistakes start-ups make

#63

Post by andrewtj »

staunch wrote: post #18
aristus wrote: post #6

I disagree with 1.3. "Serving static content is the easiest possible task for any web server." Yes, but keeping connections open for slow clients (esp with KeepAlive on) is not a good use of your 500MB Mongrel process' time. On the other hand, KeepAlive is a handy thing to have. Using a proxy like nginx or varnish to serve static files (and even dynamic data) if you have the proper KeepAlive and Nagle bits flipped ca…

It's almost always a bad idea to use anything other than a non-blocking/async server to handle static content. I think it's simpler/easier (maybe faster) to serve content from a separate sub-domain (static.site.com or whatever). Using a reverse proxy works too, but unless you're caching dynamic content it's probably no benefit and it's less efficient.

A good reverse proxy will buffer client and server side so that your heavy app can be available to serve the next request whilst the light proxy feeds the page back to a slow client.

Under certain circumstances serving static files from separate hostnames can be beneficial as HTTP clients are supposed to limit the number of simultaneous connections per hostname.

Re: Sysadmin mistakes start-ups make

#64

Post by mmelin »

SwellJoe wrote: post #32
lsb wrote: post #29

Earlier quoted context omitted.

I'm getting 10k uniques a month and SQLite is working fantastically, as the db behind one Sinatra process, behind one Thin.

I'm getting 10k uniques a month I think I see your problem.

Also known as 10-15 hits an hour.

My microwave can probably serve that traffic.

Re: Sysadmin mistakes start-ups make

#65

Post by bodhi »

gaius wrote: post #37
michaelbuckbee wrote: post #30

I'd guess the real number one mistake is insufficient paranoia about backups. I know lots of companies doing TDD but that have never done a full test restore from their backups.

The easiest way to do this is to make your backups the mechanism by which you refresh your Dev/QA environment from Production. It means your Ops team are very nearly doing a DR exercise every week.

It's a great idea, we were doing that at my old company a few years ago. It also means that you have fewer problems with migrations when pushing them out to production.

Re: Sysadmin mistakes start-ups make

#66

Post by jrockway »

teej wrote: post #61
jrockway wrote: post #49

Fork is actually a very fast system call. It never blocks, and (on Linux), only involves copying a very small amount of bookkeeping information. If you exec right after the fork, there is basically no overhead. However, forking a new shell to parse "mv foo bar" is more expensive than just using the rename system call. And it's easier to check for errors, and so on. SQLite is also not as slow as people think it is; yo…

> SQLite is also not as slow as people think it is; you can easily handle 10s of millions of requests per day with it Scaling relational databases to the point of 10s of millions of requests is extremely non-trivial. Unless you can show me personally or can show me evidence otherwise, don't make this claim. You're doing a disservice to the people that have worked countless hours to eke every last millisecond of perfo…

10 million per day is about 100 per second. SQLite performs about this quickly. I wrote a test script and it did 125 (unindexed) lookups per second. Then I ran two of these tests at the same time, and the rate stayed about the same. I have 8 cores, so I made 8 processes, and it was the same. 125 requests/second * 8 * 86400 seconds/day = 86_400_000 requests per day.

I added another thread writing as quickly as possible to the mix (7 readers, 1 writer), and this brought the read rate down to about 45/reads per second per thread. Still more than 10 million per day, so technically I am right.

Also, I don't doubt that MySQL and Postgres (and BDB) are both significantly faster than this. It's just that SQLite is not going to guarantee "instant failure" of your project, as the article implies.

(One thing to note -- every time you type a character in Firefox's address bar, you are doing an SQLite query. It is Fast Enough for many, many applications.)

Re: Sysadmin mistakes start-ups make

#67

Post by teej »

jrockway wrote: post #66
teej wrote: post #61

Earlier quoted context omitted.

> SQLite is also not as slow as people think it is; you can easily handle 10s of millions of requests per day with it Scaling relational databases to the point of 10s of millions of requests is extremely non-trivial. Unless you can show me personally or can show me evidence otherwise, don't make this claim. You're doing a disservice to the people that have worked countless hours to eke every last millisecond of perfo…

10 million per day is about 100 per second. SQLite performs about this quickly. I wrote a test script and it did 125 (unindexed) lookups per second. Then I ran two of these tests at the same time, and the rate stayed about the same. I have 8 cores, so I made 8 processes, and it was the same. 125 requests/second * 8 * 86400 seconds/day = 86_400_000 requests per day. I added another thread writing as quickly as possibl…

I respect your research into the matter, but it frankly doesn't matter one bit. Here's why.

Your peak load/sec is never going to be close to your average load/sec. I've seen be between 2x-5x average load.

Read performance on simple selects is straightforward to scale. Most solutions to that problem just put the data in memory, via the database cache or memcache.

Part of the difficulty is in scaling writes. What if your 1000 queries/sec are all inserts on the same 500M row table? What if they are updates updates on a table that's 10M rows long? This is when you have to make hard decisions about sharding and the like.

I certainly believe that SQLite is "fast enough" for many applications. I also know that many applications are NOT doing 10s of millions of requests a day.

Re: Sysadmin mistakes start-ups make

#68

Post by gstar »

jrockway wrote: post #66
teej wrote: post #61

Earlier quoted context omitted.

> SQLite is also not as slow as people think it is; you can easily handle 10s of millions of requests per day with it Scaling relational databases to the point of 10s of millions of requests is extremely non-trivial. Unless you can show me personally or can show me evidence otherwise, don't make this claim. You're doing a disservice to the people that have worked countless hours to eke every last millisecond of perfo…

10 million per day is about 100 per second. SQLite performs about this quickly. I wrote a test script and it did 125 (unindexed) lookups per second. Then I ran two of these tests at the same time, and the rate stayed about the same. I have 8 cores, so I made 8 processes, and it was the same. 125 requests/second * 8 * 86400 seconds/day = 86_400_000 requests per day. I added another thread writing as quickly as possibl…

That -is- fast, but I still have trouble reconciling that deep down in my computer, a human readable SQL query gets built, and then another process parses that SQL. Seems so wasteful building and then parsing a human readable string for something that's happening on the same machine.

I know nothing of SQLites's internals, but wouldnt it make more sense to parse the query once and then store a compiled version of the query for subsequent lookups? Like you might do with a regexp?

Re: Sysadmin mistakes start-ups make

#69

Post by tptacek »

SwellJoe wrote: post #35

One of the most common problems we see is DNS misconfiguration. It seems most folks just haven't read the grasshopper book. If you're doing anything on the Internet, you need a basic understanding of DNS. Once you grasp the fundamentals, most DNS problems become completely transparent, but I've seen people spend weeks trying to solve DNS problems due to lack of understanding.

Or, you can skip the O'Reilly book --- and most of the mistake you could make with the simplest BIND setup --- and just use djbdns.

Re: Sysadmin mistakes start-ups make

#70

Post by bcl »

I'd say their biggest mistake is usually not hiring a sysadmin who also has development experience (or developers without sysadmin experience). I've found that my knowledge in both realms has been invaluable in determining how to design the infrastructure and how to write the code.
Post reply on HN