Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google App Engine turns 2

googleappengine.blogspot.com

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Google App Engine turns 2

#11

Post by jeff18 »

axiom wrote: post #7
jeff18 wrote: post #4

Earlier quoted context omitted.

How do you reconcile that statement with the fact that tons of developers do actually use it? Myself included. ;)

We use it too. That's why I can make that statement. A DB request takes over 100ms on the datastore. That's insane and unusable for about 90% of applications. Even their implementation of memcached is several orders of magnitude slower than actual memcached.

You claim that about 90% of applications require sub-100ms latencies on datastore writes. Can you back this up? On the contrary, it seems to me that very few applications need to be blocked by writing to the datastore, let alone cases where 100 milliseconds make them unusable.

Secondly, 3 millisecond memcached access is several orders of magnitude too slow? 3 nanosecond memcache access would be quite a physical achievement. ;)

Re: Google App Engine turns 2

#12

Post by alecthomas »

axiom wrote: post #2

Until they speed up the datastore by a factor of 10 the app engine just isn't a viable platform. They don't even use it internally for that very reason (this is coming from colleagues who work there.)

AppEngine is used quite extensively internally.

Re: Google App Engine turns 2

#13

Post by axiom »

jeff18 wrote: post #11
axiom wrote: post #7

Earlier quoted context omitted.

We use it too. That's why I can make that statement. A DB request takes over 100ms on the datastore. That's insane and unusable for about 90% of applications. Even their implementation of memcached is several orders of magnitude slower than actual memcached.

You claim that about 90% of applications require sub-100ms latencies on datastore writes. Can you back this up? On the contrary, it seems to me that very few applications need to be blocked by writing to the datastore, let alone cases where 100 milliseconds make them unusable. Secondly, 3 millisecond memcached access is several orders of magnitude too slow? 3 nanosecond memcache access would be quite a physical achie…

If you're getting 3ms memcached access then you must be using a different app engine than I am.

And it's 100ms reads not writes. Writes are slower. Much slower.

Re: Google App Engine turns 2

#14

Post by markkoberlein »

khelloworld wrote: post #5

As you begin your 3rd year, here are a few things I wish for you: 1. May you be able to work with your brother, Google Apps, to allow SSL support for custom domains. 2. May you be able to lift your 1 MB cap per urlfetch request to at least 10 MB so that your users can write some cool apps using the YouTube API. 3. May you inherit some snazzy search skills this year from Google Search, your grand daddy, and allow your…

I would agree that the lack of support of SSL on custom domains and the size limitations (my main issue is with the upload limits) have kept GAE from really taking off. I have used it for a small non-profit app and I host my personal consulting site on it. However, because of the limitations I'm building my startup on rackspace instead of GAE.

I really love GAE as a development platform, especially as a single developer, but I hope they will work through their roadmap faster.

Re: Google App Engine turns 2

#15

Post by jeff18 »

axiom wrote: post #13
jeff18 wrote: post #11

Earlier quoted context omitted.

You claim that about 90% of applications require sub-100ms latencies on datastore writes. Can you back this up? On the contrary, it seems to me that very few applications need to be blocked by writing to the datastore, let alone cases where 100 milliseconds make them unusable. Secondly, 3 millisecond memcached access is several orders of magnitude too slow? 3 nanosecond memcache access would be quite a physical achie…

If you're getting 3ms memcached access then you must be using a different app engine than I am. And it's 100ms reads not writes. Writes are slower. Much slower.

According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3].

Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;)

[1] http://code.google.com/status/appengine/detail/memcache/2010...

[2] http://code.google.com/status/appengine/detail/datastore/201...

[3] http://code.google.com/status/appengine/detail/datastore/201...

Re: Google App Engine turns 2

#16

Post by kalid »

jeff18 wrote: post #15
axiom wrote: post #13

Earlier quoted context omitted.

If you're getting 3ms memcached access then you must be using a different app engine than I am. And it's 100ms reads not writes. Writes are slower. Much slower.

According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3]. Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;) [1] http://code.google.com/status/appengine/detail/memcache/2…

I don't think anyone would accuse google of falsifying data, but stats can be misleading. I might have missed it, but I didn't see a standard deviation for those numbers -- if you're doing several reads and they happen to be "above average" then you could see the random appearance of sluggishness.

Just checking now (http://imgur.com/8Jb82) there are some really long requests (800ms!) which might get smoothed out in an average.

Re: Google App Engine turns 2

#17

Post by axiom »

jeff18 wrote: post #15
axiom wrote: post #13

Earlier quoted context omitted.

If you're getting 3ms memcached access then you must be using a different app engine than I am. And it's 100ms reads not writes. Writes are slower. Much slower.

According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3]. Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;) [1] http://code.google.com/status/appengine/detail/memcache/2…

Look, if you're running a basically static site that serves a few 100k page views per day, then the app engine is fine and a great choice. Hell, a $20/month linode instance would do just as well. Get two $20 instances and you can handle all the burst traffic you'll ever need.

If on the other hand you're running a web app where you can't cache a decent chunk of your data, and you need to do 4-5 db reads per request, then you have a painfully slow system that your users will bitch about.

To take just one example, we've had to push all our db writes into task queues because we'd consistently hit the 30 simultaneous dynamic requests limit when we had more than 100 users on the site at once. To take another, we couldn't grab more than 2000 db entries in a request without hitting the 30 second http request time limit (which we were using to generate a csv dump of a chunk of a user's data.)

Over the last ~8 months of using the app engine we've found that we're just spending way too much time dealing with it's various limitations (no SSL for third party domains, 3000 file limit, slow ass datastore, no threads, no comet push...) that the benefits of easy deployment + sort-of-automatic-scaling just aren't worth it. So now we're moving over to linode + EC2 over the next few months.

p.s. we've found memcached to be around 10ms latency on average... who knows maybe our app is on a special slow instance? and yes, that's 10x slower than normal memcached.

Re: Google App Engine turns 2

#18

Post by e98cuenc »

axiom wrote: post #2

Until they speed up the datastore by a factor of 10 the app engine just isn't a viable platform. They don't even use it internally for that very reason (this is coming from colleagues who work there.)

It depends a lot on each particular project. Panoramio and pubhubsubbub are two of the most well known Google projects built in AppEngine.

Re: Google App Engine turns 2

#19

Post by e98cuenc »

axiom wrote: post #7
jeff18 wrote: post #4

Earlier quoted context omitted.

How do you reconcile that statement with the fact that tons of developers do actually use it? Myself included. ;)

We use it too. That's why I can make that statement. A DB request takes over 100ms on the datastore. That's insane and unusable for about 90% of applications. Even their implementation of memcached is several orders of magnitude slower than actual memcached.

The time for a DB request depends a lot on the particular request. If you're querying by key then it's usually ~30 ms.

Their memcached is only "much" slower than actual memcached if you run your "actual" memcached in the same computer or some rack as your client servers. Google memcached vs external memcached have almost identical performance, it all depends on where are your servers.

If you were running your memcached servers in the same datacenter, but different rack, you will find out that you're only slightly faster or same speed than Google's memcached service.

I want AppEngine to improve its speed as much as you want, trust me, but with current limits is already possible to build a web application. We did it in Panoramio, and others did it too.

Re: Google App Engine turns 2

#20

Post by jeff18 »

axiom wrote: post #17
jeff18 wrote: post #15

Earlier quoted context omitted.

According to the independently verifiable health dashboard, the current memcache latency average for today is about 7ms[1]. The average datastore get() time is around 50ms[2]. The average datastore put() time is around 100ms[3]. Are you accusing Google of falsifying this trivially reproducible and abundant data? That would be quite an interesting story. ;) [1] http://code.google.com/status/appengine/detail/memcache/2…

Look, if you're running a basically static site that serves a few 100k page views per day, then the app engine is fine and a great choice. Hell, a $20/month linode instance would do just as well. Get two $20 instances and you can handle all the burst traffic you'll ever need. If on the other hand you're running a web app where you can't cache a decent chunk of your data, and you need to do 4-5 db reads per request, t…

I think you have many valid concerns about App Engine which are definitely worth discussing, but your over-the-top rhetoric and exaggerated numbers make it hard to do that!

P.S. have you considered AppScale to migrate off of Google's infrastructure? Google has been funding a project that is a complete clone of the App Engine API, but can run on arbitrary hardware using MongoDB or a plethora of other middleware.

I think we've hit the max comment depth, and I think it's time to turn on the noprocast settings again. ;)

Post reply on HN