Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Google App Engine Ready for Prime Time?

sachinrekhi.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Is Google App Engine Ready for Prime Time?

#11

Post by mdasen »

It isn't really ready for prime-time. AppEngine still has some limits that make it hard to develop.

Examples:

The 1MB file upload limit. It's not that I want to have users uploading 500MB tars or something, but I don't want to put a message in my app saying, "please resize you photo to be less than 1MB before uploading where we'll probably alter the photo again."

They've indicated that you'll be able to purchase more storage and bandwidth which is great, but noticeably absent was the ability to send more than the 2,000 email limit (http://googleappengine.blogspot.com/2008/12/system-status-da...). The problem might not be that I want to send more than 2,000 emails in a single day. The problem is that these quotas are rolling windows. Plus, the whole point is that AppEngine scales without me having to do anything. If it doesn't accommodate a decent sized social app, I might as well just put something on Slicehost or EC2 since at that low a usage rate, scaling isn't that hard.

There's no map/reduce or cron-like jobs. The datastore in AppEngine acts as a bit of a straight-jacket to ensure you don't do operations that don't scale. However, without map/reduce and cron jobs, it can mean that you just can't do certain things. And yes, I know that you could fake it by running another server, making an HTTP request to AppEngine and cutting up a process that might take a minute into chunks that AppEngine would execute, but that's just silly. The whole point of AppEngine is not having to deal with your own server and having an integrated environment that just works.

If the purpose of AppEngine is for developers not to worry about deployment and scaling, then it just fails because of some of the limitations. AppEngine doesn't scale. Maybe that's just a matter of the quotas being removed/being able to pay for more, but as of right now it doesn't scale at all. And it doesn't seem to be an amazingly high priority at Google. Software development takes time, but it's been a while since they said developers would be able to buy more and it hasn't come - beyond the recent blog posting I linked to showing some screen shots.

AppEngine could be a wonderful platform to develop on, but not until some of these problems get fixed.

Re: Is Google App Engine Ready for Prime Time?

#12

Post by davepeck »

litewulf wrote: post #9
davepeck wrote: post #3

Two weeks ago, my client’s datastore indexes became corrupted. The failure was demonstrably AppEngine's. Attempts to manually rebuild indexes failed. The AppEngine team worked for several days to resolve the problem, which we appreciated. But such failures are disheartening when you’re busy signing up paid customers. I suppose the term “beta” is so overused that it has nearly lost its meaning. Sometimes, however, it…

Do you have any pointers to what was wrong and how it was resolved?

Unfortunately, the issue still lurks.

Basically, some combination of puts to one of our models would cause queries to return inconsistent results. For example, the query:

SELECT * FROM MyModel WHERE property=True

would return N results, but

SELECT * FROM MyModel WHERE property=True ORDER BY foo

would return far fewer than N results.

It appears that _something_ we were doing exercised an AppEngine bug that caused the indexes to get corrupted. (Note that this would happen even from AppEngine's data viewer -- which is to say, we got N by paging through ALL results, not just one fetched batch.)

We worked with Google but designing a simple repro case proved difficult -- the corruption happened only under high load, and only on one of our models.

Trial and error showed us that if we manually fetched and re-put() the "broken" rows in our table (aka the ones that didn't show up when we used ORDER BY,) the indexes would right themselves. So I wrote a bit of code to do this, and started running a cron job on a local box to invoke that code regularly. Lame, but only temporary.

Last time we exchanged mail with the AppEngine team, they told us they were aware that this was a bug on their end and were working on a fix. However, to my knowledge, they're still working on it...

Re: Is Google App Engine Ready for Prime Time?

#13

Post by vsiva68 »

davepeck wrote: post #12
litewulf wrote: post #9

Earlier quoted context omitted.

Do you have any pointers to what was wrong and how it was resolved?

Unfortunately, the issue still lurks. Basically, some combination of puts to one of our models would cause queries to return inconsistent results. For example, the query: SELECT * FROM MyModel WHERE property=True would return N results, but SELECT * FROM MyModel WHERE property=True ORDER BY foo would return far fewer than N results. It appears that _something_ we were doing exercised an AppEngine bug that caused the…

Yup, I filed issue #901:

http://code.google.com/p/googleappengine/issues/detail?id=90...

Seems like a serious bug, but I haven't seen any updates as yet.

Re: Is Google App Engine Ready for Prime Time?

#14

Post by davepeck »

vsiva68 wrote: post #13
davepeck wrote: post #12

Earlier quoted context omitted.

Unfortunately, the issue still lurks. Basically, some combination of puts to one of our models would cause queries to return inconsistent results. For example, the query: SELECT * FROM MyModel WHERE property=True would return N results, but SELECT * FROM MyModel WHERE property=True ORDER BY foo would return far fewer than N results. It appears that _something_ we were doing exercised an AppEngine bug that caused the…

Yup, I filed issue #901: http://code.google.com/p/googleappengine/issues/detail?id=90... Seems like a serious bug, but I haven't seen any updates as yet.

I got the impression we weren't the only ones. Good to see a bug filed, will star it!

Were you ever able to find an isolated repro case?

Re: Is Google App Engine Ready for Prime Time?

#15

Post by ivankirigin »

Anyone found a good tutorial for pointing mydomain.com to myapp.appspot.com ?

The process is ridiculous and horribly undocumented.

Also, custom domains can't have SSL. That's a deal killer right there for most startup projects.

I am using GAE for some basic pages. Where I would make a static page, I'm running Django. The deployment process is a breeze, and any of the pages can scale to be a proper app. I'll probably move http://kirigin.com there soon.

Re: Is Google App Engine Ready for Prime Time?

#16

Post by sachin »

ivankirigin wrote: post #15

Anyone found a good tutorial for pointing mydomain.com to myapp.appspot.com ? The process is ridiculous and horribly undocumented. Also, custom domains can't have SSL. That's a deal killer right there for most startup projects. I am using GAE for some basic pages. Where I would make a static page, I'm running Django. The deployment process is a breeze, and any of the pages can scale to be a proper app. I'll probably…

You are right, it is a pretty confusing process. Here is the best docs I could find on the topic. You basically setup your domain first with Google Apps for your domain. Then you link the Google Apps for your domain account with your Google App Engine project. The additional issue is that you can't map naked domains (http://domain.com) to Google App Engine directly. Instead you have to do that within your DNS provider (I set it up in GoDaddy).

http://knol.google.com/k/larry-h/google-apps-discussion-grou...

Re: Is Google App Engine Ready for Prime Time?

#17

Post by litewulf »

davepeck wrote: post #12
litewulf wrote: post #9

Earlier quoted context omitted.

Do you have any pointers to what was wrong and how it was resolved?

Unfortunately, the issue still lurks. Basically, some combination of puts to one of our models would cause queries to return inconsistent results. For example, the query: SELECT * FROM MyModel WHERE property=True would return N results, but SELECT * FROM MyModel WHERE property=True ORDER BY foo would return far fewer than N results. It appears that _something_ we were doing exercised an AppEngine bug that caused the…

Good Lord thats scary sounding.

I believe the app engine datastore has been stated as being built upon bigtable. I suppose the index creation is "newer" code than the core of bigtable, but I'd like to hope that the Google bigtable code is bugfree at this point...

Re: Is Google App Engine Ready for Prime Time?

#18

Post by snprbob86 »

litewulf wrote: post #17
davepeck wrote: post #12

Earlier quoted context omitted.

Unfortunately, the issue still lurks. Basically, some combination of puts to one of our models would cause queries to return inconsistent results. For example, the query: SELECT * FROM MyModel WHERE property=True would return N results, but SELECT * FROM MyModel WHERE property=True ORDER BY foo would return far fewer than N results. It appears that _something_ we were doing exercised an AppEngine bug that caused the…

Good Lord thats scary sounding. I believe the app engine datastore has been stated as being built upon bigtable. I suppose the index creation is "newer" code than the core of bigtable, but I'd like to hope that the Google bigtable code is bugfree at this point...

Nothing is ever bug free. The only thing the Big Table team can hope for is "statistically unlikely to contain a serious bug in the code paths commonly executed by our high traffic products".

Re: Is Google App Engine Ready for Prime Time?

#19

Post by davepeck »

snprbob86 wrote: post #18
litewulf wrote: post #17

Earlier quoted context omitted.

Good Lord thats scary sounding. I believe the app engine datastore has been stated as being built upon bigtable. I suppose the index creation is "newer" code than the core of bigtable, but I'd like to hope that the Google bigtable code is bugfree at this point...

Nothing is ever bug free. The only thing the Big Table team can hope for is "statistically unlikely to contain a serious bug in the code paths commonly executed by our high traffic products".

True; on the other hand, you don't exit beta if you have a known data corruption bug.
Post reply on HN