The Unofficial Guide to Migrating Off of Google App Engine

Post by mhusby » Sat, Feb 26, 2011, 4:32 PM UTC

jasonkester wrote: post #31

mhusby wrote: post #10

Earlier quoted context omitted.

I think in this reference they are not talking about storing live data in memcache, just copies of objects for faster access. So when you are getting the information about a user you check memcache first, if its not there look it up in datastore and add it to memcache. Then when the user loads a second page you can get the information directly from memcache instead of having to look it up again.

I think that's what he's talking about. The point in question is why are you doing this on a new application? The sort of caching you describe is something you put in place after your site starts seeing a lot of traffic, and running queries for every request start to slow things down to the point where optimizing them doesn't help. If you're designing your app well and running it on a solid stack, that should be some…

  from google.appengine.api import memcache
  from google.appengine.ext import db

  class User(db.Model):
    email = db.StringProperty()
    password = db.StringProperty()
    sessionKey = db.StringProperty()

    def userKey(self):
      return "User.session="+self.sessionKey

    def put(self):
      super(User, self).put()
      memcache.delete(self.userKey())
      memcache.add(self.userKey(), self, 600)

    def getCurrent(self, request):
      user = memcache.get(self.userKey())
      if user is None:
        user = User.all().filter('sessionKey = ', request.header.cookie.get('user'))
        memcache.add(self.userKey(), user, 600)
      return user

Post by jasonkester » Sat, Feb 26, 2011, 6:16 PM UTC

mhusby wrote: post #42

jasonkester wrote: post #31

Earlier quoted context omitted.

I think that's what he's talking about. The point in question is why are you doing this on a new application? The sort of caching you describe is something you put in place after your site starts seeing a lot of traffic, and running queries for every request start to slow things down to the point where optimizing them doesn't help. If you're designing your app well and running it on a solid stack, that should be some…

I guess I can see that argument, but at the same time its so easy to add memcache that it really seems silly not to add memcache for you models at are accessed a lot. Here is a simple user model, adding memcahce is a total of 7 lines. I just cant see the case where you wouldn't add this to start with: from google.appengine.api import memcache from google.appengine.ext import db class User(db.Model): email = db.String…

Post by powera » Sat, Feb 26, 2011, 6:51 PM UTC

kamens wrote: post #38

powera wrote: post #5

A few points (disclaimer: I work at Google on projects including App Engine) 1) It's not fair at all to say that Google "continue to ignore the two critical issues of uptime and data store latency". The HR datastore is specifically designed to address the concerns about variable latency of datastore operations, and to prevent both planned and unplanned downtime. 2) In my experience, high CPU cost for datastore operat…

Can you elaborate on queries not having "the right indexes"? Since the development server automatically adds indexes for all queries run on local dev machines and the production machines cannot run a query for which the necessary indexes don't exist, it's easy to assume that your indexes must be right. I'd love to hear more about tweaks to existing indexes that can alleviate these sorts of problems.

Post by itsnotvalid » Sat, Feb 26, 2011, 7:50 PM UTC

bane wrote: post #41

mike_esspe wrote: post #30

Earlier quoted context omitted.

For my app the real problem with GAE is high cost.

Can you illuminate that a little? Among the competitors, GAE appears to be among the lowest cost platforms. I'm sure there's certain types of applications that can cause it to become expensive, but I'm not sure I can predict well enough what those are.

Post by fletchowns » Sat, Feb 26, 2011, 8:09 PM UTC

mhusby wrote: post #42

jasonkester wrote: post #31

Earlier quoted context omitted.

I think that's what he's talking about. The point in question is why are you doing this on a new application? The sort of caching you describe is something you put in place after your site starts seeing a lot of traffic, and running queries for every request start to slow things down to the point where optimizing them doesn't help. If you're designing your app well and running it on a solid stack, that should be some…

I guess I can see that argument, but at the same time its so easy to add memcache that it really seems silly not to add memcache for you models at are accessed a lot. Here is a simple user model, adding memcahce is a total of 7 lines. I just cant see the case where you wouldn't add this to start with: from google.appengine.api import memcache from google.appengine.ext import db class User(db.Model): email = db.String…

  @Cached
  public class MyEntity {
      @Id Long id;
      ...
  }