Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving The New York Times Games Platform to Google App Engine

open.nytimes.com

Post reply on HN
Print view View original

121–130 of 152 posts

Re: Moving The New York Times Games Platform to Google App Engine

#121

Post by neya »

neya wrote: post #85

Hey community, let me share my experience with AppEngine. I work in a small firm, where we've developed a massive Software Application comprising of 12 medium-sized apps. I went with Phoenix 1.3 w/ the new umbrella architecture. With AppEngine, the beauty is that you can have many custom named microservices under one AppEngine project and each microservices can have many versions. You can even decide how much percent…

Full disclosure: I DON'T work with Google, I DON'T sell their services/products. I run a startup myself, with an Saas on top of AppEngine. This is just my documented (positive) experience with the stack above. I get paid nothing by Google, nothing in $ nor credits this is just my own personal documented experience.

Being a long time HN member, I would responsibly disclose if I were somehow affiliated with Google (trust me, I wish I was).

Re: Moving The New York Times Games Platform to Google App Engine

#122

Post by neya »

smn1234 wrote: post #107
neya wrote: post #85

Hey community, let me share my experience with AppEngine. I work in a small firm, where we've developed a massive Software Application comprising of 12 medium-sized apps. I went with Phoenix 1.3 w/ the new umbrella architecture. With AppEngine, the beauty is that you can have many custom named microservices under one AppEngine project and each microservices can have many versions. You can even decide how much percent…

The way you've delivered this "experience" makes you sound like you either work for Google or were asked to make a sponsored statement - for credits or $.

My bad, I've added the disclosure.

Re: Moving The New York Times Games Platform to Google App Engine

#123

Post by kuschku »

eicnix wrote: post #105
oblio wrote: post #95

Earlier quoted context omitted.

> * ECS: Two containers can not use same port on same node. Anti pattern to containers. Could you elaborate for this? I'm not sure I understand, are you saying that 2 containers cannot be mapped to the same host port? Because that would seem normal, you can't bind to a port where there's already something listening. But I guess I must be missing something.

Nowaday you would often run containers with a container network(flannel, calico, etc.) that assigns an unique IP per container thus avoids conflicting port mappings regardless how many containers with the same port run on a single host.

Or you have them on a physical private network, each bound to a separate IP, but yes.

Re: Moving The New York Times Games Platform to Google App Engine

#124

Post by bpicolo »

funkjunky wrote: post #94
bpicolo wrote: post #32

Earlier quoted context omitted.

Yeah. In particular, I was really looking for something for "here's how to do most of the basics for Cloud Storage". If you look at https://googlecloudplatform.github.io/google-cloud-python/la... , for example, there's not even a top level navigation index that I can read through to guess what function I might need by name.

(Google cloud support here) The pages you linked to are supposed to serve as client libraries reference only. If you want higher level instructions and examples, always start with the main Google Cloud docs first. On any page that offers instructions, the top of the code windows offers a selection of client library languages/CLI tools/REST API available to do whatever that task is. For cloud storage, start here: http…

Heya,

Thanks for the link! You're right that this is what I was looking for. Unfortunately, that hadn't shown up in a convenient place while I was googling around. Would be good to add direct links to those from the client lib references, because those pop up for e.g. "google storage python" first. (Unless they're already there and I didn't see them).

Links from the READMEs of the repo might be useful too to help people get there? https://github.com/GoogleCloudPlatform/google-cloud-python

Re: Moving The New York Times Games Platform to Google App Engine

#125

Post by tim333 »

vira28 wrote: post #69
obulpathi wrote: post #13

Two things keep coming up while comparing GCP and AWS: * This accomplishment would not have been possible for our three-person team of engineers with out Google Cloud (AWS is too low level, hard to work with and does not scale well). * We’ve also managed to cut our infrastructure costs in half during this time period (Per minute billing, seamless autoscaling, performance, sustained usage discounts, ... )

This thread here came at the right time. Today whole day I attended the DynamoDB training. Honestly, one thing that I understood is its cost based on reads and writes per second. Irrespective of the amount of read data per operation (whether its 1 bytes or 100 bytes), its always charged for 1KB. So, as a work around what they suggested is using a Kinesis, a Lambda and an another service to make the write operation as…

>too many moving pieces for a simple thing to achieve

Maybe put all the users data in a blob with a JSON.stringify type thing and then JSON.parse it when you get it back?

Re: Moving The New York Times Games Platform to Google App Engine

#126

Post by kuschku »

londons_explore wrote: post #118
smn1234 wrote: post #107

Earlier quoted context omitted.

The way you've delivered this "experience" makes you sound like you either work for Google or were asked to make a sponsored statement - for credits or $.

I would hope someone on HN would disclose a conflict of interest when presenting their opinion. Disclosure: Work for one of the cloud providers, but not on cloud itself.

Sadly, this is rarely true. Many of the people defending AMP for example turn out to be Google employees that didn't disclose it (found via their comment history), and people that complain about AMP are not just privacy activists, but often also involved in publishers or advertisers that lose money from it.

I doubt it would be much different in this thread.

Re: Moving The New York Times Games Platform to Google App Engine

#127

Post by steven_h »

oblio wrote: post #95
obulpathi wrote: post #76

Earlier quoted context omitted.

Yep, same problems with many other services: * Kinesis Streams: Writes limited to 1K/sec and 1 MB / shard, reads limited to 2K/shard. Want a different read/write ratio? Nop, not possible. Proposed solution: use more shards. Does not scale automatically. There is another service called Kinesis Streams that does not offer read access to streaming data. * EFS: Cold start problems. If you have small amount of data in EFS…

> * ECS: Two containers can not use same port on same node. Anti pattern to containers. Could you elaborate for this? I'm not sure I understand, are you saying that 2 containers cannot be mapped to the same host port? Because that would seem normal, you can't bind to a port where there's already something listening. But I guess I must be missing something.

The OP is talking about how when using a classic load balancer in AWS, your containers will be deployed all exposing the same port, kind of like running "docker run -p 5000:5000" on each ec2 in your cluster. Once the port is in use, you can't deploy another of that container on the same ec2 node.

The solution is to use AWS's Application Load Balancers instead, which will allow you dynamically allocate ports for your containers and route traffic into them as ECS Services.

Re: Moving The New York Times Games Platform to Google App Engine

#128

Post by mattmanser »

wutwutwutwut wrote: post #84
ringaroundthetx wrote: post #78

Earlier quoted context omitted.

Yeah Azure is like that too. Stuff is just really immature and you'll get blocked by lots of things that make no sense. If you have to get vendor locked in your quest to not have to manage servers while learning unintuitively misnamed parts of a computer, choose Amazon.

I use Azure all the time (app services, storage, cloud services,VMs, SQL, CDN etc) and almost never run into this issue. Can you share some example on what you mean?

I use all that stuff and constantly run into it, are you using it in anger?

Here's a few I can remember off the top of my head:

- A (relatively) huge 6ms lag between the website and the DB

- One (random) site will mysteriously max out on memory on app-pool startup and take all the others down

- Their scheduler has no concept of timezones

- Their scheduler uses your local time when setting up the job, but UTC for other parts (this has been an open issue for over a year I think)

- Files will get mysteriously locked in deployments and the deployment process will silently fail

- Deployments will suddenly take an absolute age for no reason

- The entire admin UI will slow to an absolute crawl for hours on end

- Some admin tasks always claim they've failed, even though they've succeeded

- Their API wrapper is just wrong on almost every level

Add on top of that the worse management UI I've ever had to deal with and it makes Azure very painful to use at times. Some-one thought nesting menus in a standardised format was a good idea. It wasn't. Everything is fairly terribly named too. Want to see how your deployments doing? That's under "Deployment Options".

Performance is also dogshit compared to the cost, my 4 year old laptop is faster than their "premium" offerings.

Re: Moving The New York Times Games Platform to Google App Engine

#130

Post by brianwawok »

lftl wrote: post #73
brianwawok wrote: post #68

Earlier quoted context omitted.

Deploy is 15 seconds on GKE. Wonder why app engine is so bad.

With GKE you don't necessarily update your load balancer rules with a deploy correct? The linked thread points the blame for app engine deploys on waiting for GCLB to update.

Correct, you have software based load balancing under the covers.

My Google Load Balancers never move.. It is a single thing that points each node (physical machine) in the cluster, and distributes traffic between then.

Each node knows how to route traffic to each app. So when I deploy that app, the software load balancer at the node level will slowly move traffic over from old app to new app. Entire thing is MAGICAL. And 0 downtime, very very fast deploys.

Edit - But yes this explains iy. Changing the google load balancers is like a 5 minute ordeal. Total pain. Nice that with GKE you only need to touch them when your node count changes, which can be very rare (~monthly for me)

Post reply on HN