Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving The New York Times Games Platform to Google App Engine

open.nytimes.com

Post reply on HN
Print view View original

81–90 of 152 posts

Re: Moving The New York Times Games Platform to Google App Engine

#81

Post by vira28 »

karmakaze wrote: post #79
ciguy wrote: post #77

As a DevOps consultant I've actually worked with clients migrating stacks to and from GCE/AWS (Yeah, both ways, not the same client). What I've found in aggregate is that GCE is a bit easier to use at first as AWS has a LOT of features and terminology to learn. When it comes down to it though, many GCE services felt really immature, particularly their CloudSQL offering. One client recently moved from GCE to AWS simpl…

Why CloudSQL instead of Cloud Spanner? If for existing SQL workloads I can understand but for new services I'm admiring Spanner over DynamoDB

Looks like Spanner is a relational database. DDB is just a key-value pair. So, is it fair to compare against them or am i missing something?

Re: Moving The New York Times Games Platform to Google App Engine

#82

Post by vira28 »

obulpathi wrote: post #76
vira28 wrote: post #69

Earlier quoted context omitted.

This thread here came at the right time. Today whole day I attended the DynamoDB training. Honestly, one thing that I understood is its cost based on reads and writes per second. Irrespective of the amount of read data per operation (whether its 1 bytes or 100 bytes), its always charged for 1KB. So, as a work around what they suggested is using a Kinesis, a Lambda and an another service to make the write operation as…

Yep, same problems with many other services: * Kinesis Streams: Writes limited to 1K/sec and 1 MB / shard, reads limited to 2K/shard. Want a different read/write ratio? Nop, not possible. Proposed solution: use more shards. Does not scale automatically. There is another service called Kinesis Streams that does not offer read access to streaming data. * EFS: Cold start problems. If you have small amount of data in EFS…

Agree totally. The solution to overcome those shortcomings in AWS, is to sort of put bandaid's with more services (at least their suggestion). I do understand, its not feasible to provide service which fits for everyone, however it will be good if they solve the fundamental problem.

One more to add in the list.

In DynamoDB during peak (or rush hour) you can scale which increases the underlying replica's(or partitions) to keep the reads smooth. However, after the rush hour there is no way to drop those additional resources. May be someone can correct me, if I am wrong.

Re: Moving The New York Times Games Platform to Google App Engine

#83

Post by zjaffee »

obulpathi wrote: post #30
fha wrote: post #21

Earlier quoted context omitted.

> AWS . . . does not scale well GCP may have some advantages over AWS, but the reverse is also true and it's hard to take what you say seriously when you say something like that.

Adding more context. Sorry for missing it out in first place. I mostly work in Big Data Space. Google Clouds Big Data stuff is built for Streaming / Storing / Processing / Querying / Machine Learning at Internet Scale data (PubSub / Bigtable / Dataflow / BigQuery / Cloud ML). AWS scales to terabyte level loads. But, beyond that, its hard and super costly. Google's services autoscale to Petabyte levels / millions of u…

Hold on, please do not say google cloud scales well, yes they do have services that make a ton of claims, but unlike AWS, things don't work as promised which is magnified by the fact that their support is way worse.

Additionally, Big Query is far more expensive than Athena, where you have to pay a huge premium on storage.

The biggest difference is that what amazon provides you in infrastructure, where as google provides you a platform. While app engine is certainly easier to use than elastic bean stalk, you have very little control over what is done in the background once you let google do its thing.

Re: Moving The New York Times Games Platform to Google App Engine

#84

Post by wutwutwutwut »

ringaroundthetx wrote: post #78
ciguy wrote: post #77

As a DevOps consultant I've actually worked with clients migrating stacks to and from GCE/AWS (Yeah, both ways, not the same client). What I've found in aggregate is that GCE is a bit easier to use at first as AWS has a LOT of features and terminology to learn. When it comes down to it though, many GCE services felt really immature, particularly their CloudSQL offering. One client recently moved from GCE to AWS simpl…

Yeah Azure is like that too. Stuff is just really immature and you'll get blocked by lots of things that make no sense. If you have to get vendor locked in your quest to not have to manage servers while learning unintuitively misnamed parts of a computer, choose Amazon.

I use Azure all the time (app services, storage, cloud services,VMs, SQL, CDN etc) and almost never run into this issue. Can you share some example on what you mean?

Re: Moving The New York Times Games Platform to Google App Engine

#85

Post by neya »

Hey community, let me share my experience with AppEngine. I work in a small firm, where we've developed a massive Software Application comprising of 12 medium-sized apps. I went with Phoenix 1.3 w/ the new umbrella architecture.

With AppEngine, the beauty is that you can have many custom named microservices under one AppEngine project and each microservices can have many versions. You can even decide how much percentage of traffic should be split between each of these microservices.

What's awesome is, in addition to the standard runtimes (Ruby, Python, Go, Java, etc.) Google also provides something called custom VMs for AppEngine, meaning you can push docker based setups into your AppEngine service, with basically any stack you want. This alone is a HUGE incentive to move to AppEngine because usually custom stack will require you to maintain the server side of things, but with Docker + AppEngine, zero devops. Their network panel is also very intuitive to add/delete rules to keep your app secured.

I've been using AppEngine for over 4 years now and every time I tried a competitive offering (such as AWS Beanstalk, for example) I've only been disappointed.

AppEngine is great for startups. For example, a lesser known feature within AppEngine is their real-time image processing service API. This allows you to scale/crop/resize images in real time and the service is offered free of charge (except for storage).

Works really well for web applications with basic image manipulation requirements.

https://cloud.google.com/appengine/docs/standard/python/imag...

The best part is, you call your image with specific parameters that'll do transformations on the fly. For example, /image.jpg?s=120 will return a 120px image. Appending -c will give you a cropped version, etc.

I really hope to see AppEngine get more love from startups as it's a brilliant platform, much more performant than it's competitors' offerings. For example, I was previously a huge proponent of Heroku and upon comparing numbers, I realized AppEngine is way more performant (in my use case). I'm so glad we made the switch.

If you're looking/considering to move to AppEngine, let me know here and I'll try my best to answer your questions.

Re: Moving The New York Times Games Platform to Google App Engine

#86

Post by PaulRobinson »

g09980 wrote: post #18
tyingq wrote: post #6

Earlier quoted context omitted.

The "inelastic" might have been a shot at AWS. When pressed, the AWS people do use phrases like "pre-warming", "over provisioning" and "advance notice" around their ELB/ALB setup and ECS. Google's cloud salespeople pitch that they don't require any of that.

Curious if the need for pre-warming ELB/ALB still applies. Last time this came up, an AWS employee mentioned it is no longer necessary ( https://news.ycombinator.com/item?id=14052079 ), but would be nice if this was documented.

The poster was talking about ALBs. I'm not experienced enough with them to know whether the claim is true.

I do know on ELBs though, pre-warming is essential for high throughput.

Re: Moving The New York Times Games Platform to Google App Engine

#87

Post by themacguffinman »

karmakaze wrote: post #79
ciguy wrote: post #77

As a DevOps consultant I've actually worked with clients migrating stacks to and from GCE/AWS (Yeah, both ways, not the same client). What I've found in aggregate is that GCE is a bit easier to use at first as AWS has a LOT of features and terminology to learn. When it comes down to it though, many GCE services felt really immature, particularly their CloudSQL offering. One client recently moved from GCE to AWS simpl…

Why CloudSQL instead of Cloud Spanner? If for existing SQL workloads I can understand but for new services I'm admiring Spanner over DynamoDB

Cloud Spanner is unnecessarily expensive if you don't have the kind of performance requirements it was designed for.

Re: Moving The New York Times Games Platform to Google App Engine

#88

Post by sidlls »

Jedi72 wrote: post #25

Getting pro GCP articles to the top of HN must no-doubt be a high priority for the Google marketing team. This is the nature of modern advertising, sneakily trying to subvert your thinking by masquerading as something else.

Really why would they care? HN is a tiny, tiny slice of the programming world.

Re: Moving The New York Times Games Platform to Google App Engine

#89

Post by remus »

tyingq wrote: post #6
vs2 wrote: post #2

"Due to the inelastic architecture of our AWS system, we needed to have the systems scaled up to handle our peak traffic at 10PM when the daily puzzle is published." WT... I had to reread this to make sure I didnt misunderstand... why not work on making the current arhictecture elastic?! #cloudPorn

The "inelastic" might have been a shot at AWS. When pressed, the AWS people do use phrases like "pre-warming", "over provisioning" and "advance notice" around their ELB/ALB setup and ECS. Google's cloud salespeople pitch that they don't require any of that.

It's funny they'd mention that, and then later in the same article say they had to use a Cron job to scale up their gcp solution for a daily spike.
Post reply on HN