Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving The New York Times Games Platform to Google App Engine

open.nytimes.com

Post reply on HN
Print view View original

41–50 of 152 posts

Re: Moving The New York Times Games Platform to Google App Engine

#41

Post by obulpathi »

look_lookatme wrote: post #35
obulpathi wrote: post #30

Earlier quoted context omitted.

Adding more context. Sorry for missing it out in first place. I mostly work in Big Data Space. Google Clouds Big Data stuff is built for Streaming / Storing / Processing / Querying / Machine Learning at Internet Scale data (PubSub / Bigtable / Dataflow / BigQuery / Cloud ML). AWS scales to terabyte level loads. But, beyond that, its hard and super costly. Google's services autoscale to Petabyte levels / millions of u…

You need petabyte scale for the NYT Crossword?

Quoting from the article: "This accomplishment would not have been possible for our three-person team of engineers to achieve without the tools and abstractions provided by Google and App Engine."

Talking about the use case from the article, they release the puzzle at 10 and need to have infra ready to serve up all the requests. On AWS, you need to pre warm load balancers, increase the quota of your Dynamo DB, scale up instances so that they can withstand the wall of traffic, ... and then scale down after the traffic. All this takes time, people and money. Adding few other things author mentioned: Monitoring/Alerting, Local Development, Combined Access and App Logging ... will take focus from developing great apps to building out the infrastructure for apps.

Re: Moving The New York Times Games Platform to Google App Engine

#42

Post by jonknee »

revelation wrote: post #11

I was sure this was about some multiplayer game thing, but no, it's a crossword. Not entirely sure what they are even scaling here, I was expecting an article about a CDN..

It’s a paid service that hundreds of thousands of people use. Not world changing stuff, but considering competitors are still using Java applets and Flash, I for one applaud their efforts.

Re: Moving The New York Times Games Platform to Google App Engine

#43

Post by ledgerdev »

bsaul wrote: post #12

Has anybody had successful experience deploying docker containers on appengine ? Last time i tried, i had such a bad experience in terms of deployment speed ( time to build the image, then upload it, then waiting for the stuff to deploy) that i reverted to managing my own gce instance. But maybe i had bad luck..

Funny you mention that, I was just experiencing this yesterday in the flex environment. Re-deploy time for a 3 character text change on one page is 8+ minutes for each re-deploy. This is insane.

Does anyone know how to speed that up?

Re: Moving The New York Times Games Platform to Google App Engine

#44

Post by ledgerdev »

tlarkworthy wrote: post #16
bsaul wrote: post #12

Has anybody had successful experience deploying docker containers on appengine ? Last time i tried, i had such a bad experience in terms of deployment speed ( time to build the image, then upload it, then waiting for the stuff to deploy) that i reverted to managing my own gce instance. But maybe i had bad luck..

Yes uploads are fast and you can spin up a VM directly from an image now

How many minutes/seconds do you mean by fast? Can you point to guidance on how to make that happen because I am seeing 8+ minute re-deploy times.

Re: Moving The New York Times Games Platform to Google App Engine

#45

Post by ben_jones »

theDoug wrote: post #34
Jedi72 wrote: post #25

Getting pro GCP articles to the top of HN must no-doubt be a high priority for the Google marketing team. This is the nature of modern advertising, sneakily trying to subvert your thinking by masquerading as something else.

We actually spent the entire day at the Giants game today. ¯\(ツ)/¯ There's no incentive for high ranking HN posts, or any HN posts, actually. If there were, you wouldn't see others continually submit our news here before we do. This was a nice and unprompted post for everyone in GCP to read, as well. (Disclosure: I work on GCP as a product marketer.)

A GCP product marketer responding to negative commentary on HN within ~20 minutes of it being posted strikes me as automated.

EDIT: It is a double standard though, HN readers want access and responses from people on the GCP team but at the same time tinfoil hat subliminal marketing etc.

Re: Moving The New York Times Games Platform to Google App Engine

#46

Post by mattbillenstein »

vs2 wrote: post #2

"Due to the inelastic architecture of our AWS system, we needed to have the systems scaled up to handle our peak traffic at 10PM when the daily puzzle is published." WT... I had to reread this to make sure I didnt misunderstand... why not work on making the current arhictecture elastic?! #cloudPorn

I'm not sure why people are so enamored with ELB -- just terminate SSL at your web boxes using nginx and publish the public ips of all of these machines in your DNS records.

You remove a bunch of ELB per-request costs doing it this way and you can scale it however you see fit.

Re: Moving The New York Times Games Platform to Google App Engine

#47

Post by jacquesm »

obulpathi wrote: post #41
look_lookatme wrote: post #35

Earlier quoted context omitted.

You need petabyte scale for the NYT Crossword?

Quoting from the article: "This accomplishment would not have been possible for our three-person team of engineers to achieve without the tools and abstractions provided by Google and App Engine." Talking about the use case from the article, they release the puzzle at 10 and need to have infra ready to serve up all the requests. On AWS, you need to pre warm load balancers, increase the quota of your Dynamo DB, scale…

Could you please post the URL for the resource and the number of hits it receives? I'm interested in high load websites and I have a hard time picturing how this could lead to petabytes.

Re: Moving The New York Times Games Platform to Google App Engine

#48

Post by Osmose »

Jedi72 wrote: post #25

Getting pro GCP articles to the top of HN must no-doubt be a high priority for the Google marketing team. This is the nature of modern advertising, sneakily trying to subvert your thinking by masquerading as something else.

Heaven forbid a software engineer post about a major technical change their team just accomplished. It couldn't be because they want to raise their team's profile for hiring, or discuss the pros and cons of their experience, or raise their own profile as an engineer.

Re: Moving The New York Times Games Platform to Google App Engine

#49

Post by jacquesm »

ben_jones wrote: post #45
theDoug wrote: post #34

Earlier quoted context omitted.

We actually spent the entire day at the Giants game today. ¯\(ツ)/¯ There's no incentive for high ranking HN posts, or any HN posts, actually. If there were, you wouldn't see others continually submit our news here before we do. This was a nice and unprompted post for everyone in GCP to read, as well. (Disclosure: I work on GCP as a product marketer.)

A GCP product marketer responding to negative commentary on HN within ~20 minutes of it being posted strikes me as automated. EDIT: It is a double standard though, HN readers want access and responses from people on the GCP team but at the same time tinfoil hat subliminal marketing etc.

> A GCP product marketer responding to negative commentary on HN within ~20 minutes of it being posted strikes me as automated.

Nonsense, there are many Googlers on HN and all it would take is a colleague poking another with a relevant thread.

Re: Moving The New York Times Games Platform to Google App Engine

#50

Post by sb8244 »

ben_jones wrote: post #45
theDoug wrote: post #34

Earlier quoted context omitted.

We actually spent the entire day at the Giants game today. ¯\(ツ)/¯ There's no incentive for high ranking HN posts, or any HN posts, actually. If there were, you wouldn't see others continually submit our news here before we do. This was a nice and unprompted post for everyone in GCP to read, as well. (Disclosure: I work on GCP as a product marketer.)

A GCP product marketer responding to negative commentary on HN within ~20 minutes of it being posted strikes me as automated. EDIT: It is a double standard though, HN readers want access and responses from people on the GCP team but at the same time tinfoil hat subliminal marketing etc.

I would be reading this every 10 minutes for new comments if it was my product up for discussion.
Post reply on HN