Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure Functions – Significant Improvements in HTTP Trigger Scaling

azurefromthetrenches.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#21

Post by dillondoyle »

doh wrote: post #17
dillondoyle wrote: post #16

I'm in the middle of trying to move a very simple ETL-light script to Google (want to use bigquery over redshift). The idea was to use Cloud Functions. I have ran into scale problems very fast at Google and now am having to use App Engine and add more complication which I don't have personal engineering skill/capacity. Google support first bumped me up to 12000 max queries per 100 seconds and said that's the limit, b…

I don't know your flow, but how about using a queue system to prevent the bursts of data? We're internally using pubsub (and nats) sending billions of messages every day. We don't use the push mode as Spotify [0] but pull from the queue instead which allows us to run on our pace. We do write into bigquery (and citus) and each components is doing it on their own pace based on what they're capable of process at the mom…

Pub Sub is the same quota hits (or +1 /connection). I could use pub sub but I still have to accept the incoming POST before publishing a message.

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#22

Post by dillondoyle »

dillondoyle wrote: post #21
doh wrote: post #17

Earlier quoted context omitted.

I don't know your flow, but how about using a queue system to prevent the bursts of data? We're internally using pubsub (and nats) sending billions of messages every day. We don't use the push mode as Spotify [0] but pull from the queue instead which allows us to run on our pace. We do write into bigquery (and citus) and each components is doing it on their own pace based on what they're capable of process at the mom…

Pub Sub is the same quota hits (or +1 /connection). I could use pub sub but I still have to accept the incoming POST before publishing a message.

To be clearer it's same quota problem using cloud functions. Moving to something without the same quota/scale problems like Compute Engine or just DIY containers might make sense but then I'm back to a complicated system (Snowplow) that we have in AWS that's too complicated for me alone

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#23

Post by doh »

dillondoyle wrote: post #22
dillondoyle wrote: post #21

Earlier quoted context omitted.

Pub Sub is the same quota hits (or +1 /connection). I could use pub sub but I still have to accept the incoming POST before publishing a message.

To be clearer it's same quota problem using cloud functions. Moving to something without the same quota/scale problems like Compute Engine or just DIY containers might make sense but then I'm back to a complicated system (Snowplow) that we have in AWS that's too complicated for me alone

Understood. I think there is a bit simpler setup that you could utilize, but it does require a custom deployment on GCE.

Request -> load balancer -> auto scaled pubsub writer -> pubsub -> client (write to BQ, ...)

If it doesn't have to be a post request, then you could just write into pubsub directly. Pubsub has a wide support of libraries [0] so you could do it from any language. Pubsub also scales well. We're sending in around 300k messages per sec and have very few problems with it.

[0] https://cloud.google.com/pubsub/docs/reference/libraries

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#24

Post by viggity »

evfanknitram wrote: post #4
joncrane wrote: post #3

Is this the Azure equivalent of AWS Lambda? What factors other than some specific affinity to Microsoft makes a company choose Azure over AWS?

Yes it is. What do you mean by Microsoft-specific stuff? I run a bunch of stuff there. When I started using these things their PaaS offerings were much more mature than the AWS counterpart (since they focused more on IaaS).

We run all PaaS except for two very special case VMs. It is an effin' delight not having to worry about security, maintenance, updates. I just deploy my code (app services) and query my data (azure table storage, azure sql). Shit just works. Every time I need to deal with the VMs I cringe.

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#25

Post by cobookman »

doh wrote: post #23
dillondoyle wrote: post #22

Earlier quoted context omitted.

To be clearer it's same quota problem using cloud functions. Moving to something without the same quota/scale problems like Compute Engine or just DIY containers might make sense but then I'm back to a complicated system (Snowplow) that we have in AWS that's too complicated for me alone

Understood. I think there is a bit simpler setup that you could utilize, but it does require a custom deployment on GCE. Request -> load balancer -> auto scaled pubsub writer -> pubsub -> client (write to BQ, ...) If it doesn't have to be a post request, then you could just write into pubsub directly. Pubsub has a wide support of libraries [0] so you could do it from any language. Pubsub also scales well. We're sendi…

You can make it simpler.

Request -> gae -> pubsub -> dataflow to bq template.

https://cloud.google.com/dataflow/docs/templates/provided-te...

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#26

Post by CyanLite2 »

joncrane wrote: post #3

Is this the Azure equivalent of AWS Lambda? What factors other than some specific affinity to Microsoft makes a company choose Azure over AWS?

Cheaper costs, CosmosDB, better AI integration, compliance, enterprise integration, more regions, etc... etc...

CosmosDB was the difference for us.

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#27

Post by dillondoyle »

doh wrote: post #23
dillondoyle wrote: post #22

Earlier quoted context omitted.

To be clearer it's same quota problem using cloud functions. Moving to something without the same quota/scale problems like Compute Engine or just DIY containers might make sense but then I'm back to a complicated system (Snowplow) that we have in AWS that's too complicated for me alone

Understood. I think there is a bit simpler setup that you could utilize, but it does require a custom deployment on GCE. Request -> load balancer -> auto scaled pubsub writer -> pubsub -> client (write to BQ, ...) If it doesn't have to be a post request, then you could just write into pubsub directly. Pubsub has a wide support of libraries [0] so you could do it from any language. Pubsub also scales well. We're sendi…

Yup that's pretty much where I'm going although without pub/sub ideally. it's marketing analytics so if a few inserts fail it doesn't matter. But might not be able to get the 0 -> 1000s instant working without Google's abstraction messaging in front (similar setup on AWS w. kinesis but it's too complicated for me)

I did think about doing something like transforming the POST in AWS Lambda (the emitter data is hosted there too) and transform into something I can ingest direct into pub/sub.

I can't control the emitting data. if I did it would be a much easier problem, each POST request contains very little data if there were multiple entries per single POST it would solve most of my problems, instead of sending 5k tiny JSON posts /sec lol

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#28

Post by daxfohl »

dillondoyle wrote: post #16

I'm in the middle of trying to move a very simple ETL-light script to Google (want to use bigquery over redshift). The idea was to use Cloud Functions. I have ran into scale problems very fast at Google and now am having to use App Engine and add more complication which I don't have personal engineering skill/capacity. Google support first bumped me up to 12000 max queries per 100 seconds and said that's the limit, b…

My feeling is that Google cloud will always be hit and miss because their biggest customer by far is Google. So they'll do unbelievable scale on the stuff they use, and just passable on everything else.

It's also why I think that Facebook never joined the rat race. Most of what they do has no relevance to the average company.

Amazon, an online retailer, was really the most natural place to start an outsourcing of banal technologies for the largest market segment. And has grown from there.

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#29

Post by helpfulgoogler »

dillondoyle wrote: post #16

I'm in the middle of trying to move a very simple ETL-light script to Google (want to use bigquery over redshift). The idea was to use Cloud Functions. I have ran into scale problems very fast at Google and now am having to use App Engine and add more complication which I don't have personal engineering skill/capacity. Google support first bumped me up to 12000 max queries per 100 seconds and said that's the limit, b…

According to [1], the standard quota is 1,000,000 invocations per 100 seconds and can be increased. Maybe try asking again for increased quota? If the documentation is incorrect, please file an issue [2].

Counter-intuitively, ending the response before processing the data may actually be hurting your p99 response time. According to [3], instances which are not currently handling a request get very little CPU and generally don't make any progress. This means that this work will still be waiting to be done when the next request comes in and can slow it down. You may even end up with an instance trying to process several requests' data and also trying to handle another request on top of that. This last request is going to get an overloaded instance and likely be very slow. Further, the function isn't guaranteed to ever be run again, so that deferred work might not even ever happen.

It probably will cause you to use more of your invocations per 100 seconds quota, but you really shouldn't start background processing tasks in a Google Cloud Function that continue after completing a request.

[1] https://cloud.google.com/functions/quotas [2] https://issuetracker.google.com/issues/new?component=187195&... [3] https://cloud.google.com/functions/docs/bestpractices/tips

Re: Azure Functions – Significant Improvements in HTTP Trigger Scaling

#30

Post by gaius »

joncrane wrote: post #3

Is this the Azure equivalent of AWS Lambda? What factors other than some specific affinity to Microsoft makes a company choose Azure over AWS?

What factors other than some specific affinity to Microsoft makes a company choose Azure over AWS?

More locations, tho' every major cloud is opening more all the time. A much better hybrid story, tho' this may change once (if) AWS get their alliance with VMware working properly. Better layered applications (X-as-a-service) but that might be a matter of taste - AWS is more about giving you building blocks to build your things, Azure has the blocks but is better at giving you integrated "things" out of the box (e.g. IaaS vs PaaS). Per-second billing, generally better budget control.

I've a bit of experience with both Azure and AWS. There's nothing I could build in one that I couldn't build in the other, tho' depending on what that is, it might be more effort in one of them. For my own personal stuff, I use Azure, it's more fun :-)

Post reply on HN