I couldn't get through the article on mobile, the text was about 3x too large
Improving API Response Times by Migrating from Cloud Functions to Cloud Run
31–40 of 40 posts
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#32I understand why this is technically right, but I don't like it. The promise of Cloud Functions is to give them code and they run it. They should be optimizing how it's run and you shouldn't have to do anything different to optimize performance. I feel like this is a failure on Google's part. Does anyone disagree?
I believe the team responsible for functions, both for GCP, AWS and Azure, do great work in optimizing functions, far, far better work than we could have done on our own.
The problem in our case is that they are architectured to only support a single request. It's not possible to serve a request faster by starting a new anything than serving it from something that is already running.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#33Earlier quoted context omitted.
It is a full-fledged REST API, but we don't do a lot of caching (other than through Cloudlfare) as most of our endpoints either have volatile data or do various operations.
Is it hypertext-driven? When I say REST API I don’t mean just an HTTP-based interface.
It's REST but we don't do HATEOAS. All indicators points to it being a lot of work with little to no reward.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#34Earlier quoted context omitted.
Is it hypertext-driven? When I say REST API I don’t mean just an HTTP-based interface.
It's REST but we don't do HATEOAS. All indicators points to it being a lot of work with little to no reward.
> It's REST but we don't do HATEOAS.
So, it's nothing at all like REST.
> All indicators points to it being a lot of work with little to no reward
That depends what you are doing, but, yes, REST is the wrong solution for lots of problems, but that's not a good reason for calling whatever the non-REST thing is that you’ve chosen to do instead “REST”.
REST is at it's core hypertext as the engine of application state, it is an architectural pattern developed for and as part of the process of refining standards for the WWW; everything else about it is details of how you do HATEOAS. If you aren't doing HATEOAS, you aren't, even approximately, doing REST.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#35I couldn't get through the article on mobile, the text was about 3x too large
Thank you for letting us know!
We set our front-end guy on it and it should be a lot better now.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#36I understand why this is technically right, but I don't like it. The promise of Cloud Functions is to give them code and they run it. They should be optimizing how it's run and you shouldn't have to do anything different to optimize performance. I feel like this is a failure on Google's part. Does anyone disagree?
I believe the team responsible for functions, both for GCP, AWS and Azure, do great work in optimizing functions, far, far better work than we could have done on our own. The problem in our case is that they are architectured to only support a single request. It's not possible to serve a request faster by starting a new anything than serving it from something that is already running.
Don't you think it's bad design to only allow one request per function? We shouldn't have to migrate to different cloud services to be able to unlock a common pattern. Optimizing the function to allow handling multiple requests while resources are available should be something that the functions infrastructure auto-magically does.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#37I understand why this is technically right, but I don't like it. The promise of Cloud Functions is to give them code and they run it. They should be optimizing how it's run and you shouldn't have to do anything different to optimize performance. I feel like this is a failure on Google's part. Does anyone disagree?
I agree wholeheartedly. "Best practices" suggested by AWS, GCP are to architect these services with one function per endpoint, with API gateway handling the routing. Sounds good but if you need reasonable tail-end response latencies in the event of a cold start, you need to forget all about the serverless abstraction and go back to a single container with all of your functionality.
On the other hand I can see the appeal of one instance, one request. I don't know if the answer is to enable (optional) request parallelism or to optimize cold start times down further, but OP's requirements feel like they are way too modest to basically force them out of the serverless paradigm.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#38You mentioned that Cloud Run allowed you to reserve a number of always-running instances, but that feature is available for Cloud Functions as well [0]. [0] https://cloud.google.com/functions/docs/configuring/min-inst...
Hi! Author here. Functions still only run a single request at a time, while Cloud Run can run as many as the container can handle. We want to build for multiple smaller requests, so when we get ~10k requests incoming in a short timespan it's a lot more useful to have Cloud Run. I may be mistaken, but I believe cold start is roughly the same for both.
Obviously switching away from Cloud Functions does away with the architectural limitation of processing requests serially, but from my (limited) understanding of Node.js internals, Express applications are single-threaded anyway.
Do you have any idea of where your performance gains are coming from? Is there a lot of async, non-blocking waiting happening in the process of fulfilling requests against your API?
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#39Earlier quoted context omitted.
Hi! Author here. Functions still only run a single request at a time, while Cloud Run can run as many as the container can handle. We want to build for multiple smaller requests, so when we get ~10k requests incoming in a short timespan it's a lot more useful to have Cloud Run. I may be mistaken, but I believe cold start is roughly the same for both.
Obviously switching away from Cloud Functions does away with the architectural limitation of processing requests serially, but from my (limited) understanding of Node.js internals, Express applications are single-threaded anyway. Do you have any idea of where your performance gains are coming from? Is there a lot of async, non-blocking waiting happening in the process of fulfilling requests against your API?
The NodeJS application is single-threaded, but the Node EventLoop can handle a huge amount of parallel requests, especially since most of the time per request is spent waiting for calls to the DB or external APIs.
I believe the majority of our performance gain is from having far fewer cold starts during large traffic loads.
Re: Improving API Response Times by Migrating from Cloud Functions to Cloud Run
#40Earlier quoted context omitted.
I believe the team responsible for functions, both for GCP, AWS and Azure, do great work in optimizing functions, far, far better work than we could have done on our own. The problem in our case is that they are architectured to only support a single request. It's not possible to serve a request faster by starting a new anything than serving it from something that is already running.
Don't you think it's bad design to only allow one request per function? We shouldn't have to migrate to different cloud services to be able to unlock a common pattern. Optimizing the function to allow handling multiple requests while resources are available should be something that the functions infrastructure auto-magically does.
I think it's a decent architectural decision.
It's a lot easier to develop solid code that handles a single request than code than handles multiple requests in parallel.
Functions are used for a lot more than HTTP, we use them for a lot of DB event handling such as post-processing, in which case cold starts does not really matter.
Maybe we'll one day get to a point where the cloud service knows the best way to run our code?