AWS Lambda and .zip is a recipe for serverless success • Hacker News Board

Re: AWS Lambda and .zip is a recipe for serverless success

#21

Post by skybrian » Sun, Mar 18, 2018, 4:56 PM UTC

sheetjs wrote: post #16

skybrian wrote: post #11

Earlier quoted context omitted.

You have it backwards. Zip files have a table of contents letting you unzip individual files without decompressing the whole thing, and you don't need to load it all into memory.

The table of contents in a zip file is at the end , not the beginning, so you need random access to the full file if you want to properly jump to the right location. Some zip files use the length feature of local file headers so you can jump ahead in the file without processing the data, but many skip the length so you have to work through the DEFLATE algorithm to find the end of the first file (to find where the sec…

Yes, you can and should use random file access to load things from a zip file. For example, Go's zip package takes an io.ReaderAt interface, not an io.Reader. Using sequential I/O for this would be inefficient.

Re: AWS Lambda and .zip is a recipe for serverless success

#22

Post by faitswulff » Sun, Mar 18, 2018, 5:00 PM UTC

TAForObvReasons wrote: post #18

Does anyone have experience with alternatives to AWS Lambda (like azure functions)? How hard is it to switch between serverless providers?

I played with Firebase (Google Cloud) Functions recently and the developer experience was pretty good. It's like if serverless (https://serverless.com/) had first party support, in the form of the Firebase CLI. Deploying was simple, the Firebase dashboard is nice. That said, I only used it for a side project, so I don't know about any pains scaling. I would definitely use it again for another side project, though.

Re: AWS Lambda and .zip is a recipe for serverless success

#23

Post by keithwhor » Sun, Mar 18, 2018, 5:12 PM UTC

Reading through this comment thread, if anybody is interested in a much more streamlined “serverless” experience and superior API Gateway, check out StdLib [1]. We’ve built a product for getting APIs built and shipped, full stop.

Bonus: .tar.gz format for submitting packages instead of .zip (though our CLI handles it automatically), and a bunch of freebies - everything from automatically generated documentation [2], SDKs, simple local development and testing, and more.

Disclaimer: Am founder.

Disclaimer 2: At the risk of being borderline spammy, we’re hiring aggressively. If you’re passionate about building a future that uses serverless technologies to make software development and APIs more accessible, e-mail me. (And thank you, pre-emptively.)

[1] https://stdlib.com/

[2] https://stdlib.com/@messagebird/lib/numbers/

Re: AWS Lambda and .zip is a recipe for serverless success

#24

Post by Shoothe » Sun, Mar 18, 2018, 5:24 PM UTC

dev1n wrote: post #17

fapjacks wrote: post #9

Cloud commoditized datacenters. People then tightly coupled their applications to cloud providers like AWS. After some time, containers came along and commoditized the cloud providers. Those applications no longer needed to be tightly coupled to a specific cloud provider. This reduced both costs and the risk of being invested in a single cloud provider. Serverless (AWS Lambda) is just the cloud's way of trying to "de…

API Gateway is the literal manifestation of the worst product I have ever had the misfortune of configuring. This is mostly due to AWS's incompetence at delivering any semblance of working documentation around the product. Looking forward to their new "open source" documentation.

I can second this. I wanted to write a simple provisioning tool a while ago that sets up HTTP endpoint on API Gateway and forwards the call to Lambda and the process was extremely complicated.

Re: AWS Lambda and .zip is a recipe for serverless success

#25

Post by repsilat » Sun, Mar 18, 2018, 5:26 PM UTC

alexcroox wrote: post #3

I actually find zipping and uploading manually a PITA. Started using the Serverless framework [1] recently and now I just run `serverless deploy --stage dev --aws-profile profilename` from my repo (which is an npm script). [1] https://serverless.com/

Why would you do it manually? Write a two-line script to do it. Downloading a framework to run an npm script that does the equivalent of running `zip` followed by `scp` (or `aws s3 cp` or whatever) seems crazy to me.

Re: AWS Lambda and .zip is a recipe for serverless success

#26

Post by userbinator » Sun, Mar 18, 2018, 5:34 PM UTC

idbehold wrote: post #5

You know what would be better than a .zip? A .tar.gz file. Zip files cannot be unzipped without having the whole thing in memory. What's funny is that the Lambda environments (at least for Node) do not have the `zip` executable available which means that to zip/unzip something inside Lambda I need one written in javascript which has to be included in the zip file I upload to AWS instead of being able to use the nativ…

Zip files cannot be unzipped without having the whole thing in memory.

Where/how did you come upon that piece of misinformation?

I can still remember using PKZIP to extract archives of over 100MB, on a machine with only 4MB of RAM, many years ago.

Re: AWS Lambda and .zip is a recipe for serverless success

#27

Post by staticassertion » Sun, Mar 18, 2018, 5:42 PM UTC

fapjacks wrote: post #9

Cloud commoditized datacenters. People then tightly coupled their applications to cloud providers like AWS. After some time, containers came along and commoditized the cloud providers. Those applications no longer needed to be tightly coupled to a specific cloud provider. This reduced both costs and the risk of being invested in a single cloud provider. Serverless (AWS Lambda) is just the cloud's way of trying to "de…

I just don't get the "lambda locks you in" thing. You write a server the same way you always would, but like an extra 50LOC exists to make it work on lambda. So if I wanted to deploy elsewhere what would the big deal be?

Lambda also provides a lot more than containerized services, as the article mentions - I no longer have to patch my system, which is a huge operational/ security burden that many companies struggle with. The time to configure API Gateway/ Lambda feels trivial compared to the work involved in maintaining a patched FAAS solution.

Re: AWS Lambda and .zip is a recipe for serverless success

#28

Post by paulgb » Sun, Mar 18, 2018, 5:45 PM UTC

alexcroox wrote: post #3

I actually find zipping and uploading manually a PITA. Started using the Serverless framework [1] recently and now I just run `serverless deploy --stage dev --aws-profile profilename` from my repo (which is an npm script). [1] https://serverless.com/

Another cool thing about Serverless is it can deploy a Python WSGI app (e.g. a Flask app) with little change in the code. It simplifies local testing and makes it easier to leave Lambda and move to dedicated hosting if I ever decide to.

Re: AWS Lambda and .zip is a recipe for serverless success

#29

Post by djhworld » Sun, Mar 18, 2018, 5:50 PM UTC

fapjacks wrote: post #9

Cloud commoditized datacenters. People then tightly coupled their applications to cloud providers like AWS. After some time, containers came along and commoditized the cloud providers. Those applications no longer needed to be tightly coupled to a specific cloud provider. This reduced both costs and the risk of being invested in a single cloud provider. Serverless (AWS Lambda) is just the cloud's way of trying to "de…

AWS Lambda itself isn't vendor lock in.

If you write your Lambda function to put the interaction between the AWS Lambda service at the surface level of your application (i.e. the entry/exit point), and then write your business logic inbetween, you can create a function that is quite easily tranferrable between cloud providers.

The vendor lock in creeps in more around the surrounding services that the cloud provider offers. You can protect yourself from this though by writing good quality hooks that have pluggable backends, e.g. writing/reading to an object store should be abstracted

Re: AWS Lambda and .zip is a recipe for serverless success

#30

Post by dev1n » Sun, Mar 18, 2018, 5:52 PM UTC

Shoothe wrote: post #24

dev1n wrote: post #17

Earlier quoted context omitted.

API Gateway is the literal manifestation of the worst product I have ever had the misfortune of configuring. This is mostly due to AWS's incompetence at delivering any semblance of working documentation around the product. Looking forward to their new "open source" documentation.

I can second this. I wanted to write a simple provisioning tool a while ago that sets up HTTP endpoint on API Gateway and forwards the call to Lambda and the process was extremely complicated.

People (as in I) would pay good money for this tool.