221–230 of 272 posts
Earlier quoted context omitted.
> Uploads to an S3 bucket can trigger a lambda… don’t complicate things. I read this and was getting ready to angrily start beating my keyboard. The best satire is hard to detect.
unfortunately they ruined it at the end with that /s
I think the "local maximum" we've gotten stuck at for application hosting is having a docker container as the canonical environment/deliverable, and injecting secrets when needed. That makes it easy to run and test locally, but still provides most of the benefits I think (infrastructure-as-code setups, reproducibility, etc). Serverless goes a little too far for most applications (in my opinion), but I have to admit s…
Docker is much like microservices. Appropriate for a subset of apps and yet touted as being 'the norm' when it shouldn't be. There are drawbacks to using docker, such as security patching and operational overhead. And if you're blindly putting it into every project, how are you mitigating the risks it introduces? Worse, the big reason it was useful, managing dependency hell, has largely been solved by making develope…
Even if you allow yourself the disadvantage that any non-Docker solution won’t be language-agnostic: how do you get the code bundle to your server? Zip & SFTP? How do you start it? ./start.sh? How do you restart under failure? Systemd? Congrats, you reinvented docker but worse. Want to upgrade a dependency due to a security vulnerability? Do you want to SSH into N replicated VMs and run your Linux distribution specific package update command, or press the little refresh icon in your CI to rebuild a new image then be done?
Docker is the one good thing the ops industry has invented in the last 15 years.
Like the article says, I think serverless has it's place, but I don't think it's for most applications. I can't see myself _ever_ using serverless services as a core part of my application for pretty much any startup, if I can avoid it. The infrastructure overhead is actually worse, IMO. Everything is so platform specific and it's much stranger to test and develop against locally. Each platform has a different way to…
Concerning how IT departments in most non-software companies are, the minimal operational burden is a massive advantage and the productivity is great once you have a team with enough cloud expertise. Think bespoke e-commerce backends, product information management systems or data platforms with teams of a handful of developers taking responsibility for the whole application lifecycle.
The cloud expertise part is a hard requirement though but luckily on AWS the curriculum is somewhat standardized through developer and solutions architect certifications. That helps if you need to do handovers to maintenance or similar.
That said, even as a serverless fan, I immediately thought of containers when the performance requirements came up in the article. Same with the earlier trending "serverless sucks" about video processing on AWS. Most of the time serverless is great but it's definitely not a silver bullet.
Earlier quoted context omitted.
Let me tell you about all the fun I'm having trying to execute my amazon lambda app locally so I can test before deploying...
Localstack makes that pretty easy. Before Localstack I had a pre-staging environment (dev) target I would deploy to. Their free/community offering includes a Lambda environment; you deploy your dev "Lambda" locally to docker, using the same terraform / deploy flow you'd normally use but pointed at a Localstack server which mimics the AWS API instead. Some of their other offerings require you to pay though (Cloudfront…
The front end was entirely in python AWS lambdas, with a message queue that talked to the (slow) GPU backend.
The "lambda tax" was about 100ms on average. given that an average request was around 4 seconds it seemed ok.
Earlier quoted context omitted.
I'm working on a project right now that should be two or three services running on a VM. Instead we have 40+ services spread across a K8s cluster with all the Helm Chart, ArgoCD, CICD pipeline fun that comes with it. It drives me absolutely nuts. But hey if the company wants to pay me to add all that stuff to my resume, I guess I shouldn't complain.
Yeah, the previous company I worked for started with a Django monolith that someone had come in and taken an axe to essentially at random until there were 20 Django "microservices" that had to constantly talk to each other in order to do any operation while trying to maintain consistency across a gigantic k8s cluster. They were even all still connected to the same original database that had served the monolith! Unfor…
Earlier quoted context omitted.
Well i partly agree, and if i would be the one building the counterpart, i prolly had used presigned s3 urls also. In this specific case im getting oldschool file upload request from software that was partly written before the 2000s - noones gonne adjust anything any more. And ye, just accepting giant size uploads is far from good in terms of "Security" like DoS - but ye we talking about stupidly somewhere between 10…
I find with these types of customers it’s always easier to just ask them to save files locally and grant me privileges to read the data. Sometimes they’ll be on Google, Dropbox, Microsoft, etc and I also run a SFTP for this in case they want to move them over to my service. Then I either batch/schedule the processing or give them an endpoint to just to trigger it (/data/import?filename=demo.csv) It’s actually so comm…
Earlier quoted context omitted.
> Easy peasy. /s It actually is though. I don't need to build a custom upload client, I don't need to manage restart behavior, I get automatic restarts if any of the background workers fail, I have a dead letter queue built in to catch unusual failures, I can tie it all together with a common API that's a first class component of the system. Working in the cloud forces you to address the hard problems first. If you a…
> I don't need to build a custom upload client GP said this is an app from the 2000s. For S3 you do need to generate a presigned URL, so you would have to add this logic there somewhere instead of "just having a generic HTTP upload endpoint". Unless the solution is "don't have the problem in the first place" the cloud limitations are just getting in the way here.
I think developers are drowning in tools to make things "easy", when in truth many problems are already easy with the most basic stuff in our tool belt (a compiler, some bash scripts, and some libraries). You can always build up from there. This tooling fetish hurts both companies and developers.
I am not at all a fan of Python but even so any script that you write in Bash would be better in Python (except stuff like installation scripts where you want as few dependencies as possible).
If it's worth saving in a file, it's worth not using Bash.